DTS-HD to LPCM Lossless

[deleted user]

Pretty sure this was already done by a few people in the past and it was indeed bit-perfect. Most in the community only have a rather old version of the DTS suite to their disposal though, so newer formats can't be tested sadly.

[schorman]

One of the big differences between the two methods is how much more typing you need to do to get ffmpeg to split your channels into separate files! With eac3to you just use output.wavs instead of output.wav.

compare:

>eac3to input.dtshd output.wavs

>ffmpeg -i input.dtshd -filter_complex "channelsplit=channel_layout=5.1[FL][FR][FC][LFE][BL][BR]" -map "[FL]" -c:a pcm_s24le output_L.wav -map "[FR]" -c:a pcm_s24le output_R.wav -map "[FC]" -c:a pcm_s24le output_C.wav -map "[LFE]" -c:a pcm_s24le output_LFE.wav -map "[BL]" -c:a pcm_s24le output_SL.wav -map "[BR]" -c:a pcm_s24le output_SR.wav

[deleted user]

Damn! Thanks for that mate, that is actually very helpful. Yes, splitting into channels was always a major issue with ffmpeg. But I'd probably still opt for converting to flac with ffmpeg first (or to wav/w64) and then split with eac3to, if it supports those formats.

[crampedmisfit1990]

Been a long time since I've replied to this thread...

Have a question, Using ffmpeg, If I convert a 24 bit DTS-HD to .Wav and the bitrate changes, is that normal? Example: The DTS-HD track is 1536 kbps and when converted to .Wav it is now 2304 kbps.

[pipefan413]

Been a long time since I've replied to this thread...

Have a question, Using ffmpeg, If I convert a 24 bit DTS-HD to .Wav and the bitrate changes, is that normal? Example: The DTS-HD track is 1536 kbps and when converted to .Wav it is now 2304 kbps.

1536 kbps is highly unlikely to be the bitrate of a lossless DTS-HD Master Audio file that contains 2304 kbps PCM samples. What it is more likely to be is the interpreted bitrate of the lossy DTS core (it's actually more like 1509 kbps I believe, at the very least according to the official encoder). Where did you get the 1536 kbps figure from?

I have an idea: just encode a PCM track in DTS-HD MA, then decode it using various decoder, and compare the result; they should be bit-by-bit identical - if not, mention here which one gives a different result!

I just saw this thread but I've been doing tests lately to verify my encodes and the results have been... odd.

First point, which isn't odd at all, is that the files having been through the encoder are extended to fill the nearest whole DTS audio frame because... well, obviously that's just how DTS works, it organises the audio into frames of a designated length (at 48 kHz that's 512 samples). It just means when you decode again, it'll keep those added 00 bytes on the end, so it won't be bit perfect due to the extra zeroes. But you can literally just delete them in a hex editor and it'll match, so that's a non-issue as long as you know about it.

Secondly, I'm sure this is widely known (at the very least by @schorman and @Chewtobacca), but there are also zero bytes inserted at the very beginning of the bitstream by the DTS Suite encoder, which at 48 kHz amount to 1024 samples of silence. I did a fairly comprehensive breakdown of the structure of the entire DTS Suite header on another forum but the short version of the bit pertaining to what is relevant here is that part of the header data includes an instruction to apply a "codec delay" of 1024 samples, meaning that those 1024 samples of added silence are not actually included in the output audio if you author them in pro software. However, for most purposes, it would be better to just remove those 1024 samples with a negative delay of "-21ms" in eac3to ("eac3to input.file output.file -21ms").

Now... here's the weird bit.

With 4 encodes done in the official encoder for one particular film (I'm not going to mention which one or who made the synced files because it's probably irrelevant anyway, it just illustrates the point) I found that only 2/4 were bit-perfect matches to the input PCM after going through the Suite and coming out as DTS-HD MA, being converted back to PCM in eac3to, then having the extra zero bytes removed from the start (the 1024 samples) and end (the extra added to make up a whole DTS frame).


ENCODE 1: Dolby Surround matrixed 2.0 PCM -> 2.0 (L+R) DTS-HD MA -> 2.0 PCM

After removing the extra 00 bytes from the start and end, this is a bit-perfect match to the source.


ENCODE 2: Dolby Surround matrixed 2.0 PCM -> 2.0 (Lt+Rt) DTS-HD MA -> 2.0 PCM

After removing the extra 00 bytes from the start and end, this is a bit-perfect match to the source.


ENCODE 3: 1.0 PCM -> 1.0 DTS-HD MA -> 1.0 PCM

This looks different from source, with previously zero bytes in between bitstream data now being filled with something slightly different. It isn't consistent but it looks like it's basically somewhere in between 00 and whatever the next byte is (as if it's sort of smoothing out the waveform or something). Apart from that it's much the same, and it doesn't sound different, as long as the extra zero bytes are removed as before.


ENCODE 4: 5.1 PCM -> 5.1 DTS-HD MA -> 5.1 PCM

Again, different. Similar to the mono one except it doesn't just seem to be 00 bytes being replaced with slightly different data, although the difference in bytes is about the same (as in a 00 in the source will be filled with something that's very nearly 00 in the output and so on).


I'm really confused about what might cause that, since they were all encoded exactly the same way, with everything being done in either eac3to (conversions), the DTS-HD MA Suite (DTS encoding), or a hex editor (removing 00 bytes from the end). I'm trying to investigate with further testing however I don't have enough disk space to do so easily and keep having to delete the intermediate files (5.1 PCM gets big fast, especially when you're keeping it as both separate channels and a combined file for both input and output to compare).

I think my next step is to try decoding the .dtshd files with ffmpeg instead of eac3to to see if there's any difference at all...

EDIT: I think that's exactly the problem... eac3to is not always decoding the .dtshd 100% accurately! When I decode the .dtshd output to .flac in ffmpeg, then use eac3to to convert the .flac back to .pcm, the raw PCM I fed to the encoder and the raw PCM out the other end now match (after I delete the extra 00 bytes at each end). Bit-perfect. Good! So the problem isn't the encoder, it's eac3to's not-quite-accurate .dtshd decoding, apparently. That's a significant discovery for me, since it completetly changes my workflow to rely slightly less on eac3to (I'll just have to go via ffmpeg .flac as an intermediate step from now on, and redo a crapload of conversions for historic files).

This presents a very big concern for any .dtshd files shared after doing the -21ms eac3to adjustment to remove the 1024 samples of silence that the DTS Suite encoder adds to the start of the bitstream, because (and yes, I've just verified this repeatedly) simply applying the -21ms adjustment introduces the imperfect behaviour of eac3to's DTS (DCA) decoding. That's a bit of a nightmare unless you have another method to remove the silence, such as an equivalent adjustment in ffmpeg instead (which I don't yet know how to do) or straight up hex editing (which I can probably figure out since it appears to involve the same number of bytes each time, I'll need to do further testing to be certain of that though and it's nearly 4am).

Going to get really stuck into testing this tomorrow but I'm confident that eac3to has a problem with .dtshd decoding that ffmpeg doesn't!

[Chewtobacca]

Secondly, I'm sure this is widely known (at the very least by schorman and Chewtobacca), but there are also zero bytes inserted at the very beginning of the bitstream by the DTS Suite encoder, which at 48 kHz amount to 1024 samples of silence.

Yes, I flagged this up to schorman, and to spoRv, who posted about it here. For eac3to, try replacing the built-in dcadec with this one.

@crampedmisfit1990
Yes, 2304 kbps is normal. Presumably, you are decoding a DTS-HD MA 2.0 track, and the 1536 kbps that you cited is that of the core.

[pipefan413]

Yes, I flagged this up to schorman, and to spoRv, who posted about it here.

I know, I was just acknowledging it as part of what needs to be reverted to get back to a bit perfect match.

For eac3to, try replacing the built-in dcadec with this one.

I'll check that later, thanks. Although I already did upgrade the built in version (I can't remember what to) which got rid of the "XLL output not lossless" message that's usually there. I'd not be surprised if it's this version I already have but I'm not sure without checking.

Since the above post I've also found that ffmpeg also couldn't decode some files, at least one in particular, but this time eac3to could. I'm guessing if I mux then demux it, ffmpeg might manage, but but I'm not at my computer just now. So this is a pain in the butt. I think on this file eac3to actually does look like it's accurate but I don't have the PCM that went inro the DTS-HD MA for that specific file so it may not be (but it has lots of pure 00 bytes though, which seemed to be the thing eac3to wasn't very good at reproducing from some encoded .dtshd).

[pipefan413]

Right. So!

For the files ffmpeg doesn't want to decode ("unknown error"), workarkunds that work are:

option 1: mux them to .mka in MKVToolNix to strip the DTS Suite header, *then* decode with ffmpeg (bit perfect)

option 2: straight up delete *only* the header in a hex editor (bit perfect as long as you know exactly what you're doing)


Unfortunately I have less good news about this quirk of libdcadec. I expect MKVToolNix also uses libdcadec, because when I tried doing the -21ms adjustment in that instead of eac3to, the results were exactly the same (inaccurate bitstream for the problem files). I also replaced my existing version of libdcadec as used by eac3to with the one @Chewtobacca linked above, but that didn't change anything.

I think the obvious solution is probably to figure out how to trim off those 1024 samples of silence from the start using ffmpeg instead of eac3to/MKVToolNix in order to take advantage of its more accurate decoder, but I haven't yet worked out how to do that successfully. A look online suggests that it should be possible with the ffmpeg command "-itsoffset" followed by the time, and the documentation suggests a negative offset is possible so it should theoretically be "ffmpeg -itsoffset -21ms -i dts_suite_output.dtshd sync_corrected.dtshd" but when I do that and decode to .flac then raw .pcm it turns out to have just completely ignored the offset. Apparently that command used to be borked in ffmpeg then got fixed but I'm using the latest ffmpeg and still can't get it to work. Also, even that only works if I'm decoding to .flac, but if I try to apply the offset then output back to .dtshd again, it just errors. The whole point of me doing this is to try to apply the offset to a .dtshd stream from the encoder but keep it encoded as .dtshd.

Anybody more familiar with ffmpeg who knows how to apply a negative offset to .dtshd audio?

[Chewtobacca]

I also replaced my existing version of libdcadec as used by eac3to with the one @Chewtobacca linked above, but that didn't change anything.

Try Arcsoft or Sonic. It's a shame that the libdcadec development was (as far as I know) swallowed up into ffmpeg, because madshi confirmed ages ago that eac3to doesn't use a special version. In other words, theoretically someone could compile a new libdcadec.dll, and we could just drop it in.

[pipefan413]

I also replaced my existing version of libdcadec as used by eac3to with the one @Chewtobacca linked above, but that didn't change anything.

Try Arcsoft or Sonic.  It's a shame that the libdcadec development was (as far as I know) swallowed up into ffmpeg, because madshi confirmed ages ago that eac3to doesn't use a special version.  In other words, theoretically someone could compile a new libdcadec.dll, and we could just drop it in.

That's my next move precisely, yep. Sonic apparently has some limitations (e.g. doesn't do 7.1) and I so far haven't managed to find a working copy of the Arcsoft decoder. If anybody might be in a position to help with that, please gimme a PM.