Laserdisc PCM Capture Guide

[zoidberg]

Probably something to do with the end frames, like I said closing eac3to preserves the wav file before it gets a chance to purge it

[pipefan413]

Hmm, maybe I spoke too soon...
eac3to didn't produce any errors when converting the .dts files for each side to a separate .dts file (eac3to input.dts output.dts), but getting libDcaDec Invalid Bitstream Format errors toward the ends of the files when trying to convert them to separate .wav files (eac3to input.dts output.wavs).

I personally don't need to convert the files and don't have any project plans for Hard Target, but does this mean the capture is corrupt?  The .dts files at least seem to play fine during a spot check in foobar.

Just captured my first DTS disc and looked at the hex to figure out why this is. A while ago, before I'd actually seen any DTS captures from LaserDisc with my own eyes, I suspected that...

Out of curiosity, might this just be because the ends are not rounded to whole DTS frames (512 samples at 48 kHz for example)? If so, could be fixed in hex editor or something easily enough, in theory.

Turns out I was more or less correct: the final frame was invalid, having only been partially captured. It can be fixed by patching in a missing piece of data that can be copied from elsewhere in the file, or you can just wipe out the entire frame and it basically won't matter (delete it in a hex editor and the file decodes correctly). If we're talking about a frame from the very very end of a side, it should be silence or near silence anyway (by "near silence" I mean silence but with some kind of tiny bit of background noise floor from  the original recording process). Or, of course, you could just do what @zoidberg suggests (although this will leave a dodgy partial frame in your .dts file, it's probably going to end up getting deleted if/when you join up the side changes anyway).

As soon as I saw a DTS file after I'd thrown it through DTS Parser, I immediately realised what was going on and got a bit excited because I immediately had a very good idea of what might be going on, then I did a bit of digging to check. The only reason I worked this out straight away was that I had previously spent a good bit of time a long time ago picking apart not just the DTS header but also the DTS-HD container file structure as well, so when I looked at the hex data it all fell into place immediately; I definitely wouldn't have known what the hell I was looking at otherwise.

This isn't just about the problem at the very end of the file, but also explains why @schorman's header fix works as it does.


DTS codec structure (frames, blocks, and samples)

DTS sorts its audio samples into "frames", which are further organised into "blocks" of a set number of audio samples. For 48 kHz audio (e.g. DVD/BD), each normal frame contains 16 blocks, each containing 32 audio samples (which is why a DTS frame from DVD or Blu-ray contains 512 samples: 16 blocks x 32 samples = 512). For LaserDisc, which has its digital audio output set at 44.1 kHz, each frame contains 32 blocks of 32 samples (32 x 32 = 1024 samples per frame).

The headers written by DTS Parser look like this:

7F FE 80 01 FC 7C E0 02 63 A0 0D 3A 80 09

... but they should look like this:

7F FE 80 01 FC 7C DF F2 62 C0 0D 3A 80 09

You can fix that in a few seconds with a simple replace all operation. But what is that actually doing? And is part of it possibly relevant to what's causing decoding problems with the DTS Parser output? (Spoiler: yes.)

To understand what's going on in the hex code it needs broken down to the binary level. Only 5 bytes are relevant here, so I'll only cover those ones:

7C = 01111100
E0 = 11100000
02 = 00000010
63 = 01100011
A0 = 10100000

... needs to be changed to

7C = 01111100 (unchanged)
DF = 11011111
F2 = 11110010
62 = 01100010
C0 = 11000000

These 5 bytes contain the following metadata, most - but not all - of which appears to be parsed correctly by DTS Parser:

ftype:
Frame type, being normal ("1") or termination ("0").

short:
The number of samples by which the termination frame falls short of the normal length of 32 samples; for normal (non-termination) frames, the value is 31 (counting from 0 as digital systems do, that's 32).

cpf:
CRC flag indicator (if there is a CRC present in the header, the value will be "1"; for LaserDiscs, it's "0").

nblks:
Number of blocks (of samples) per frame. For 48 kHz DTS from DVD, it's 16 blocks of 32 samples, so nblks = 15 (again, counting from 0, that's 16). For 44.1 kHz DTS from LaserDisc, it's 32 blocks of 32 samples (32*32 = 1024 samples per frame), so nblks = 31.

fsize:
Number of bytes per frame (frame size). For a 48 kHz 1536 kbps stream, this would be 2013 bytes, represented as "00011111011100" in binary (2012 in decimal, or 7DC in hex), but since LaserDiscs contain 44.1 kHz 1235 kbps streams instead, it should be "00110111111111" (3583 in dec, DFF in hex) because each frame is 3584 bytes long.

amode:
Audio channel arrangement. For all DTS LaserDiscs this is the classic 5.1 layout, but note that the LFE channel is not counted here because of how the codec derives it from the file. L+C+R+Ls+Rs is represented by the decimal value "09", which in binary is "001001".

sfreq:
Sample frequency. For LaserDiscs this is always 44.1 kHz, represented by the DTS codec by the dec/hex value "8" which is "1000" in binary (for 48 kHz from DVD/BD it's the hex value "D" which is "1101" in binary).

rate:
Bitrate, which for LaserDiscs is always 1234 kbps, represented by the decimal value "22" which in binary is "10110". For some reason I haven't yet had time to investigate,  DTS parser thinks it's 11101 (29), which is partly why if you try to play back a raw capture straight out of DTS parser it's all garbled and sped up.

mix / dynf / timef / auxf / hdcd:
These are all 1-bit flags that I think I've talked a bit about before when I was researching the header for picking apart DTS-HD specifically, but the main thing here is that they're all 0 for LaserDiscs and therefore can be left alone.


To summarise all that using LaserDisc numbers:

- All DTS frames except the final one in the bitstream are "normal" frames and the last one is a "termination" frame.

- Every normal frame contains 32 blocks of 32 audio samples (32 x 32 = 1024 samples); the termination frame may contain fewer than 32 blocks.

- 32 blocks of 32 samples at a sample rate of 44.1 kHz (44100 samples per second) and a bitrate of 1234 kbps ends up with a frame size of 3584 bytes. DTS Parser correctly reports "Format" as 44.1 kHz and "FrameSize" as 3584 bytes, so everything seems to be in order, and yet the output doesn't work properly (plays back in MPC too fast and garbled).


So what's going wrong, then?

Straight out of DTS Parser, the values are written as follows:

ftype = 1 (bin)
short = 11111 (bin) = 31 (dec), meaning each block contains 32 samples
nblks = 0011111 (bin) = 31 (dec), meaning 32 blocks
fsize = 00111000000000 (bin) = 3584 (dec)
amode = 001001 (bin) = 9 (dec/hex)
sfreq = 1000 (bin) = 8 (dec/hex)
rate = 11101 (bin) = 1D (hex) = 29 (dec)

The most obvious issue can probably be spotted from what I've already told you, if you look closely and remember that values in data terms are counted from 0. Look at fsize. Remember I said that each frame is 3584 bytes long? Count from 0 and 3584 becomes 3583. But DTS Parser actually writes fsize into the output file as 3584! This is kinda weird because only this value appears to suffer from this error; all the others are indeed counted from 0, but this one's counted from 1 for some reason. Whoops!

But what else? Well, this one's a bit more obfuscated because the values are coded. I'm not 100% sure exactly why, but DTS Parser is writing the bitrate wrong as well. This is why the files play back too fast. If you look at the information reported in the DTS Parser GUI, the bitrate is identified as "open", presumably pretty much meaning "variable" or "undefined". The actual* bitrate for LaserDisc DTS is 1234 kbps, which is coded into the codec as the binary value "10110" = hex "16" = dec "22". As a result of not being able to identify this from the raw data for some reason or another, DTS Parser writes it as "11101" = hex "1D" = dec "29", which overestimates the bitrate of the stream. I'm guessing this is why the output files play back too fast. This is sheer conjecture, but I'm wondering if maybe this is a knock-on effect of getting the frame size wrong; if it thinks frames are 1 byte bigger than they are then it maybe sees that each frame appears to have a bizarre bitrate (as a result of overestimating the frame size) and can't find an appropriate value in a lookup table or whatever, so it just writes it as "open", which is wrong. (Note: I haven't actually picked the DTS Parser application's code apart, so I'm only making deductions here from its behaviour.)

* Put (very) simply, DTS bitrate isn't quite what it's usually written as... so for example, 1536 kbps targeted bitrate ends up actually being 1509.75 kbps. The DTS-HD Master Audio Suite writes these in its GUI as rounded down actual values rather than the "targeted" values that are more commonly used online etc. (so they don't say 1536 kbps, they call it 1509 kbps). But I'll let you venture down that rabbit hole yourself, if you so desire.

OK, so fsize and rate are both wrong. That explains why changing those particular bytes in all the headers of the DTS Parser output from "E0 02 63 A0" to "DF F2 62 C0" makes the majority of the file decode correctly.

But what about that decoding crash on the final frame of the file?

Well, remember what I said above about there being "normal" frames and "termination" frames, and how termination frames don't necessarily contain the 32 full blocks that all normal frames do? That being the case... why does *every* header get registered and written as a normal frame, including the last one?

If you find and replace "7F FE 80 01 FC 7C E0 02 63 A0 0D 3A 80 09" with "7F FE 80 01 FC 7C DF F2 62 C0 0D 3A 80 09" then you'd only be rewriting the normal frames and leaving the termination frame alone. But certainly in the case of my capture of TOY STORY, there *was* no termination frame at the end of the first side, DTS Parser just wrote it as another normal frame instead, but the end was packed out with zeroes.



But there's something weird about the last frame apart from that. Although it's the right length (3584 bytes)...



... it ends in a big block of zeroes. But compare that to the previous frames, which are also silent, and...



... they all have data in the last 62 bytes (or, strictly speaking, the last 15 are still empty but the preceding 47 aren't). And all of those trailing silent frames have the *exact same* data in that spot. If you look at what the data actually looks like, it appears to be the final "block" of the DTS frame, which seems to be why decoders crap out when they can't find it: the header is telling them to locate and decode 32 blocks, but there are only 31 blocks and the last one is longer than it's supposed to be.

So from what @williarob initially quite logically surmised...

There must be some sort of EOF marker in there, so DTS Parser just stops parsing at that point, it won't look for another BOF after that point.

... that might *theoretically* indeed be correct if the player successfully reproduced the entire digital bitstream right up to the very last bit, as you'd presumably capture everything including that very last silent termination frame (with the header to match), but if the player stops even a teensy bit before then, you miss that and probably part of the last frame you *do* capture. As a result, instead of ending in a valid termination frame, the data stream instead ends with *part* of a normal frame, which - due to being a fragment of a whole frame - cannot be decoded.

When I consider what I've seen with other (non-DTS) captures, this makes perfect sense. At the start and the end of a side, the player has to determine the precise moment to start and stop reading, and it isn't absolutely 100% precise at doing that every single time. So I could do 3 captures of the same side of the same disc from the same player and get slightly different amounts of data at the start/end of each one, but the rest will be identical. This is why there are slight variations between captures but the rest of the data is 100% identical, as long as everything else works correctly.

So anyway, what happens if you patch that data into the end of the last frame, overwriting those zeroes? It still won't end the stream on a termination frame, but it will at least give it the correct number of blocks so that it actually contains 32 blocks like the header says, right?




Whaddayaknow? It now decodes properly!

But this is just a bodge, right? Patching from a previous silent frame doesn't actually reconstruct data we didn't capture off the actual disc. So surely the only correct way to proceed would be to do a fresh capture but this time cross your fingers and hope that you squeeze every last bit out of the player.

Well, I tried that a few times. Each time I attempted a capture of the end of the disc, I still didn't see a termination frame, or anything resembling one. I got *less* in my subsequent captures than the original one, and then when I eventually managed to get one that had a bit more data in it, it was just more zeroes.

So I suspect the reason @williarob's captures don't parse through the side change is not necessarily that there is a "stop" flag (in the form of a termination frame in the DTS stream), but rather than the raw data does not "line up" correctly. The player stops playing, probably in the middle of a DTS frame (so a partial chunk of the last frame it *does* play is recorded, but not the full thing), and the rest is just zeroes as it would be if you had been recording PCM and that dropped to null input. DTS Parser (mostly) correctly writes all the data from each frame it finds in the file, until it runs out of whole frames, whereupon it realises that there are no more frame headers where they should be (so it's come to the end point in a 3584-byte frame and doesn't see another frame header nearby) and stops parsing the file. But if you split the file around this point and feed both of the separate files to DTS Parser, well then it does exactly the same thing as before with the first bit, then knows to ignore all the zero bytes at the beginning of what is now your second file (for side 2) until it finds the first frame header, which it then uses to start parsing the bitstream as usual.


Here's a visual breakdown of all the relevant bits of the DTS frame header, showing how the hex code breaks down to binary and which sections pertain to which attributes. I've added examples to show what the frame header looks like straight off a LaserDisc and parsed by DTS Parser, then after it's been fixed as described in detail above:

https://docs.google.com/spreadsheets/d/1...Yv1WPO1MrQ




Does... any of this really matter in practical terms?

Well, no, not very much at all for most people, I expect. I just find this stuff interesting because I like pulling stuff apart. The main thing though is knowing how to deal with it. So what will get you around the problem?

1. Just do what @zoidberg said: decode the data you've captured, but get your trigger finger ready so that you can quickly shut eac3to before it gets time to delete the file it created. I found that this didn't work well for me because I didn't want to sit staring at the CLI waiting for it to finish then suddenly panic and try to close the window before it reset to a new command prompt and deleted the file. Also, that means that your newly created .dts file will contain a faulty final frame, which I'm not keen on.

2. Delete the partial frame from the end of the parsed file, leaving only complete frames. (Just search backwards from the end for "7F FE 80 01" and delete everything from that "7F" onwards, since that's the beginning of the header for the last frame.) I think this is probably the best solution imo.

3. Patch the missing end of the last captured frame from previous wholly captured frames, if they're totally zeroed frames (as in the example above).

4. Theoretically, you could try to capture it again in an attempt to grab every single last bit from the end of the side this time (maybe in another player, or by playing side A as "side B" on a double-side player or vice versa). This doesn't have to be the entire disc, because you can just use a hex editor to patch in the last wee bit over the top of what you already got. But when I tried this, in practice, it still seemed like there was no termination frame in the stream, or if there was, none of my players wanted to capture it (and evidently it didn't matter in the slightest).

5. If you want to get *real fancy* for basically no good reason except curiosity, you could instead edit in the right number of blocks but leave the last block as a partial one, then edit the header of the last frame so that its type is "termination" instead of "normal" (ftype 1 -> 0) then work out how many samples are actually left in the last block of the last frame and edit "short" to reflect that. But this seems pretty much pointless in practical terms as you'll not be gaining much (apart from a successful decode) and you still won't have a 100% complete "bit perfect" capture if that's what you're really looking to get. The only way to do that is by trying again.


EDIT: To clarify something from discussion further down, here's a visual representation of how Audacity renders the waveform to show you where the DTS signal drops to silence (which is still encoded as DTS), and then when the DTS bitstream stops playing entirely and the signal drops to zero data (effectively interpreted as "silence" since the signal from the player is PCM, it just happens to contain a different type of data from actual PCM audio)...



You can even see where the data is missing from the end of the frame if you zoom in on the bit of the waveform containing the DTS data (labelled as "silent DTS frames" in the above image), before the signal drops out and goes to zeroes:



One final clarifier: when I'm saying that this last frame is "partial", I don't mean it is literally not the right amount of bytes after feeding it through DTS Parser. It is, I think I showed that in one screenshot. The problem seems to be that the end of the frame is all zero bytes, when it seems to need to end in meaningful data (based on the fact these frames are silent, perhaps this is a corresponding "footer", or end marker, to indicate the end of the frame just like the header indicates the beginning of the frame). But the only reason that the frame has the correct length in the output .dts is that DTS Parser knows the length of each frame, even though it writes the number into the header wrong: it knows that the frame is supposed to be 3584 bytes long, and although the bitstream actually ended at some point just before the final 62 bytes were actually recorded, there are available zero bytes still rendered into the 2-channel PCM signal that the DAW is recording (because the player has stopped reading the disc but you haven't pressed "stop" on the recording yet), DTS Parser just includes the corresponding number of bytes from the zero signal to close off the last frame. But this isn't valid data to close off the DTS frame, so you get the decoding error.

[zoidberg]

I'd hardly call this a bodge, this is brain surgery level stuff!

I'll have to check one of my captures but I'm pretty certain the invalid error comes up even after 'lopping' off the end via eac3to, perhaps it only removes the last valid frame and leaves the remaining partial junk frame behind. Perhaps DTS Parser can be updated to remove this frame?

Regarding side breaks, the best way to deal with them is to treat each side as a separate file, parse each file and repair the headers then stitch the sides together with mkvtoolnix (creating a .mka file), which can then be demuxed back to a .dts file for editing. Although if there's an easier method I'd be glad to hear it.

[BusterD]

Yes indeed, thank you Pipefan! I often wish that I was more technically inclined to figure out stuff like this...

[pipefan413]

I'd hardly call this a bodge, this is brain surgery level stuff!

I'll have to check one of my captures but I'm pretty certain the invalid error comes up even after 'lopping' off the end via eac3to, perhaps it only removes the last valid frame and leaves the remaining partial junk frame behind. Perhaps DTS Parser can be updated to remove this frame?

Regarding side breaks, the best way to deal with them is to treat each side as a separate file, parse each file and repair the headers then stitch the sides together with mkvtoolnix (creating a .mka file), which can then be demuxed back to a .dts file for editing. Although if there's an easier method I'd be glad to hear it.

Aye, that works, but you can just directly concatinate the files, which is quicker and I trust it better. In Linux it's just "concat", in Windows command prompt "copy /b" does the same job. It just sticks the data from file 1 onto the data from file 2, which for DTS and AC-3 works fine because it's just a string of frames with no additional header on the start anyway. That's what I did on things like NEAR DARK when I made matched encodes of the modern SC logo, to stick onto the start (after chopping off the old one). But I agree, capturing a side at a time or splitting a joined cap in two is fine, no difference really because the silence

But yes, same principle anyway so whatever you find works for you!

As for whether it happens after cutting the ends off, yes it will, if you're cutting into the file before parsing. If it's after parsing, it will still happen, unless you precisely chop off to the nearest whole frame. Because the silence on the end isn't DTS silence, it's just PCM zeroes, and as far as I can see won't be parsed into the .dts file (no frame header on the "silence" after all because it's just zero bytes and not encoded DTS silence). But if you use something like DelayCut or direct editing of the parsed .dts file in a hex editor, you'll not have that issue. Personally I'd rather just directly hex edit to the end of the last frame which is easy: just search backwards from the end for the last header and delete from the start of that to the end.

I'm explaining this badly, I just had to do something really difficult IRL and my brain isn't engaged. I'll try to show you visually what I mean later on.

[little-endian]

Great and elaborate analysis of yours, pipefan413!

For LaserDisc though, which has all digital audio at 44.1 kHz[...]

Just for the sake of completeness - all except the extra modulated AC3 audio which runs at 48 kHz (which might be the very reason for using the channel normally dedicated for the analog FM audio instead of using the ones for PCM, like DTS decided to do, but I am no expert).

I cannot test it myself right now, so I just wanna "through that onto the pile" so to say - how is the recreated result of "bsconvert", which is part of the ac3filter package? Compared to the rather buggy DTS Parser, it also has the advantage to indirectly notify of any gaps in a given capture as it will list the frame position in a seperate line output whereas in the best case with no capture errors, it will simply "count" till the end and that's it.

As for the slight differences on every capture, that is my experience as well, also for AC3 frames. However, I consider that "issue" to be rather academic just like with the sample offset when doing DAE on good old audio cds.

[pipefan413]

Great and elaborate analysis of yours, pipefan413!

For LaserDisc though, which has all digital audio at 44.1 kHz[...]

Just for the sake of completeness - all except the extra modulated AC3 audio which runs at 48 kHz (which might be the very reason for using the channel normally dedicated for the analog FM audio instead of using the ones for PCM, like DTS decided to do, but I am no expert).

I cannot test it myself right now, so I just wanna "through that onto the pile" so to say - how is the recreated result of "bsconvert", which is part of the ac3filter package? Compared to the rather buggy DTS Parser, it also has the advantage to indirectly notify of any gaps in a given capture as it will list the frame position in a seperate line output whereas in the best case with no capture errors, it will simply "count" till the end and that's it.

As for the slight differences on every capture, that is my experience as well, also for AC3 frames. However, I consider that "issue" to be rather academic just like with the sample offset when doing DAE on good old audio cds.

All of what I said was for DTS; I don't have an AC-3 demodulator or player (I prefer the picture on older players from the 3 I've got and even those predate AC-3). I want to get an AC-3 player and demod in 2021 but I won't have enough money for a good while after all this.

[zoidberg]

The editing to which I referred only applies to DTS, for PCM I just import directly into audacity as it can edit bit-perfect. For AC-3 captures I edit the raw capture directly then convert with besplit
Also modding a player for AC-3 is relatively easy, getting hold of a demodulator less so

[pipefan413]

The editing to which I referred only applies to DTS, for PCM I just import directly into audacity as it can edit bit-perfect. For AC-3 captures I edit the raw capture directly then convert with besplit
Also modding a player for AC-3 is relatively easy, getting hold of a demodulator less so

Right, I get that you were talking about DTS; what I'm saying is that it sounds like you're talking about cutting the "silence" off the end that lies after the actual DTS data stream. So it's being packaged as "PCM" for the simple fact that this is how the format stores it, but you get zeroes before the actual data (DTS) begins, then zeroes after, and those zeroes are rendered as silence by the DAW (Reaper for instance).

So, if what you mean when you say you lopped the end of is that you removed that silence, then you're not removing any DTS frames whatsoever, you're only removing zeroes from the end of the file that occur *after* the DTS frames. And therefore when you parse that, you'll still end up with "DTS silence" (as in, silent frames of DTS) that are encoded into the bitstream at the end of the file and/or side change. And based on what I'm seeing, it's highly likely (possibly almost guaranteed) that you'll have a partially captured frame right at the end of that data stream, which is why people are seeing this decoding error: the decoder hits the partial frame and notices that the data is incomplete so can't decode it.

But since images speak more clearly than words in many cases, here's a visual for what I'm describing:



I'm guessing that when you say you lopped off the end, you're talking about the bit where the zero data is, not the actual DTS bitstream, which is where all the DTS frames (including the final, partial one) reside.


Hell, you can even see where the data is missing from the end of the frame if you zoom in on the bit of the waveform containing the DTS data (labelled as "silent DTS frames" in the above image), before the signal drops out and goes to zeroes:




One final clarifier: when I'm saying that this last frame is "partial", I don't mean it is literally not the right amount of bytes after feeding it through DTS Parser. It is, I think I showed that in one screenshot. The problem seems to be that the end of the frame is all zero bytes, when it seems to need to end in meaningful data (based on the fact these frames are silent, perhaps this is a corresponding "footer", or end marker, to indicate the end of the frame just like the header indicates the beginning of the frame). But the only reason that the frame has the correct length in the output .dts is that DTS Parser knows the length of each frame, even though it writes the number into the header wrong: it knows that the frame is supposed to be 3584 bytes long, and although the bitstream actually ended at some point just before the final 62 bytes were actually recorded, there are available zero bytes still rendered into the 2-channel PCM signal that the DAW is recording (because the player has stopped reading the disc but you haven't pressed "stop" on the recording yet), DTS Parser just includes the corresponding number of bytes from the zero signal to close off the last frame. But this isn't valid data for the DTS frame to actually be valid, so you get the decoding error.

[spoRv]

FYI: PCM on laserdisc should (may) be 44056Hz...