FFV1 encoding for archival

[pipefan413]

I did some fairly extensive comparisons a while back (that I don't think I necessarily posted anywhere in any kind of organised thread or anything like that) between several different mathematically lossless video encoders, when I was first trying to decide on the best one to use for archiving all my LaserDisc captures. At that time, I was comparing HuffYUV, Lagarith (a derivative of HuffYUV), UtVideo, and MagicYUV. At the risk of oversimplifying to the point of seeming dismissive, my results basically indicated that although UtVideo and MagicYUV were both excellent overall (both offering much better flexibility than Lagarith does), and both seemed to be faster (with MagicYUV being *incredibly* quick), Lagarith ultimately offered the most efficient compression. Since that was my primary concern (I didn't really need the extra flexibility or speed for 8-bit 4:2:2 YUV LD captures anyway) I settled on using Lagarith.

However, since then, I've been hearing good things about ffmpeg's FFV1 codec, which apparently is used extensively for archival by national libraries and the like. I've also been increasingly concerned with whether I should switch to 10-bit captures for at least some of my LaserDiscs (the ones where I am particularly bothered about preserving the video in the highest possible archival quality, which isn't necessarily all of them). 10-bit captures are orders of magnitude larger than the 8-bit ones, but I wondered if the supposedly extremely efficient compression of FFV1 might help to narrow that gap and make it viable to keep hundreds of 10-bit LD caps archived. So I figured I'd give it a bash.

Since I was aiming for the best possible compression, I opted to use its 2-pass mode... and immediately ran into a problem. When compressing a 10-bit video, the first pass (or a single-pass encode) worked fine, but as soon as it ran the second pass, I got something like this:

Very pretty! Not exactly what I was after, though. (Reminds me of how bismuth crystallises.) One thing that's kinda cool is you can clearly see the 4 slices in the output (see how the image is split into 4 boxes).

I spent more than 24 hours farting about trying to debug my command line to see what I'd done wrong but as I ran out of things to change I became increasingly convinced it was just broken. Sure enough, I eventually found something that I didn't see the first time I tried to look online to see what the issue might be: somebody raised a bug report 5 years ago saying that 10-bit 2-pass encodes were broken, and it still hasn't been fixed.

Summary of the bug:
When source video has more than 8 bits per component and ffv1 codec is used in 2 pass mode, resulting file is corrupted.

(https://trac.ffmpeg.org/ticket/5405)

So, for now, 10-bit needs to be 1-pass only.

But what about 8-bit captures, for the (probably vast majority) of LDs I don't particularly need a high quality version of the video from? Mostly it's the audio I'm after, anyway.

LAGARITH: 55.5 GB
FFV1, 1-pass, Golomb Rice coder (default for 8-bit): 50.8 GB
FFV1, 1-pass, range coder (default for 10-bit+): 52.1 GB
FFV1, 2-pass, Golomb Rice coder: 50.8 GB

Yeah, you read that right: the 2-pass encode was exactly the same size as the 1-pass encode. No, it isn't a tiny bit smaller. Both of them are exactly 54,547,736,802‬ bytes. The data is not identical though, they're just the same size. So all the 2-pass encode does is take longer; it doesn't compress this particular file any better.

I wonder if maybe the 2-pass mode might be more worthwhile for something like a 1080p/4K+ video, but it would appear that for standard definition LaserDisc captures it's completely pointless.