Interlaced video in HQ codec

[pipefan413]

I have a featurette that's been framed incorrectly and it's really annoying me so I want to fix it. Unfortunately, it's:

1. Very low res (it's 4:3 SD in NTSC standard, 720 x 480)
2. Interlaced MPEG2

I've tried several ways of getting this de-interlaced for editing purposes but it always looks fairly awful. I've not really dabbled with Avisynth yet, but I think if possible it might be better just to leave the footage interlaced instead. Maybe if I explain what I'm trying to achieve, someone will be able to suggest the best way to go...

The bit of the video that I want to keep is a roughly 704 x 396 frame in the middle, because there are thin black bars on either side (~5-8px, some tape distortion so subjective) and much larger black bars on the top and bottom letterboxing from the 4:3 full frame. I want to keep it compatible with Blu-ray as it currently is but I'm not sure how to get it into an editable format without deinterlacing it first, and obviously 704 x 396 is not a BD compliant resolution either so I'm probably also going to have to upscale it to something like 720/1080 (704x396 upscales to either 1280x720 or 1920x1080 without any black bars).

Do you reckon I'm going to have to convert it to progressive to do the crop without it looking like crap? I know I can use ffmpeg to feed it through deinterlacing (but I think the only option is yadif, which is iffy in my experience) before cropping and upscaling, so it looks horrible even if I do all 3 in a single command. I think ideally I'd rather keep it interlaced, but I'm not sure how to retain interlacing in ffmpeg while also doing the crop and upscale.


Note: I just realised that the reason it's 720 x 480 interlanced but the actual image is only about 704 pixels wide is probably because 16/720 samples are used for horizontal sync/blanking when transmitting 720x480 digitally with rectangular (taller than they are wide) pixels. Therefore, I'm wondering if this video might have been encoded with the image squished horizontally. A newer encode is actually available but that looks much worse overall due to several aliasing issues. I imagine it probably isn't as simple as just stretching the image horizontally to correct this, but I can do that if that's all it needs. The older source is interlaced with the usual "one then the other" line arrangement but the newer one uses interleaved fields instead and seems to look worse for it.Basically, it's a horrible mess regardless of which encode I source from.

[deleted user]

Two things.

1. Look into AVISynth's QTGMC, best deinterlaced I know atm.

2. The reason it's 720 is because that's the DVD standard afaik. The actual resolution of playback is defined by the pixel aspect ratio (PAR), which for 4:3 NTSC is under 1, so the image gets squished horizontally upon playback a little.

[Sabbelbacke]

Hey there,

720x480 is NTSC, 720x576 would be PAL.

First, check if everyhting is "ok" so far:

Download mediainfo and check for the following:

- Is the Video really flagged as Interlaced (it might still be interlaced, but flagged as progressive - which might need your attention in further processing).
- Check which field comes first (stored as BFF or TFF = Bottom or Top field fiirst)
- check the aspect ratio TomArrow mentioned.
- Check for framerate. It probably is 29,97

This probably is a demux from a DVD or TV Recording? To know if the original source is a TV Camera or a film is good to know for future processing, there might be things like Telecine present.

Now do a visual check to ensure there is no weird stuff going on. An easy way to do this would be:

download virtualdub2. Open your file. Go into the video tab, select filter: "Deinterlace".. You find the option "unfold field side by side". This takes the odd and the even pairs of fields and puts them in two seperate pictures side by side.

If you know look at the filtered result frame by frame, you should see two "clean" pictures side by side without any combing effects left or weird colour shadows from frame to frame.

Now pay attention to a steady motion scene. Check if every frame is different from the previous - if you find "doublettes", then you probably have a Telecine introduced - the footage originally was another framerate - and therefore some of the fields or frames or doubled... You probably want to get rid of this (which is possible).

If everything checks out so far - I´d second Tom Arrows recommendation of QTGMC, which is one (if not the one) of the most powerfull deinterlacers around. It originated as Avisynth filter, today, also a Vapoursynth version is around. If you are not up for Videoskripting, you could simply try one of the GUIs around... Selur´s hybrid, Staxrip, Ripbot... could be some examples. For a quick deinterlace test with YADIF (another not so bad deinterlacer, but less powerfull than QTGMC), you can try virtualdubs implementation..

This is a wide field, so I´ll leave it here for now and wait for your findings. You could upload a small peace of the footage in case you want me/us to take a look at it...

Regarding the crop you want to make: In theorie, leaving it as is (720x480 frame) and just cropping or adding a border visually keeps the best resolution. Every croping and upscaling afterwards introduces some loss of sharpness - but there are so many potent upscalers around these days, I wouldn´t worry too much... So you probably will end up cropping and then upscaling to the desired end-resolution. Which upscaler is best depends on your footage - this is a wide field and starts with simple bicubic interpolation and ends with AI upscaling with neural networks. In any case you would want to keep the aspect ratio in mind (right click in cirtualdubs preview to select different ratios... 1:1 is what actually is stored in the file..)

[pipefan413]

Two things.

1. Look into AVISynth's QTGMC, best deinterlaced I know atm.

2. The reason it's 720 is because that's the DVD standard afaik. The actual resolution of playback is defined by the pixel aspect ratio (PAR), which for 4:3 NTSC is under 1, so the image gets squished horizontally upon playback a little.

1. Will do. Is there no way to just leave it interlaced and do the crop and scale? Need to deinterlace?
   
   
2. This is something I've never fully understood methinks. I think what's happened here is that the video probably should have been encoded as widescreen NTSC (with the requisite PAR >1) but it was instead exoded as 4:3 NTSC (with <1 PAR) hence both the horizontal squish with pillarboxing and letterboxing. What I don't know is whether it's possible to fix with cropping and/or stretching the existing image.
   

Hey there,

720x480 is NTSC, 720x576 would be PAL.

First, check if everyhting is "ok" so far:

Download mediainfo and check for the following:

- Is the Video really flagged as Interlaced (it might still be interlaced, but flagged as progressive - which might need your attention in further processing).
- Check which field comes first (stored as BFF or TFF = Bottom or Top field fiirst)
- check the aspect ratio TomArrow mentioned.
- Check for framerate. It probably is 29,97

This probably is a demux from a DVD or TV Recording? To know if the original source is a TV Camera or a film is good to know for future processing, there might be things like Telecine present.

Now do a visual check to ensure there is no weird stuff going on. An easy way to do this would be:

download virtualdub2. Open your file. Go into the video tab, select filter: "Deinterlace".. You find the option "unfold field side by side". This takes the odd and the even pairs of fields and puts them in two seperate pictures side by side.

If you know look at the filtered result frame by frame, you should see two "clean" pictures side by side without any combing effects left or weird colour shadows from frame to frame.

Now pay attention to a steady motion scene. Check if every frame is different from the previous - if you find "doublettes", then you probably have a Telecine introduced - the footage originally was another framerate - and therefore some of the fields or frames or doubled... You probably want to get rid of this (which is possible).

If everything checks out so far - I´d second Tom Arrows recommendation of QTGMC, which is one (if not the one) of the most powerfull deinterlacers around. It originated as Avisynth filter, today, also a Vapoursynth version is around. If you are not up for Videoskripting, you could simply try one of the GUIs around... Selur´s hybrid, Staxrip, Ripbot... could be some examples. For a quick deinterlace test with YADIF (another not so bad deinterlacer, but less powerfull than QTGMC), you can try virtualdubs implementation..

This is a wide field, so I´ll leave it here for now and wait for your findings. You could upload a small peace of the footage in case you want me/us to take a look at it...

Regarding the crop you want to make: In theorie, leaving it as is (720x480 frame) and just cropping or adding a border visually keeps the best resolution. Every croping and upscaling afterwards introduces some loss of sharpness - but there are so many potent upscalers around these days, I wouldn´t worry too much... So you probably will end up cropping and then upscaling to the desired end-resolution. Which upscaler is best depends on your footage - this is a wide field and starts with simple bicubic interpolation and ends with AI upscaling with neural networks. In any case you would want to keep the aspect ratio in mind (right click in cirtualdubs preview to select different ratios... 1:1 is what actually is stored in the file..)

I know a good chunk of what you just said (primarily the more straightforward bits) but I appreciate how thorough this is and there's definitely useful stuff in here.

I'm aware the NTSC standard contains 720x480i and yes the video is indeed flagged as interlaced; there are two sources for the same video, but one is interlaced with top field first whereas the other (newer, more aliased) one is interleaved fields instead. Both are demuxed from Blu-ray (bonus content) but appear to be from tape sources as it's a behind the scenes documentary. And yes, both run at 29,97 fps.

The checks you just described will be interesting. Just tried doing what you suggest in VirtualDub and I'm not 100% sure what I'm looking for but I'm pretty sure there *are* still issues with the deinterlaced fields, though that might just be *because* they're deinterlaced (as in I'm looking at every 2nd field so there will be "jumps" in the image where it isn't showing one field or another). And the video is low res anyway. Have a look at the rope here for example:



I don't really know if that's purely an interlacing issue or caused by something else entirely. It is however present in both versions and looks much less terrible if deinterlaced on the fly in a player like VLC than the newer (interleaved) source does. Curiously, though, the newer source *does* reshape the image, stretching it (compared to the above) to fill the horizontal width of the 720px space. Would you say that is more correct, or less so?

I've not yet checked for telecine issues / frame duplication but can do.

Re. the softening of the image from upscaling, I know this will happen but don't know if it will subjectively look OK or not so was going to try it and see. I just didn't want to do anything to it to make it even worse than necessary e.g. just using YADIF (which didn't look great at all).

The whole thing is a horrid mess, basically seems to be the long and short of it. I don't know if there's a whole lot I can do to make it more tolerable.

Thanks both.

[Sabbelbacke]

hm, hard to tell exactly - could you upload two small pieces of these two sources (without re-encode, preferably the same scene with a mixture of motion and stills)? I could take a look, maybe I find something usefull..

[alleycat]

This might be completely different, but I just had an issue where someone had taken SD 4:3 footage and put it on a DVD, but they had somehow flagged it as 16:9 so it was stretched when playing. To fix it I took the video into DVD lab pro and applied the aspect ratio patch tool, in my case changing the footage to 4:3 and now it looks fine, and I’m pretty sure it didn’t reencode it either as it did it very quickly. Not exactly sure what it does but it worked great for me. I’ll wonder if when your DVD was made the footage was incorrectly flagged as well?

[Sabbelbacke]

You can change the header where the aspect ratio is stored and "fix" footage like this without touching the actual video itself. Sometimes a remuxing is necessary, but that depends on the tools used - DVD lab pro seems to do it "seamlessly", which is nice.

[pipefan413]

This might be completely different, but I just had an issue where someone had taken SD 4:3 footage and put it on a DVD, but they had somehow flagged it as 16:9 so it was stretched when playing. To fix it I took the video into DVD lab pro and applied the aspect ratio patch tool, in my case changing the footage to 4:3 and now it looks fine, and I’m pretty sure it didn’t reencode it either as it did it very quickly. Not exactly sure what it does but it worked great for me. I’ll wonder if when your DVD was made the footage was incorrectly flagged as well?

Yeah this is if anything the other way around: the frame of actual video is widescreen 1.85:1 or thereabouts, but it's been stuck into a 4:3 NTSC frame. So watching it on a widescreen telly means you have black pillarboxing on either side *and* huge letterboxing on top and bottom. Irritating, because if it *had* been encoded as 16:9 then it would have almost filled the screen... which is exactly what I'm trying to achieve, with as little degradation as possible. The trouble is, because they didn't do that, it's only about a 704 x 396 pixel area of useable image that needs to be upscaled and doing it in any way that doesn't involve simply doubling the fields is going to introduce some issues. It's also horrid quality in the first place, so an upscale won't look brilliant regardless of what I do, I'm just trying to minimise introducing new artefacts.

What I've done thus far (while working full time and mucking about with multiple other projects, so I've not been posting as quickly as I'd like here) is deinterlace it, crop and upscale with an AviSynth script to get a prores .mov file that looks passable and certainly significantly better with what Handbrake can manage using things like YADIF/decomb. However, the script I put together uses ffmpeg which means I can't then use x264 for the encoding (I'd rather go straight from source to the final encode since I don't want to do any edits, just reframing/scaling). I'd prefer to use similar settings to those found here for future-proofing reasons (I might like to stick it on a disc one day) but haven't yet figured out how to rework my ffmpeg-based AviSynth script to work with x264. As far as I can tell, I'd have to use other plugins to load the MPEG2 input file, and I did start trying to do this (converted it to a .d2v and loaded it) but ran into issues that I haven't yet invested time in trying to work around (e.g. it appeared not to have loaded the plugin needed to read the .d2v, so I manually loaded it in the script, but then it said I couldn't use a 32-bit plugin in 64-bit AviSynth despite the fact most of the other plugins that were working were also 32-bit).

You can change the header where the aspect ratio is stored and "fix" footage like this without touching the actual video itself. Sometimes a remuxing is necessary, but that depends on the tools used - DVD lab pro seems to do it "seamlessly", which is nice.

That would be the ideal thing to do if it were possible but because this isn't simply an aspect ratio problem (it isn't widescreen squished to 4:3 or vice versa, it's actually been encoded with big black bars in a 4:3 frame) I don't think that alone will suffice. I'm going to have to go the AviSynth route methinks. If I can't work out how to rework my script to allow using x264 instead of libx264-via-ffmpeg encoding then I might just encode from the prores file that I've got from my ffmpeg output and move on with my life. But I'd ideally prefer to not have prores as an intermediate step so that as little messing with the original source has been done as possible.

[Sabbelbacke]

Yes, sadly it has become a bad habit of stroing 4:3 footage in a 16:9 flagged Frame, including the black bars left/right. Maybe authoring studios want to avoid having a squished display on modern 16:9 TVs, maybe some can´t interpret the DA ratios korrekt - I rather assume it´s just sloppy encoding of people not knowing the basics any more....
Instead of encoding it using the full fram, one looses a lot of resolution of the original picture. I´ve seen this on a few kids-dvds I bought recently.

Going for the Avisynth route is great, but a lot of learning to do.

If you want it a little eaier, take a look at ripbot. It has a robust GTQMC implementation for the deinterlacing, a good degrain function and cropping as well as resizing already built in. Encoding to x264 or x265 is possible too (and even dsitributed encoding, if you like). Switch to GTQMC in the settings.. This way you might be able to convrt the footage in one go..

Another tool would be staxrip, which gives a little more freedom..

Basicaly, you´d want to do this:

crop the input picture. Deinterlace it with GTQMC. Maybe degrain it, depending on the picture quality or your likings. Then resize it to AR 1:1 according to your input. If you want to process this further and need a losless intermediate file - use a losless codec or set x264 to Q0 which is losless (workaround).

If IVTC is neede, you will find switches in the above mentioned software.

[pipefan413]

Yes, sadly it has become a bad habit of stroing 4:3 footage in a 16:9 flagged Frame, including the black bars left/right. Maybe authoring studios want to avoid having a squished display on modern 16:9 TVs, maybe some can´t interpret the DA ratios korrekt - I rather assume it´s just sloppy encoding of people not knowing the basics any more....
Instead of encoding it using the full fram, one looses a lot of resolution of the original picture. I´ve seen this on a few kids-dvds I bought recently.

Going for the Avisynth route is great, but a lot of learning to do.

If you want it a little eaier, take a look at ripbot. It has a robust GTQMC implementation for the deinterlacing, a good degrain function and cropping as well as resizing already built in. Encoding to x264 or x265 is possible too (and even dsitributed encoding, if you like). Switch to GTQMC in the settings.. This way you might be able to convrt the footage in one go..

Another tool would be staxrip, which gives a little more freedom..

Basicaly, you´d want to do this:

crop the input picture. Deinterlace it with GTQMC. Maybe degrain it, depending on the picture quality or your likings. Then resize it to AR 1:1 according to your input. If you want to process this further and need a losless intermediate file - use a losless codec or set x264 to Q0 which is losless (workaround).

If IVTC is neede, you will find switches in the above mentioned software.

I've got it working well if I go the intermediate route via prores but I don't really want to do that, just trying to use the most direct path like I said. What I did initially was this:

SetFilterMTMode("QTGMC", 2)
FFmpegSource2("inputfileasMPEG2.mkv")
AssumeTFF()
Crop(10, 40, -6, -44)
QTGMC(Preset="Slower", Edithreads=6)
BicubicResize(1280,720)
Prefetch(threads=14)

I didn't bother with degrain because the source appears to be tape rather than film anyway, and I tried a few different upscalers and found that this one looked best to my eyes (for this footage anyway; I saw a lot suggestions to use Spline64 but it sharpened the crappy low res source way too much and it looked terrible). This outputs a 720p upscaled and deinterlaced version of the video in the correct aspect ratio, in a format that depends on whatever I do in ffmpeg. The trouble is, I don't know without sitting scrutinising it in detail how to lift the entire x264 syntax and make it work in ffmpeg, and I definitely haven't yet worked out how to avoid ffmpeg in AviSynth so that I don't have to use ffmpeg to encode as well. It might be best just to accept that using ffmpeg + libx264 is acceptable and then reconform the x264 commands to that, but I feel like it's going to annoy me if I can't get the script to work with x264.

If I can work out how to rewrite the script to work with x264 instead of ffmpeg, then I'm probably looking to basically run slightly tweaked versions of the following to see what the files look like (size vs quality):

Code:

x264 --pass 1 --bitrate 20000 --bluray-compat --level 4.1 --preset veryslow --tune film --keyint 60 --sar 1:1 --slices 4 --vbv-bufsize 30000 --vbv-maxrate 40000 --b-pyramid none --weightp 0 --colorprim smpte170m --transfer smpte170m --colormatrix smpte170m --merange 24 --stats ".stats" --output NUL input.avs
x264 --pass 2 --bitrate 20000 --bluray-compat --level 4.1 --preset veryslow --tune film --keyint 60 --sar 1:1 --slices 4 --vbv-bufsize 30000 --vbv-maxrate 40000 --b-pyramid none --weightp 0 --colorprim smpte170m --transfer smpte170m --colormatrix smpte170m --merange 24 --stats ".stats" --output dewback720.h264 input.avs
PAUSE