AutoOverlay (PKA AutoAlign) discussion

[deleted user]

Since random.next released his cool AviSynth plugin and nobody has done this yet, here's a thread for us to discuss this before he gets to finish the stable build and releasing a guide.

So, I saw spoRv's test and felt motivated, but didn't get far. Tried matching an HDTV with a WEB-DL. They are perfectly in sync and the framing is almost identical, but there are small discrepancies that don't allow me to overlay them. In particular, each source seems differently stabilized and there are lots of micro-tremors that make a simple overlay impossible. They also have somewhat different color timing, but not remarkably so. Hence my hopes in this plugin.

Tried the following, stolen from spoRv's sample: 

Code:

config=OverlayConfig(subpixel=2)
OverlayEngine(srcdehalo2.ConvertToY8(), src2.ConvertToY8(), configs=config,editor=false,maxDiff=20)
OverlayRender(srcdehalo2.ConvertToYV24(), src2.ConvertToYV24(), debug=true, noise=0,\
upsize="Spline64Resize", mode=3, gradient=10,coloradjust=1,width=1920,height=1080,opacity=0.5) 

srcdehalo2 is my WEB-DL (1080p) and src2 is my HDTV source (1080i, but already converted to 1080p in the correct framerate)

All that this does for me is to overlay the two sources with the same minor alignment issues it already had before. In other words, it's not really aligned. My goal is to have a perfect overlay so I can combine the two sources into a higher quality source through averaging.

That debug=true displays some info. x:0, y:1 Angle: 0.00. Sometimes y is 0 and sometimes x is 1. Angle seems always 0.00. Diff usually between 5 and 20. I read some remark by random.next that maxDiff should be bigger than the maximum diff inherent to the sources (I hope I didn't get that wrong), but either value doesn't really make a difference.

I also ran the README.md through Google Translate, for whatever it's worth (had to do it piece by piece):

admin edit: replaced with a more readable HTML page -> https://a.pomf.cat/oonuzp.html

link to original thread: https://fanrestore.com/thread-1922.html

[deleted user]

I finally got it to work, by setting a minOverlayArea value. I set it to 20 first, but I put it up to 90 now and it still seems to work.

Some scenes still seem to have issues, but overall it seems to work at a basic level now.

[spoRv]

Hint: it is possible to use the script also to check out if a given shot is full open matte; for example, on The Matrix HDTV, there are few shots (the interrogation scene, for example) that are almost full open matte, where I believed all were "normal" open matte (cropped at the sides, with more image on top and bottom).

[random.next]

@TomArrow
If sources are unstabilized to each other then you must disable prediction OverlayEngine.backwardFrames=0. But perfomance then will be low.
I recommend to use plugin for sources which was cropped from the same original transfer to different aspect ratio.
Also you should test different main parameters in OverlayConfig: aspectRatio range, angle range, minSourceArea, minOverlayArea. Subpixel mode may be activated after that because it reduces perfomance. The target is Diff minimaztion which is displayed in debug mode of OverlayRender. The difference between sources is shown in mode=7. With best align it is blank grey rectangle.

[random.next]

@spoRv
Thanks for translation. I'll check it and update at github soon.

[random.next]

>I finally got it to work, by setting a minOverlayArea value. I set it to 20 first, but I put it up to 90 now and it still seems to work.

If you have 16:9 and 2.35:1 sources then these parameters usually should be about 70 percent by default.
If you have sources with the same aspect ratio then you must set them manually about 90-95 in common way otherwise they will be set to 100 by default becuase sources are in the same resolution.

[random.next]

> forgot to mention: how to get the best (perfect?) alignment? And, is it possible to get a list of cropping and position values, so one could eventually modify that list to fix problematic alignments?

You should use overlay editor after analysis pass with stat file by enabling OverlayEngine.editor property.

[spoRv]

Should I make two passes, though? Still trying to figure out how this wonderful plugin of yours work - and results, so far, are very impressive! But, tell me, did you get also some imperfect result - like not aligned frames, in difficult situation? If not, you should tell me your secrets...

[random.next]

Just run analysis pass with statFile="your_path".
Then add editor=true and visual editor will be opened. I guess you use AvsPmod or something like this.
If the sources from the same transfer (digital mastercopy) then they usually can be aligned to each other in 100% frames with some manual correction in the editor after analysis.

[spoRv]

I use VirtualDub and CuteHTML

I've noted the editor, but put it apart because got no patience! but I guess I should give it another chance!