AutoOverlay (PKA AutoAlign) discussion

[random.next]

I have those "indents" if I use the following code:

Code:

#    sdtv=1.33:1    web=1.78:1    bd=2.35:1

config=OverlayConfig(subpixel=2,requiredSampleArea=5000,minSampleArea=3000,acceptableDiff=3)

OverlayEngine(bd.ConvertToY8, web.ConvertToY8, configs=config,editor=false,backwardFrames=8,forwardFrames=8)
one=OverlayRender(bd.ConvertToRGB24, web.ConvertToRGB24, debug=false, noise=0,\
upsize="Spline64Resize", mode=3, gradient=0,coloradjust=0,opacity=0)  

OverlayEngine(one.ConvertToY8, sdtv.ConvertToY8, configs=config,editor=false,backwardFrames=8,forwardFrames=8)
two=OverlayRender(one, sdtv, debug=false, noise=0,\
upsize="Spline64Resize", mode=3, gradient=0,coloradjust=0,opacity=0)

return two

Of course, overlaying the SDTV I got no "indents":

Code:

#    sdtv=1.33:1    web=1.78:1    bd=2.35:1

config=OverlayConfig(subpixel=2,requiredSampleArea=5000,minSampleArea=3000,acceptableDiff=3)

OverlayEngine(bd.ConvertToY8, web.ConvertToY8, configs=config,editor=false,backwardFrames=8,forwardFrames=8)
one=OverlayRender(bd.ConvertToRGB24, web.ConvertToRGB24, debug=false, noise=0,\
upsize="Spline64Resize", mode=3, gradient=0,coloradjust=0,opacity=0)  

OverlayEngine(one.ConvertToY8, sdtv.ConvertToY8, configs=config,editor=false,backwardFrames=8,forwardFrames=8)
two=OverlayRender(one, sdtv, debug=false, noise=0,\
upsize="Spline64Resize", mode=3, gradient=0,coloradjust=0,opacity=1)

return two

Now, I want to get rid of those "indents" (that would be done using SDTV); how can I do it? I tried using your solution, but I wasn't be able to solve it... help needed here!

You need to use sourceMask and overlayMask in engine and render to exclude logo from diff analyzing and angles from overlay. Example: https://fanrestore.com/thread-1922-post-...l#pid38501

[spoRv]

The problem here is not the logo - at least, not in this shot; what I would like to obtain is a clear mask for the border, to get rid of the "indents"; but using the black of the borders for mask, also some black inside the image would be used as mask... that's why a custom color for the border would be useful - let's say, a full red $FF0000 would never occour in the image, and so the mask will only be applied in the borders.

[deleted user]

Correct me if I misunderstood this, but didn't random.next recently show us some option for OverlayRender (was it mode=6?) where you can output a mask that basically consists of only black and white? Then there would be no problem regarding black colors in the image getting recognized as transparent.

So you basically do two passes. One normal OverlayRender and one with mode=6. The first becomes the image, the second is the mask for that image. That's how I understood it anyway.

[spoRv]

Another problem is that, due to resize, there is always a 1px/2px border near the image which is like a mix between image and border; so, the mask would never cover that tiny border, and the indent would have internal borders, got the idea?

Yes, indeed having borders with different color would not solve this problem. I guess I could move the borders left/right/down/up and crop/addborders, to get the mask "expand" and cover also that tiny border, but I'm too lazy for this - or, simply, too old and my brain is simply starting to (laser)rot!

[deleted user]

random.next would have to implement some form of pre-multiply for that (I hope that's the right term), where basically the image is always pushed to 100% alpha internally where it is above 0% alpha. The mask then recreates the original border, eliminating the imperfection. That's what TIFF images with alpha masks do.

OR, make use of RGBA mode with an actual alpha channel. That way there would be no black background in the first place. But the implementation would have to likely use some form of premultiply anyway, so it probably doesn't matter.

[spoRv]

I guess the 1px "merged" border and 1px near-black border are due to subpixel alignment and resize; cropping the final image by 2px (with an optional command) would help to have a border of one color (possibly custom )

[random.next]

>You need to use sourceMask and overlayMask in engine and render to exclude logo from diff analyzing and angles from overlay.
The first question. Why you always use bd as source clip and other clips as overlay? Overlay always at the top of source. I think bd has higher quality for this. 
So use "OverlayEngine(WEB, BD..." or "OverlayEngine(SDTV, OM...". It is logically correct. Now opacity may be default to 1.
Next. Why you don't use resampleMT as recommended? Using Spline64Resize is bad descision for engine (not render) because it is slow. BilinearResize usually provide the same result. BilinearResizeMT is much faster than Spline64Resize. If your source clip have lower resolution than overlay (as sdtv), upsize it first. Subpixel in this case can be set to default 0. 


So use sourceMask or overlayMask to exclude some areas from frames. Excluded area in mask must be pure black: PC levels for YUV. To load mask from file (black logo on white background): ImageSource("mask.png", pixel_type="Y8").ColorYUV(levels="TV->PC"). Then you are able to use the same mask in OverlayRender to break the transperency of excluded areas if you use opacity<1 or gradient and noise. 
To exclude black angles on borders use mask from OverlayRender with mode=6.

[random.next]

Hello, could this be used to auto align both T2 transfers, the 2015 and the 2017? Knowing frame adjustment varies from shot to shot.

https://fanrestore.com/thread-1202-post-...l#pid39431

[random.next]

random.next would have to implement some form of pre-multiply for that (I hope that's the right term), where basically the image is always pushed to 100% alpha internally where it is above 0% alpha. The mask then recreates the original border, eliminating the imperfection. That's what TIFF images with alpha masks do.

OR, make use of RGBA mode with an actual alpha channel. That way there would be no black background in the first place. But the implementation would have to likely use some form of premultiply anyway, so it probably doesn't matter.

The plugin only supports planar and RGB24 color spaces with mask images. RGB32 with alpha is not supported currently because it is completly different way to process transperency.
Inside the plugin OvelayRender uses  standard avisynth functions to overlay frames. Only OverlayEngine directly works with images when diff are calculated and some other support filters.

[deleted user]

I guess the 1px "merged" border and 1px near-black border are due to subpixel alignment and resize; cropping the final image by 2px (with an optional command) would help to have a border of one color (possibly custom )

It's because of resampling and is a desired effect to avoid aliasing. But it leads to problems when you're using a specific color as mask, because it creates gradients that no longer match the mask color. For example if the image is #FFFFFF and the background is #000000 and the mask color is #000000 as well, and you rotate/change subpixel alignment/resize, then resampling creates a continuum of middle values between #000000 and #FFFFFF which of course is stupid because only pure #000000 will function as mask. Transparent GIF images have the same problem, as they only have a 1-bit alpha channel (as oppposed to a full 8 bit alpha channel).

You could deactivate resampling, which would essentially result in that which we all know as "Nearest Neighbor" algorithm. As well all know, the result of that is unnacceptable for all but the most undemanding circumstances.