Restore missing corners?

[bobbster574]

hello!

I've been restoring some Dark Knight footage and as its an open matte + widescreen overlay there are inevitably missing corners. so far the overlap has been enough that i'm happy just cropping out the corners but there's one shot in particular that i'd like to at least explore the possibility of restoring the corners in some capacity.

I've done some research but mostly drew blanks.

There's the ambilight thing which uses the existing image and seems to blur/scale it to attempt to fill the rest of the frame. it does a decent job for empty sky where its all basically the same colour but as soon as some detail enters the corner area it becomes quite noticable imo. i dont think its really meant for any kind of restoration tho (more as just a way to fill a frame without just leaving it black)

I also came across AI in/outpainting. idea being use AI to fill the gaps. seems interesting, but kind of arkward to get working. for starters, filling in corners is neither true inpainting nor true outpainting. you got 2/4 sides where there is info, the other 2 theres nothing.

had a go using a couple inpainting models for ESRGAN, kinda promising for tiny corners, but beyond that it doesnt really help. understandable, im going beyond what the model is made for. and again, works well when theres no detail, but detail approaches the corner, it freaks out.



you can see in the bottom left and sorta on the top right on the ambilight where details are skewing things. and ofc on the ESRGAN the bottom corners are very noticable.

I couldnt get AI outpainting to work on corners. found a website that did an ok job on a single frame but i havent seen anything thatd work on a batch of frames. and also after experiementation i dont think itd be consistent enough to be any good.

and i couldnt find anything else on the internet that would help.

the other option that jumps out to me is manual compositing which i would totally try if i knew what i was doing. plus i dont want to spend too much time fumbling on this side quest right now.

and so i thought id ask around for ideas and thoughts and stuff bc this would be helpful for not only my current project, but potentially future projects, and also other ppls projects

[SHM]

Hi.
AI tech is not there yet.
I've lost count of the times I've researched this for Russian hybrids.
It will be, as Adobe is making huge progress in this field.
Maybe soon there'll be tools for video
Keep it up!

[mlnl]

yeah this is kinda just how it is unfortunately. corners are basically worst-case scenario for this stuff.

ambilight is doing exactly what its meant to do, which is “dont leave black bars”. once theres actual detail near the edge it falls apart fast, so youre not wrong there. its not really restoration, just a visual hack.

AI in/outpainting sounds good until you actually try it. corners confuse the hell out of the models because like you said, its not really inpainting or outpainting. they dont know what rules to follow, so as soon as detail hits the corner it starts inventing garbage. ESRGAN especially will just go nuts once you push it past tiny areas.

even if you get one frame looking ok, consistency across frames is the real problem. stuff crawling or warping in the corners is way more distracting than just cropping it out.

as far as i know theres no secret tool youre missing. ppl have been asking this exact question for years and the answers are always the same:

* crop it
* blur/fill and hide it
* or do manual comp work

manual compositing can work, but its a time sink and you basically need to steal info from nearby frames and warp it in, or paint it by hand. totally not worth it unless its a hero shot.

honestly cropping +a bit of grain or darkening the corners is probably the least painful option. looks boring but stable.

if someone had a clean automated way to fix corners properly, everyone doing restorations would be using it already. so yeah, youre not dumb and youre not missing anything obvious.