[Idea] Regraining Terminator 2! (Proof of concept/experiment)

[deleted user]

Bear with me, most of this is yet science fiction at this point, as will be clear quickly. Maybe, to a certain extent, it's also just me making an overly elaborate joke.

Okay, here goes: There is a theoretical possibility to regrain Terminator 2, using the 2015 Lionsgate release (original grain!), while retaining the color and most of the better detail of the new release. This thread is supposed to serve as a proof of concept on a single frame.

I took a screenshot of the same frame of the 2015 Lionsgate release and the 2017 Remaster from caps-a-holic.

- Loaded in Photoshop. 
- Removed black borders (to not confuse auto-align, as the cropping is slightly different). 
- Auto-aligned.
- Slightly nudged to improve align on face.
- Shake Reduction filter on 2015 release (essentially deconvolution) to bring out more of the grain structure
- High Pass filter on this shake-reduced version at around 2.8 px. 
- Auto-contrast on the high-passed 2015 screenshot, layer mode set to Overlay.
- Fine-tuned opacity a little.

Okay, a few explanations. 2.8 pixels highpass was chosen because that seemed the value at which a higher value no longer better retained grain. Now, the result of this is the new version with the old grain overlaid. 

Problem: High pass preserves all other details in that frequency range too. This is a problem because that information now exists both in the overlay and in the 2017 transfer, so it exists "doubly", which results in a sharpening and edge-enhancing effect that is quite ugly. I'll propose a possibly better solution later, but for now here's how I solved it for my little experiment:

- Blurred 2017 release at roughly 2.8px (did a little fine tuning with the slider). Now the edges under 2.8px "wavelength" were taken purely from the 2015 release while everything with lower frequency was still from the 2017 (thus retaining coloration for the most part etc.)
- Problem: Now all the fine detail from the 2017 release is lost.
- Solution: Copied sharp 2017 release on top of this whole mess and did a high pass at around 1.2px or so (where the 2015 release no longer resolves any detail). Set to overlay. Fine-tune opacity a little.

Result: Wavelength down to around 2.8 pixels: from 2017. Between 2.8 pixels and 1.2 pixels or so: From 2015 (includes grain and some mild fine structure). Under 1.2 pixels and microdetails: 2017 release (for that gain in clarity!). 3 "layers" in total.

May I present: 2015 Release, 2017 Release, Hybrid!

[Image: a88106674747673.jpg] [Image: c40b60674748093.jpg][Image: 1a8751674747893.jpg]

For your convenience, I also created a screenshot comparison for 2017 vs Hybrid: http://screenshotcomparison.com/comparison/125219

Okay, some problems: 
- Frames don't align perfectly, as you can see. 
- Some ringing from the deconvolution (at edges of frame and in lamp and possibly other places).
- It doesn't look absolutely perfect. I didn't really spend more than 10 seconds fine-tuning each value. I think the result could be improved by someone more dedicated than me to find the sweet spot and use the right filters etc. (I used Gaussian Blur for blurring the 2017 low-frequency "base"). There is some mild kind of "ghosting" or vaseline-kind of thing.
- Grain doesn't look ideal (lacking finer detail as it's an old remaster and also some remnants of compression artifacts). Could complement with digitally generated grain.
- Method is kinda not super elegant

Good sides:
- Fine detail from 2017 Retransfer mostly retained.
- Colors from new transfer retained
- Picture no longer looks terribly DNRed and looks kind of enjoyable. Face looks much more "human" now, some of the organic structure has returned from the 2015 release.
- Restored the authentic dust speck on the left side of the duct!

Now here comes the part where it's just science fiction:
- Auto-align might not work ideally on all frames. Plus, it distorts both layers, not just one, and needs slight manual adjustment. Stutter would likely result in video.
- Other scenes may be more distorted, not fixable for auto-align (that is where the recent thread about aligning sources may come in handy).
- Deconvolution in Photoshop is pretty slow. We're talking multiple seconds per image. Would take some dedication. Although the AVISynth plugin FQRestore might work too, which is faster (used it on another project already).
- Frames don't align perfectly. Would need to discard parts of the frame, likely hand-tuned to each scene as the framing is likely to vary from scene to scene in both releases.
- Fine-tuning that looks good may vary from shot to shot.
- Likely not works well for scenes with "fixes" like switched stuntman's face/fixed window.
- Would just overall be a huge pain in the ass without some decent automation and require an unrealistic amount of dedication. (I don't have that!)
- Would only work for Theatrical, as the others haven't been remastered (or released so).


Suggestion for improvement of the method:

Remember when I wrote I would give a better suggestion earlier in this post? This is it.

I simply used a high pass filter. That filter is "dumb" in that it retains all structure too, necessitating that weird workaround with 3 layers I did above, to avoid an edge-enhancing effect. I wanted to originally use a more clever fancy approach:
- Denoise 2015 version in similar manner as 2017 was denoised (Topaz Denoiser seems closest)
- Use difference-mode between original 2015 and denoised 2015 to get "only the noise".
- Apply "only the noise" to 2017 version. 
- Fine-tune.
- Et voila.

I hope you got a good laugh out of this. Cheers.

[Synnove]

Why not just get some samples of Gorilla Grain and overlay them upon the transfer? The "grain" in the old transfer might not even be original to the film (either because the grain was removed for grading, or because it was convoluted by the H.264 encoder, etc). Plus, important to remember that the grain has already been chewed up by the encoder; prob won't take to well to going through a second time.

Nuke has a really good node called "f_regrain" that allows you to add grain to footage. It gives you control over the size of the grain, how it effects each RGB channel, how it effects shadows, highlights, and mids, how much there is. Given that it's Super35 the grain would be a little larger than normal, and would definitely be coarser in the shots with optical compositing (though I think digital compositing was used for most, if not all of the film?).

For your difference method, I assume the edge enhancement is no longer an issue?

[deleted user]

@Synnove

Yes, of course that is a possibility. But the thing is, with overlaying grain you don't get back the actual detail that was swept away through the DNR. That was the whole idea behind this. Whether that makes an actual visible difference, that would remain to be seen I guess, I haven't tested that. And of course there is the pedantic question of authenticity . As said, this is pretty unworkable unless someone manages to find a solid auto-align method. So your regrain method would be the go-to approach, but whether it gives good results has to be tested. Feel free to take the screenshot I uploaded and apply your method to it and make a screenshot comparison, that might actually be pretty interesting.

I am not quite sure why the old one shouldn't be the original one. What would be the point of removing it for the grading? (I don't know much about 35mm film tech, bear with me!) And yeah, the encoding artifacts are clearly an issue as can be seen. Though I think that H264 with its psy-rd may actually improve on it a little bit, weird as it sounds. It could introduce further micro-artifacts that may even hide the imperfections in this.

And you could always complement this with a tiny bit of fine-grained Gorilla Grain / other plugins / digital noise / dithering type. I also think that it would look pretty good in motion where you can't see the tiny issues so much.

The difference method was my original one that motivated me because it seemed so clever in my head. But then it seemed like too much work to organize a working degrain/denoise plugin (I am lazy!), so I went with the simple route. Whether that would still be an issue depends on how the algorithm works I guess. I would assume that it would definitely be less of an issue though.

Edit: I assumed you meant removing the grain for 35mm grading, but that makes even less sense because DNR is digital (it's in the name). *confused*

[Synnove]

Based on the cursory glance at caps-a-holic, I'd wager not much detail was lost in the degraining process; the transfer looks incredibly crisp and free of edge enhancement. Even if that was the case, presence of grain doesn't necessarily equal detail as the difference between the fuzzy grain laden transfer and the crystal clear degrained transfer shows.

In many grading workflows, noise and grain are removed before digital color grading takes place because it can act up and overly complicate the grading process, since you're limited by how the grain would react to the changes you're making. Ideally, the picture is regrained at the end of the process in a way that makes the grain levels consistent across the footage... or at least, that's what I've been told.

I'll definitely take a shot at it; if you look in my alignment method thread in this subforum you'll see my efforts at regraining a degrained transfer (though other stuff was done to it as well). I'm also looking for an alignment method so as to combine the best aspects of multiple transfers. I've  a few promising leads.

[deleted user]

You're right, it has a lot more fine detail than the old transfer. Absolutely no doubt. But, this is usually edge detail, not texture detail. The texture detail, while not visible in a screenshot, may, through a form of dithering (many frames with grain dancing around the full value), reveal more plasticity to the eye even though it may be hard to isolate it.

Either way, I'm happy to be proven wrong by a comparison. If you do, maybe make sure to take the screenshot I uploaded, not from caps-a-holic (mine is warped through the auto-align and wouldn't be a good direct comparison to the caps-a-holic one). Obviously a regrained shot will likely look better than the raw DNRed thing, but I'd be curious whether it also looks better than the actual grain.

Maybe you're right about the process in the industry. I honestly don't know. Interesting topic for sure.

Edit: I could also imagine that the degraining in the industry may work similar to my suggested idea here - to degrain, take the difference, and apply the difference back after the grading process. That would seem like a cool idea to me.

[spoRv]

@Tom: it's possible to grab the BD grain structure, and apply it to another source; just find out a part with black image (that actually contains grain), extract it, loop the grain (also flip it vertically and horizontally to extend its duration).

More details on the process, if needed, would follow on Monday.
I did it with Aliens, and result is nice. Of course, using a readymade grainplate is easier, but with this method you can obtain what should be the original grain - if it was not added in post, of course!

[CSchmidlapp]

Wouldn't the grain structure be variable shot to shot on the OCN due to the fact it's built from multiple reels?
In that i mean the same stock vary from reel to reel produced.
Would trying to replicate the look of an inter-positive, inter-negative or release print be restoring the characteristics of what many would have considered dupe noise?

[Synnove]

http://screenshotcomparison.com/comparison/125299

I apologize that I couldn't set up a comparison to your original photo; I'm using the non-commercial version of Nuke which has a 1920x1080 export size limitation.

Here's the setup: https://i.imgur.com/MJBi6cW.jpg

This allows you to define:
-The shadow, mids, and highlights of the image,
-Select how much grain to apply to each of those areas,
-The size of the grain.

And there are separate pipelines for RG and B since blue grain has it's own qualities (notice in the screenshot that the blue grain is a little larger)

[deleted user]

Hah!

Well, I still kinda like my version more. Though I can't tell whether that's because it's actually better or because it simply has more grain. I will admit that if you added as much grain as I did, I'd possibly not be able to tell a difference. That said, your version definitely doesn't look bad and is a step up. And has the benefit of actually being workable.

[CSchmidlapp]

If im ever dealing with a noisy source, I can often get away with just cleaning up the blue channel.
And I often re grain it to help cover the compression artifects and it does seen to help with the perceived quality.

I do think the regraining you are doing looks a bit heavy, The new T2 is straight from the negative.
Like I asked above, it depends on what your trying to replicate?