ColorMatch 3D (new colormatching tool)

[deleted user]

Just something I was working on:

https://github.com/TomArrow/ColorMatch3D/releases



Give it an input and output image. Hit the red button, wait, then click the export button.

Benefits over DrDre's colormatch:
- Blazing fast. Depending on image size and your CPU, it can be less than a second
- Should be able to deal with more complex changes in theory due to a completely different algorithm

Downsides over DrDre:
- Doesn't generalize well. It will only match the colors in the image and no others, so applying the LUT to any other image will result in garbage. It might not even generalize well across an entire scene, so its use currently is to match single frames and maybe very similar frames.

Todo for the future:
- Try to integrate some interpolation option to make the 3D LUTS generalize better.
- Create an AVISynth plugin

The algorithm is childishly simple: Create an RGB cube in the size of the output LUT (32x32x32 atm). Take each color in the source image and create a vector to the color in the output image. Average all these vectors and place them in the RGB cube. Export RGB cube as LUT. Profit.

As you can see, the algorithm explains both the downsides and upsides. If I manage to do some interpolation (or maybe extrapolation is a better word), it would probably slow it down quite a bit, but maybe it could be made optional and controlled somehow, aka you could choose how much interpolation is to be done. We'll see.

Important note: Most of the options in the tool don't actually do anything, so don't bother with them. They are stuff I wanted to implement or might implement some day. The only option that really does anything currently is the "Aggregation Variable" being set to Absolute or Vector. Absolute is the old algorithm I had for months but it wasn't precise enough. Instead of averaging vectors, it was averaging absolute values, which gave decent results but in some situations the results were disappointing. The Vector algorithm is default now and solves that issue.

Edit: Since this is open source, feel free to contribute if you want to ofc.

[pipefan413]

Nice work! This could be very useful for inserting replacement frames to fill gaps and so forth.

[LucasGodzilla]

Did a little test involving a low-quality 70mm cell frames scan(?) of The Abyss and thought to post the results from Dr. Dre and TomArrow's Color Matchers with the same inputs at 250.

70mm & Cinemax Inputs

Cinemax

Regrade (Dr. Dre)

Regrade (Tom Arrow)

Although TomArrow's tool seems to get more appealing / accurate results in these two frames than Dr. Dre's, I notice banding in some areas (namely the helmets).

Don't have anything else to add really other than it's a nice starting point for something bigger @TomArrow, and I hope o see this tool built upon in the feature (whether it be by you or the community).

[deleted user]

@LucasGodzilla Ah yes indeed. Those artifacts I encountered before. I think that to get rid of them, I might have to do the interpolation I was talking about. Currently the LUT has a lot of "black spaces" in it at the edges of the used colors, those artifacts are probably when the tool you are using tries to interpolate using those "not set" values that it just sees as black (which technically they are).

Thanks for doing a test!

Edit: About the banding ... might have to do with those artifacts too or might just be a limit of the source? Worst case could add some noise before application of the LUT.

Edit 2: Did you actually do the matching with those little images? That might be the reason it didn't work properly since the downscaling ditches some of the colors necessary to form the LUT. I wonder what results you would get if you do it at full resolution. Should be fast enough to do that.

[Cactus]

So, I might be an idiot, but I can't get the program to do anything at all. It calculates a histogram, but the images doesn't change at all. Am I doing something wrong?
There are the images I've used:
https://mega.nz/file/4NlTXAyC#V1eB2g_zR-...egi2y6Ak9o
https://mega.nz/file/RElD2CQD#_6JPPYtnPv..._sbZ2APeR0

[deleted user]

So, I might be an idiot, but I can't get the program to do anything at all. It calculates a histogram, but the images doesn't change at all. Am I doing something wrong?
There are the images I've used:
https://mega.nz/file/4NlTXAyC#V1eB2g_zR-...egi2y6Ak9o
https://mega.nz/file/RElD2CQD#_6JPPYtnPv..._sbZ2APeR0

Pardon for not being clearer. You need to export the 3D LUT and apply it in a suitable program. It doesn't show a preview.

[deleted user]

@Cactus

I tried your images. First off, the way you made them would likely not work because they weren't properly aligned.

Here's the aligned images:


First, here's the result I get: Not very good. Looks weird as hell:



My explanation for why the match is this bad is that the second image has different areas graded differently, aka some areas are darker/brighter compared to the first image. Which means that there cannot be a perfect 3D LUT. In cases like this DrDre might give better results I think. However I did a little experiment and tried to equalize the brightness of the first image to match the second:



While the result is still far from good (I'm not sure something like this could even be done perfectly), it looks a lot more natural now, tho still not as good as you'd hope:

 

In short, this method isn't really very suited to matching grades that have had power windows used in them, as it will end up averaging things that don't belong together and such, since it takes everything very "literal".

[Cactus]

Thank you, TomArrow. I've been working on an extended version of Danny the Dog/Unleashed but came to a standstill a while ago. Not only because of some vastly different grading, but with the American version also having some artificial camera movement added. I shall continue my quest.

[deleted user]

New version out: https://github.com/TomArrow/ColorMatch3D...s/tag/v0.2

Added a simple interpolation algorithm. It's listed as two separate ones, but it's really just one with slight differences. Single Linear and Dual Linear. Single Linear will give acceptable results with the Vector mode. Dual Linear is pretty useless so far, I suppose the idea behind it wasn't that great.

It probably won't make a big difference when it comes to a single image, but it definitely generalizes a bit better now. In the past you would get big black patches where the image had colors that weren't in the image used for matching. Now you get at least ... something. The further away the color is from the image used for matching, the worse the result gets. But I think it might just be good enough for matching a whole scene now, tho I'm sure there are exceptions to that.

As for speed... in Debug mode it was pretty slow (about 6 seconds per image) but in the Release version (the one uploaded) it actually only takes about 2 seconds per image (depends on size of the image too), so I'd say it's okay. And it's optional, so you can decide whether to use the interpolation. There's definitely a lot of room for optimization I think but until the algorithm performs really well, I figure it's best to postpone that.

Edit: Secret tip, the Single Linear + Vector mode, combined with LUT smoothing in 3D lut creator can give pretty pretty okay results already. Maybe I'll add some kind of smoothing too to the interpolated parts.

[deleted user]

New version.

https://github.com/TomArrow/ColorMatch3D/releases/

Skipped the one in between. Last version added the option to work in CIELab instead of the sRGB of the images that are to be matched, the new function is a brightness equalization similar to what I did in the post above, but now you can just click on the option and have it done, whilst setting a blur radius for the equalization. It gives rather smooth results, but a downside is that the contrast of the reference isn't necessarily respected too well. I'll have to come up with some clever idea to further improve the results.

Smoothing the result is not implemented yet. Sadly I am right now fighting with constant BSOD issues on my PC whenever I work with Visual Studio, so it might take a while until I feel like touching it again. So far, enjoy.

Result I get with 15px equalization radius of the image above:


As you can see, the colors are alright, but the contrast is more or less ignored. Will need to come up with something clever to fix that.