Color matching - some considerations

[spoRv]

Color matching: recolor a video source (usually an higher quality version), to match a second video source used as reference.

Regrade (aka recolor) a video source without using a reference, but only following personal tastes, indications, "feelings" is not color matching, thus projects like these can't be called preservations or restorations, but just fan edits (fanfix, in particular).

As far as I know, at the present, there is NO way to regrade a video source to color match 100% a reference source! (retaining all the image quality and parts of the higher quality one)

There are several techniques to achieve this task, and I'm trying here to explore some of them; if you are aware of any other ones, please let me know!

Merge chroma

This is the best way to achieve (almost) perfect results; just align spatially and temporally the two sources, use the luma of the first source and chroma of the second (reference) source.

Pros: colors are not simply matched, but they are the original ones!
Cons: often the two sources don't match; spatially, they could have different sizes, different framing, rotated and/or distorted image; temporally, if there are some frames missing from the reference, it will be obviously impossible to match those frames. Also, the luma of the two sources could differ greatly in contrast, brightness, gamma, and could lead to wildly different final color appareance.

Color matching tools (video)

There are many softwares around that try to match, mimic the colors of a reference source and apply that to another source; again, you need to align spatially and temporally the two sources.

Pros: usually easy to use and automated, they could match the colors quite exactly
Cons: different softwares/plugins use different methods to accomplish the task, so they have different behaviours, but none could match 100% every frame, every shot, every source; where one is perfect for a given shot, fails on another, and vice versa. Also, some of them require the two sources to have the exact framing and size.

Color matching tools (image)

There are other softwares that do the same, but only to images and not video; you could find an image from a film cell, or a screenshot, then find the corresponding frame of your source, and then match the two.

Pros: this could be the only way to match two sources, if the reference video is not available.
Cons: without best consistent reference images, it's impossible to reach a good result; even with such images, hardly the matching setting obtained from a single image could be used to regrade perfectly a whole movie, but could be useful for a given shot or scene; of course, the more the images to match, the more close the matching will be.


Compare & regrade by eye

This is by far the least precise way to color matching two sources, but sometimes the only one if the previous ones are not possible, or they give not good results.

Pros: you don't need fancy softwares and a steep learning curve, just a simple editor which can allow color settings.
Cons: even if used by editors with perfect color vision and perfectly calibrated displays, it's quite difficult to make a good color match of two sources only by eye.


I noted that sometimes a mix of the previous techniques leads to the best results. Usually, I use only video matching tools, and combine the results to obtain the best final matching. For example, working on my last project, I discovered that doing a second pass (regrade a regraded source), using a different plugin, could help to improve the color matching. Also, the latest technique is to use the merged chroma source as one of the reference, and average it with other color matching regraded versions to further refine the results.

Conclusion: even if it's (still) not possible to color match two sources perfectly, with good will, experience, knowledge, a lot of time and patience, it's possible to catch the "spirit" of the color grading of an inferior quality source, and apply that to a superior quality source, without noticeable problems; almost no one will be able to spot the colors differences only watching the regraded version, unless it is put in direct comparison with the reference source..,

[spoRv]

After more than three years working, testing, failing to color match two sources perfectly, now I can say something more about automated matching softwares. I use just Avisynth for this task, so they are all scripts (filters, plugins, call them how do you like best); I will continue to call them scripts, as they are a series of written "instructions" in a text file, more or less like a computer program.

At the present time, I use about a dozen different scripts - some discovered on the net, others created from scratch, or made using a combination of the former ones. As I wrote in the main post, it's impossible to reach a 100% color match; at least, I haven't achieved that task, and I made innumerable tests. But I can say I'm refining the techniques, and slowly I'm improving the final results.

It seems that the best scripts obtain a very close match in almost all shots, while they produce artefacts in few frames; these usually appear as strange colored mosaics or small random pixels, encountered especially in blown out whites. Other problems that affect these scripts are "pulsing" image, where brightness varies from a frame to another. Another one is encountered during black shots, where the matched result have a raised brightness, so it's really a dark gray, not black anymore.

There are other scripts with no problems at all, but color matching is very bad - usually they match very well just few shots, while others are left almost untouched, or they works well with some kind of colors and badly with others.

So, there are three solutions that I can think of...

First, use the scripts with no problems, but the color fidelity will be at best sufficient, while most of the time will be sub par - there is no point to get an artefact-free regraded version, where the colors are sometimes like the reference clip, sometimes like the untouched source, and other times a mix between them (in rare case it happens the result is different from both!)

Second, use the best matching scripts; result will be extremely good, very close the the reference clip, but problems would be noticeable - from barely to extremely... even if it could be a better solution that the first one, it's not the optimal choice.

Third, follow the second way, but replace the problematic shots which contain artefacts, with others obtained using other lower quality script. While even if this is not the perfect solution, it's the best choice I can think about.

Indeed, in a normal movie, with a length of about 90 minutes, there are around 130000 frames; with the best scripts, maybe just a dozen shots (two or three in the worst case scenario) would contain some frames with artefacts. As it's nearly impossible to replace just the few frames using a lower quality scripts - they will be too different - the only way to go is to replace the whole shot. Considering that fast action movies (and not only) use a lot of brief shots, instead long ones, each shot to be replaced will last few seconds - I noted that many films use even shots one or two seconds long... so, at the end, on 90 minutes, probably just a minute, if not less, would consist of lower quality matched footage, while the rest have very high color fidelity.

To be clear, when I talk about high color fidelity, I mean that, when compared side by side, matched version and original, they seems, if not identical, really, really close... I always use a percentage to give an idea, even if I'm aware that's really wrong, because there is not objective measurement used. But, when I see nearly identical colors on both, I can say the images are 90-95% (and up) closer. Even using lower quality scripts, often it's possible to obtain around 80-90% color match, while usually they stay around 60-70% and few times drop into lower percentages.

Long story short: it's impossible to get a 100% color match, using whatever method - unless you upscale the color reference source, or you are so lucky that the lower quality source used as reference perfectly matches, spatially and temporally, the higher quality one, so its chroma could be directly combined with source luma.

So, at the end, getting an overall result of, let's say, 95% match in the 99% of the footage, and 70% match on the residual 1%, is the best one that could be possible to obtain right now; that's just based upon my experience.

Following examples from "The Thing", from top to bottom: LD (used as color reference), good script (best fidelity, few artefacts), mid script (good color fidelity, sometimes bad contrast), low script (usually lower color fidelity and/or contrast), BD (untouched):













As you can see, even in the worst cases, the color "spirit" of the lower quality match scripts is a lot closer to the color reference than the untouched source.

[spoRv]

I worked a lot this last week on color matching... refining this, renew that, you know... now I got an even better solution, even if it's not perfect - still few artefacts in some rare cases - I'm pretty happy with it.

What I have finally learned is that new HD media (BD and HD-DVD in particular, but should apply also to HDTV and web download) have generally a better contrast - with some notable exceptions of course... usually I find that often both HD and SD have similar contrast, while sometimes SD goes wildly different, usually with too high contrast resulting in clipping high levels; so, I think the "golden spot" for many regrading is to use a contrast of 75% from HD, and 25% from SD; in this way, eventual clipping levels from SD would be fixed, while eventual higher contrast from HD (less likely) should be mitigated a bit using SD, and in most cases, as the contrast level is similar, no difference from HD or SD.

It's true that, using this "trick", sometimes colors have less fidelity, but another thing I learned is that SD, even when got colors closer to theatrical presentation than HD, is far from perfect - always knew that, but have many confirmations lately... so, at the end, the "spirit" of the SD color remains intact, but, using better contrast, final result is often much better, and almost always without ugly artefacts.

A couple examples below, from The Matrix; note that, in this case, BD has the high contrast level set too high, so the clipped whites are particularly bad and sadly not recoverable

[spoRv]

Lately I'm starting to think that, sometimes, when the old and new versions color grading is way different - usually older are warmer, often leaning also towards magenta, while newer are colder, often leaning towards teal/green/blue - the best way is to make a mix between them; I observed this with The Arrival and Aliens, while PDB in his thread noted the same about Blade Runner...

This of course could (and should) not be applied to any title, but, in particular cases, when one seems too cold, and the other too warm, probaly the middle ground is the best compromise - can't say it would be theatrically accurate, though... but I can't say neither the contrary, so...

[DrDre]

Hello there,

I'm not sure if you're aware of this, but I developed an algorithm, and a tool a while ago for color matching, based on matching the distribution of colors in a multitude of color spaces. A thread on originaltrilogy.com is dedicated to it:

http://originaltrilogy.com/topic/Color-m...128/page/1

Here's an example of a result I get for the latter example you posted:

[Image: N8Op0RY.png]

My favourite color challenge has thusfar been color matching the 16mm Song of the South to the 35mm Song of the South.

16mm:

[Image: RxK3tJi.png]

35mm:

[Image: ODhmAsh.png]

16mm matched to 35mm:

[Image: y2lUDeO.png]

Another interesting challenge was restoring color to a scan of a severely faded 35mm print of Star Wars, using a Technicolor print reference.

Before:

[Image: MFmOsW0.png]

After:

[Image: 1afYYUo.png]

[PDB]

Know your program very well, DrDre. It is a great piece of software. Thanks for making and sharing it.

[DrDre]

Thanks!!! I'm working on a version, that allows for batch processing, for those of us that don't have access to a program, that can handle LUTs.

[PDB]

Thanks!!! I'm working on a version, that allows for batch processing, for those of us that don't have access to a program, that can handle LUTs.

Oh that would be great! Batch processing is my number one feature request and could cut down so much time for me.

[DrDre]

One particular problem I struggled with, was obtaining smooth color gradients. The method of adjusting distributions in a large number of color spaces (up to 250) is very similar to getting a piece of clay to be the right shape. You push it too hard on one side, and you will get weird distortions on another side. So, you have to force the algorithm to use smooth gradients. It will take more iterations/color spaces to reach a good fidelity, but this will also in some cases fix crush and blowout issues, and generally prevents artifacts.

[Booshman]

I've used your software quite a bit and it's some sweet voodoo. It works so well and has saved me a lot of time dicking about trying to go the manual route.