Tune colors and luma levels in one video to look similar to another video

[zee944]

I regularly do projects where I have multiple cuts on multiple prints of the same movie, and I marry them into one.
Sometimes it's quite hard to fit them together and make them look like they're from the same source. Setting the right luma levels and right colors are especially tricky. This is how I do it recently:

At first, I try to adjust gamma or contrast to achieve similar luma levels to the reference footage.

Levels(0, <gamma level>, 255, 0, 255, coring=false)
ColorYUV(cont_y=50, cont_u=50, cont_v=50)

It's often not satisfiing, so I've to go the more difficult way. I start VirtualDub, load "gradation curves" filter (by Alexander Nagiller), and draw an arbitrary levels curve in "RGB weighted" mode. Drawing the artbitrary curve is the most time-consuming part of the whole method.

http://i179.photobucket.com/albums/w286/zee944/gradation_curves.jpg

I save it as an .amp file and I load it in VirtualDub with GiCoCu filter.

ConvertToRGB32
MergeChroma(GiCoCU("arbitrary_levels_curve.amp",photoshop=true).ConvertToYUY2,last.ConvertToYUY2)

Setting up the colours is tricky too. I usually compare the reference video with the new video by looking at them and the values RGBAdjust(Analyze=True) shows. In a few cases it's enough to set the same R, G, B values in the new video. But this is rare. In half of the cases the solution is the reference values with the same multiplier. So if the reference R, G, B values were 100, 70 and 140, then 130, 91 and 182 (multiplier = 1.3) will maintain the same relation between the colours and would work. In the other half the cases, slightly different values are the good ones. To sum it up, the original RGB values are good for a take-off, the rest needs lots of trying instead of calculating.

RGBAdjust(1.232, 1.058, 0.977)

Sometimes I adjust saturation, too.

Tweak(sat=0.8)

An example:

http://i179.photobucket.com/albums/w286/zee944/TD_sample_original.jpg
http://i179.photobucket.com/albums/w286/zee944/TD_sample_new.jpg
http://i179.photobucket.com/albums/w286/zee944/TD_sample_new_adjusted.jpg

Question to those who are experienced in this kind of thing:

1. The most time-consuming thing is to draw an arbitrary levels curve. Is there a way I could (half-)automate it? If you have a 'reference' frame and the same frame from another print, how would you do it yourself?
2. Can the optimal color balance be foolproofly calculated? A larger bare surface can ruin my simple calculation. ColourLike() (http://forum.doom9.org/showthread.php?t=96308) is for the same goal, but I could never get it to work, and just by judging from the thread it isn't free of side effects.

If I can't improve it, I hope at least this will give a few ideas to others.

Thanks for any input.

[mikeytown2]

Interesting problem, would any of these histogram functions improve your workflow? http://avisynth.org/mediawiki/Histogram
Using levels/color2 mode with ColorYUV http://avisynth.org/mediawiki/ColorYUV


a=ImageReader("TD_sample_original.jpg",0,0,pixel_type="RGB32").ConvertToYV12()
b=ImageReader("TD_sample_new.jpg",0,0,pixel_type="RGB32").ConvertToYV12()
c=b

b
ColorYUV(gain_y=32,cont_y=32,cont_u=64,off_u=-4,cont_v=96,off_v=4)
Tweak(0,0.8)
Histogram("levels")
b=last

a
Histogram("levels")
a=last

c
Histogram("levels")
c=last

StackVertical(a,b,c)



Then use ColorBalance (http://forum.doom9.org/showthread.php?p=1180090#post1180090) with colors2


a=ImageReader("TD_sample_original.jpg",0,0,pixel_type="RGB32").ConvertToYV12()
b=ImageReader("TD_sample_new.jpg",0,0,pixel_type="RGB32").ConvertToYV12()
c=b

b
ColorYUV(gain_y=32,cont_y=32,cont_u=64,off_u=-4,cont_v=96,off_v=4)

ConvertToRGB24()

ColorBalance(last, \
[<separator="Shadows">] \
[<"S: Cyan-Red", -30.0, 30.0, 9.4>],[<"S: Magenta-Green", -30.0, 30.0, -5.7>],[<"S: Yellow-Blue", -30.0, 30.0, -17.0>], \
[<separator="Midtones">] \
[<"M: Cyan-Red", -30.0, 30.0, -11.2>],[<"M: Magenta-Green", -30.0, 30.0, 1.3>],[<"M: Yellow-Blue", -30.0, 30.0, 2.4>], \
[<separator="Highlights">] \
[<"H: Cyan-Red", -30.0, 30.0, 13.2>],[<"H: Magenta-Green", -30.0, 30.0, 0.0>],[<"H: Yellow-Blue", -30.0, 30.0, 1.9>], \
Select([<"Preserve Luminosity", 0, 1, 0>], false, true), \
Select([<"Clone Gimp", 0, 1, 1>], false, true), \
Select([<"16 Bit Color", 0, 1, 0>], false, true)) \

ConvertToYV12()

Tweak(0,0.75)
Histogram("color2")
b=last

a
Histogram("color2")
a=last

c
Histogram("color2")
c=last

StackVertical(a,b,c)


Lastly you can see that the white lights need to be yellow, and this also shows up in the color2 histogram. Still not sure what the best way to do this is, need to color it somehow.


I'm assuming that you have tried just about every filter here
http://avisynth.org/mediawiki/External_filters#Levels_and_Chroma


Setting curves can give very good results, it just takes time as I'm sure you have found out.

[Mug Funky]

that's a pretty good result there actually.

i think the big problem is one of these looks to have come off a release print, and the other from an intermediate negative film.

also the middle one is waaay too saturated.

just for fun i'm plugging away at this in photoshop, but it's a tricky one, especially without my trackballs (photoshop sucks for matching colours)

[edit]

one hint is that you need to pull out at least 50% of the saturation (probably more), and add a bucketload of red in the gain. the blacks need a dash of blue, and the highlights need to be selected and brightened toward orange.

[2Bdecided]

Is 8-bits a bottleneck for this kind of problem? Where one colour changes to another, some of the edges look a bit cartoon-like in the "adjusted" version.

If the geometry of the two images matched perfectly, you could calculate a pixel>pixel RGB mapping, and then use some kind of curve fitting to generate smooth R, G and B mapping functions.

If the geometry only matches approximately, then maybe working from the (smoothed) histograms and trying to calculate the shifts needed to make the two is the way to do. Which isn't a very helpful thing to say, as it's exactly what you've asked - an automated method of doing what you're doing by hand.

Cheers,
David.

[zee944]

Interesting problem, would any of these histogram functions improve your workflow?

The key is in the histograms, I guess; but without some sort of automatic matching I can only use them as a guide. This ColorBalance thing is interesting, but if I'm not mistaken, you're basically do what I do: manually adjust the colors.

I'm assuming that you have tried just about every filter here
http://avisynth.org/mediawiki/External_filters#Levels_and_Chroma

Haven't tried all of them, since most of these filters are not suited for this kind of problem. Only HistogramAdjust and ColourLike use external data (frame) to match the clip for. But the same frame for a full scene? It doesn't work. :(


one hint is that you need to pull out at least 50% of the saturation (probably more), and add a bucketload of red in the gain. the blacks need a dash of blue, and the highlights need to be selected and brightened toward orange.

This fits my current color balance settings (seen on the third pic):
RGBAdjust(1.367, 0.882, 0.600)
Tweak(0.7)


Actually I'm satisfied with the sample result too (except for the too much contrast), but it took hell a lot of time to draw the right luma curve.

If the geometry only matches approximately, then maybe working from the (smoothed) histograms and trying to calculate the shifts needed to make the two is the way to do.

The geometry matches only approximately in these cases; the frames are always from different prints. Some kind of auto-histogram matching would be interesting. Actually, it wouldn't be a solution either. Somehow the difference between the two frames' luma/channel curves should be calculated and saved; then those curves should be applied to the full clip. This way the colour balance would remain consistent for the full length of the video, which is essential.

One could probably benefit from the picture content being (approximatively) the same; but I'm dreaming now.

[JohnnyMalaria]

What you are trying to do is correct the white balance for indoor lighting. The correct way to do this is to add/subtract an offset on each of the chroma channels. This is how camcorders change the white balance in the first place. Don't work in RGB space. To determine the offset for each chroma channel, calculate the average value. Subtract this from each pixel. You can use just one frame as a reference or correct each frame independently.

i.e., new chroma = old chroma - average of all chroma values in reference frame ( + the zero offset, e.g., 128 for unsigned 8-bit representation)

Our software uses this method. I'm sure AVISynth has all the necessary mathematical functions to do it, too. An example of before and after (look at the picture of the garage scene):

http://www.enosoft.net/products/enodvproc/How-To/WhiteBalance/jpg/Garage%20-%20Scene%20Adj%20Split.jpg

The right hand side is the original with a green cast. The left hand side is corrected using the method above.

HTH

[Gavino]

i.e., new chroma = old chroma - average of all chroma values in reference frame ( + the zero offset, e.g., 128 for unsigned 8-bit representation)

Our software uses this method. I'm sure AVISynth has all the necessary mathematical functions to do it, too.
I think this is what ColorYUV(autowhite=true) does, except that it doesn't use a reference frame, so the documentation (http://avisynth.org/mediawiki/ColorYUV)has a caveat:But be careful - it isn't very intelligent - if your material is a clear blue sky, autowhite will make it completely grey!
It could certainly be done (including reference frame) using ScriptClip and AverageChromaU/V.