New Filter: WhiteBalance

[SomeJoe]

My first attempt at writing an AVISynth filter, so I hope I did it right. :D

This filter can correct the white balance of video that was filmed with a camera where the white balance was incorrectly set or didn't have an adjustment. Especially useful for home video filmed indoors on consumer camcorders, or filmed outdoors under overcast skies.

Has some features that result in more accurate white balance adjustment that attempting to use RGBAdjust() or ColorYUV().

Available here (http://pages.sbcglobal.net/wilsondr/WhiteBalance100.zip). Documentation included in the .zip archive.

Please post any comments, good, bad, and ugly. :)

[communist]

Thank you for your filter :)
Either I'm using it wrong or it is very picky / accurate about which values to turn to white or not. The following is a construction model and it was completely white. Using ColorYUV(autowhite=true) it is just as it should be:
http://www.stud.uni-goettingen.de/~s304280/D9/wb_comp_source.jpg
http://www.stud.uni-goettingen.de/~s304280/D9/wb_comp.jpg
Script

AviSource("wb_sample.avi")
WhiteBalance(r1=147, g1=180, b1=244)

Tweaking the value I can get that green smear away but then I either end up having a blue or red tint.

Would it be possible to add an option to input an (rectangular) area (from one frame) with near white / black pixels that are then averaged - so one can specify a block as a sample and the filter will work with the values obtained from there to calculate the Whitebalance?

[SomeJoe]

Hi communist, thanks for trying the filter.

I think definitely on this footage that you're going to need to specify two points with maximum brightness separation for the filter to work as intended. I took your source and came up with the following:

The lightest area of a neutral object appears to be an interior wall of the model, with RGB=144/182/245.

The darkest area of a neutral object appears to be in the shadow of the model on the table surface (extreme left side of the picture), with RGB=77/115/152.

Try:

WhiteBalance(r1=144,g1=182,b1=245,r2=77,g2=115,b2=152)

On your other question, I'm wary of trying to do some type of automatic selection of values, similar to ColorYUV's "autowhite=true". Obviously, (especially on footage like your source), it's difficult to program the logic necessary for the filter to accurately determine what object/surface is supposed to be white. It's not necessarily the object with the highest luminance.

[hanfrunz]

@SomeJoe:

nice filter! You could add a simple GUI (Look here) (http://forum.doom9.org/showthread.php?t=63575) with a color picker. Will you publish the sourcecode?

regards,
hanfrunz

[SomeJoe]

Hi hanfrunz,

Thanks for the link. A GUI might be very nice for this filter. Unfortunately, my programming roots are "old school" (8-bit assembly for microcontrollers and Pascal), so object-oriented C++ is a bit new to me. It may take me some time to figure out.

I've uploaded a new .zip file which includes the source. It's quite straightforward -- the structure is lifted straight from SimpleSample. I commented it reasonably well, but there are a few places that could use more comments.

[sh0dan]

Since you haven't published the source, I cannot comment on your algorithms.

"autowhite" is very simple. It makes sure that the average u/v is 128 (grey). This is far too simple for proper white balance correction. But my experiments on RAW photo development has given me some insights into WB. Unfortunately all of it is in RGB-space, so I don't know exactly how it translates into YUV.

However, proper white balance has to be done on linear space data, that means that the sRGB gamma correction has to be "undone". Basicly this gives an algorithm of:

apply gamma(0.45) -> WB correction -> apply gamma(2.2)

This means you either have to work on float data, or 16 bit integer data. For proper WB balance, it is actually also a two-step process:

Remove WB compensation -> Apply new WB

So basicly you need to know the old WB. This is needed, because WB is not a linear/math based compensation, but a table of values based on measured values.

[SomeJoe]

Hi sh0dan,

The source is included in the .zip file. The documentation also contains a detailed explaination of what the filter is doing.

Apparently, I've either not done enough (or done incorrect) research on white balance, or you and I aren't speaking of the same thing. Nothing I uncovered when I was writing this filter mentioned anything regarding gamma correction.

As I understand it, gamma is simply a measure of the non-linear brightness associated with monitor/TV phosphors, thus is a parameter affecting luminance. My WB filter doesn't make any changes to the luminance values of the frame (i.e. all Y values of the changed frame are identical to the source). So I'm not sure how a gamma adjustment/compensation would be necessary.

Nevertheless, I'm feeling more and more unsure of the filter. Communist (above) hasn't replied back regarding my suggestion for his material, but I was quite surprised that it didn't work well for him. In my tests, I did not have any of the issues he's showing, although his source material is a fairly extreme example. Perhaps there is a fundamental reason why the algorithm I'm using isn't correct, and if so, I'm at a loss. I was taught many years ago to white balance a video scene that was being shot with a video camera by using a vectorscope. You center the vectorscope's dot onto the white point by adjusting the white balance on the camera while it's pointed at a white card lit under the same lighting you're going to shoot under. The filter is just an electronic implementation of that.

If you have further insight, please let me know or point me to some references.

[communist]

Sorry for the late reply.
I tried your values but not much luck with them. The problem with the footage might be that it is also not very stable over the time. So it might look good in a few frames but bad in others.
The best I got out of it is by using:
WhiteBalance(r1=160,g1=190,b1=220)
http://www.stud.uni-goettingen.de/~s304280/D9/Wb_3.jpg

If you want you can take a look at the source (http://www.stud.uni-goettingen.de/~s304280/D9/wb_sample.avi) (2,3 MB HuffYUV).

[SomeJoe]

Sorry for my late reply. ;)

Thanks for your source, I downloaded it and I'm doing some tests.

There may be a bug in my math routines ... I will be checking them thoroughly ... stay tuned ...

[Boulder]

Any news on this filter? I've got some stuff in which the white balance doesn't stay stable but the coloring goes from bluish to white and sometimes from yellowish to white to yellowish etc.

Or if anyone would like to have a go, I can upload some clips for testing.

[Fizick]

try
http://acobw.narod.ru/acobw.html

[Boulder]

I did but it crashed VDub 1.6.14 when previewing. Also the GUI is total gibberish as it's in Russian and the Cyrillic symbols don't even show up correctly on my PC :(

[smok3]

You center the vectorscope's dot onto the white point by adjusting the white balance on the camera while it's pointed at a white card lit under the same lighting you're going to shoot under. The filter is just an electronic implementation of that.
the original white balance on the camera happens after the CCD - in the electronic section of the camera - or should i say on cameras DSP path somewhere, so it is the same thing basically as your filter, but as sh0dan suggested it is applied to linear data and not on final curve (ok, i dont have a clue about coding so i dont know how would that translate to either RGB or YUV data, but i imagine you would have to reverse the UV part back to 'original stage' (or to approximately original stage) and then process that.)

a note: and CCD is linear device, i guess you could imagine that as sort of a foton counter maybe? - but the point is it doesnt see the world the same as eye (which is nonlinear).

p.s.
here is an example how some try to solve things in older photoshop versions:
http://www.adobeevangelists.com/pdfs/photoshop/tipsandtricks/CorrectByNumbers.pdf
(instead of getting the original curve - why not just adjust high, mid and low areas as separate..., hmm well :))

[rfmmars]

I did but it crashed VDub 1.6.14 when previewing. Also the GUI is total gibberish as it's in Russian and the Cyrillic symbols don't even show up correctly on my PC :(

This is very good fillter but it does crash, but not always.

When it isn't going to work the bottom four sliders will snap back to zero. Reload and everything is ok.

What I do is to frameserve with this filter and only this filter in the chain to another copy of VD(xxx). Hope this plugin will be updated soon.

Richard

[Boulder]

I could try using it in Avisynth as soon as I can figure out some settings to try, maybe it'll be a bit more stable then. The GUI not being in English makes it hard though :) Maybe some Russian speaking person here could send the translations to the author?

[Boulder]

Or could someone (Fizick?) please post the parameters for using in Avisynth? Based on setting things randomly, the results look quite good but I'd like to know what I really am doing to the video;)

[Fizick]

I do not use it.
More info at:
http://forum.ixbt.com/topic.cgi?id=29:22283
(in Russian). I hope now you can imagine if all MY filter description would be in Russian only.

[Boulder]

I hope now you can imagine if all MY filter description would be in Russian only.
Well, I've never had any problems with your documentation. If I ever did, I would simply ask for clarification :)

The most important thing is that there at least is documentation, be it good or not so good, also the grammar isn't that important as you can easily assume and guess certain things if you're familiar with video processing.

I was thinking on the lines of someone pointing out what it says on the GUI. All I have is lots of question marks there:

http://img67.imageshack.us/img67/682/wb6vv.th.jpg (http://img67.imageshack.us/my.php?image=wb6vv.jpg)

[Inc]

Hmmmm .....

Lets see, I got one "example" frame of a movie which represents the full luma range given of the whole movie.

In the source the luma range black-,medium- and white-balance is incorrect.

http://img115.imageshack.us/img115/9350/source24bq.jpg

I added a manually generated Border to the right side where I used Photoshop to generate that incorrect-colored black/grey/white state where I used the pipette to catch the brightest and the darkest area out of the given representative frame.

This vertical bar can be imported as a pic into your avisynth script, converting to your matching colorspace, ... generate a clip using the same framecount.

Now there should be a way where ONLY the whitebalance is calculated out of THAT clip and finally applied to your movieclip.

The result will be this:

http://img66.imageshack.us/img66/4038/balanced5fy.jpg

(I did stack horizontal the right border only for explaining).

This is just the approach by using photoshop. But the way could be included as routines in an avs-filter.

[Fizick]

I ask Sergey, he released English version AcobW today,
comments welcomed.

[Boulder]

Thank you, I'll check it later today and post my comments as well.

I'll probably create a new thread and possibly upload some sample clips of my troublesome scenes as well. This project of mine has been in limbo for several weeks now because I haven't been able to get a good white balance in some scenes. I'd like to get things done properly so that I won't have to get back to it soon ;)

[Boulder]

A quick comment on the ACOBW translations: they look just fine:) However, I think offset should be brightness?