Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th June 2013, 20:01   #1  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
Automatic White Balance (AWB) script function(s)

EDIT: attached script version 27th October 2013 This one may be called more than once from a script, e.g. to compare settings
EDIT: script version 13th November a bit faster and hopefully no more AviSynth string heap issue - but has severe bugs in AWB algorithm, please don't use at this time. Update to come soon.
EDIT: script version 1st December Finalized White Patch (Max RGB) algorithm. Also:
+ uses arrays instead of global vars: full non linear access with 100% reproduceable output
+ multi instance capable
+ unprecedented plugin check

I'll concentrate on a new approach now (after I learned so much about things that are not what they seem in consumer cameras YUV or RGB values)


Original Post

I'd like to announce a script function or maybe script function set for automatic white balance for real world footage, similar to the known function in digital cameras.

Some widely used algorithms implement theories of J. von Kries on the ability of the human visual system to recognize the color of objects under biased illumination, called "color constancy".
They are known as "Gray World", "Max-RGB" and "Shades of Gray".

Maybe I failed in finding an already existing plugin for AviSynth or VirtualDub that implements a useful automatic white balance (I do not count ColorYUV(autowhite=true) as useful). They may be not too fast since they use the runtime environment. But they show once again the power of AviSynth scripting when one refuses to write in C (as I do up to now). Here are the first appetizers, and comments are welcome.

Needs AviSynth V2.6 and dither, GRunt, mt_masktools and RT_Stats plugins.

Code:
--> see attached GrayWorldSimple.txt, Max-RGB_Simple.txt, ShadesOfGraySimple.txt
Attached Files
File Type: txt GrayWorldSimple.txt (788 Bytes, 406 views)
File Type: txt Max-RGB_Simple.txt (1.2 KB, 329 views)
File Type: txt ShadesOfGraySimple.txt (1.2 KB, 247 views)
File Type: txt _AWB_131113.txt (72.0 KB, 315 views)
File Type: txt _AWB_131201.txt (54.2 KB, 467 views)

Last edited by martin53; 1st December 2013 at 19:04. Reason: Attachment of actual script
martin53 is offline   Reply With Quote
Old 13th June 2013, 23:51   #2  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Have not tried as yet, but shall.
I have real expectations of this, hope you live up to them.

EDIT: Was the MYStats thing of any use ?
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???

Last edited by StainlessS; 14th June 2013 at 09:34.
StainlessS is offline   Reply With Quote
Old 14th June 2013, 21:32   #3  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
Quote:
Originally Posted by StainlessS View Post
...have real expectations of this, hope you live up to them.
Same for me! To be honest, the underlying concepts of the three algorithms are quite ungarnished. But there was or is plenty of research on the topic, and this is what publicly evolved from it. To repeat myself: the concepts are not my inventions.

Quote:
Originally Posted by StainlessS View Post
Was the MYStats thing of any use ?
I am convinced it will! I first tried it on itself on some clips and found that it seemed to work well. The list of challenges on my mind contains averaging over a group of frames, which in turn demands scene detection, an automatic mode which evaluates several algorithms and chooses/balances between them, and, maybe most important, masking of a color range that is valid at all for potential color temperature shift, i.e. lies along the temperature curve and is not a colorful object. I expect much from this to reduce misled corrections.

Up to now, I had to cope with color bleeding, banding etc. from the bad resolution of 8 bits in the neccessary YUV-RGB-YUV processing. Research on how to define the interesting color range will come next.

You can see from the examples that there is a lack of full featured statistics functions for RGB as there is for YUV for the task. I found workarounds though, so that's not a job stopper.
martin53 is offline   Reply With Quote
Old 14th June 2013, 22:01   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Quote:
Originally Posted by martin53 View Post
lack of full featured statistics functions for RGB as there is for YUV
Give me time and I'll do similar funcs for RGB
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 15th June 2013, 10:49   #5  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
If you like to do it for fun, go ahead. For the white balance task, it will evolve which statistic functions will be needed.
Among them are, as far as I can judge today, masked versions for AverageChromaU and AverageChromaV. The mask should be taken from the Y plane of the mask clip, but the U and V planes of the main clip be evaluated.

Also, for the "Shades of Gray" algorithm, average functions that do not average the first power of samples, but an arbitrary power are useful.
To date the clip is prepared with 16 bit bitdepth and the samples taken to the power of 6 with Dither_lut8() and then dithered down to 8 bit resolution before measuring. But of course that's a workaround.
martin53 is offline   Reply With Quote
Old 15th June 2013, 11:18   #6  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
I am just wondering about color conversion documentation.
The AviSynth v2.6 doc in Advanced Topics / ColorSpace Conversions looks very reasonable. The constants Kr, Kg and Kb are taken from the standard, and the following equations derived from that.
But at second sight, the (v - 128 = ...) and (u - 128 = ...) equations have factors of 0.5, which seems arbitrary.

The web page http://en.wikipedia.org/wiki/YUV defines Wr=Kr, Wg=Kg and Wb=Kg, but furthermore Umax=0.436 and Vmax=0.615. This of course leads to different RGB->YUV and YUV->RGB transformations.

My question is: who is right, and are the AviSynth V2.6 colorspace conversion routines trustable?
martin53 is offline   Reply With Quote
Old 15th June 2013, 14:18   #7  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,377
Quote:
Originally Posted by martin53 View Post
The AviSynth v2.6 doc in Advanced Topics / ColorSpace Conversions looks very reasonable. The constants Kr, Kg and Kb are taken from the standard, and the following equations derived from that.
But at second sight, the (v - 128 = ...) and (u - 128 = ...) equations have factors of 0.5, which seems arbitrary.

The web page http://en.wikipedia.org/wiki/YUV defines Wr=Kr, Wg=Kg and Wb=Kg, but furthermore Umax=0.436 and Vmax=0.615. This of course leads to different RGB->YUV and YUV->RGB transformations.
The difference is simply an arbitrary scale factor in U and V.
The Avisynth wiki page gives the formula to derive U and V in the range [-1, 1], while the Wikipedia article uses the range [-Umax, Umax] and [-Vmax, Vmax].

But when doing RGB<->YUV conversions on 8-bit inputs, you have to rescale anyway to fit the range [16, 240] or [0, 255] (depending on the matrix used). The Avisynth conversion routines do this correctly.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 15th June 2013, 19:47   #8  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
Quote:
Originally Posted by Gavino View Post
The Avisynth wiki page gives the formula to derive U and V in the range [-1, 1]
Thank you for answering, Gavino.
Alas, I intentionally asked for the lowercase letters u and v because I need the constants outside the builtin conversion routines.
Reading on in the Wikipedia article, I found now that I confused Y'UV with Y'CbCr anyway, and instead of calculating
U = -0.147 * R - 0.289 * G + 0.436 * B
which would only be valid for an analog device, for a digital device it must be
Cb = -0.169 * R - 0.331 * G + 0.499 * B + 128

I take 0.499 as 0.5, and so indeed the two formulas match. -0.169 comes from 0.5*Kr/(1-Kb) while the analog -0.147 comes from 0.436*Kr/(1-Kb) and so on, and it is confirmed that in AviSynth, the u and v scale factors are 0.5
(which btw. makes sense, because Y's [0,1] range is 1 wide while U,V's [-1,1] range has a width of 2. The [0,255] scaled variables share a range ot the same width, so a downscaling with 0.5 for u and v is obvious from that)
martin53 is offline   Reply With Quote
Old 17th June 2013, 21:02   #9  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
As an ancillary product, here is a script to change the 'color temperature' of a clip by a fixed offset.
It uses physically correct values to correct RGB and tries not to change the overall luminance. If that clamps RGB, you may enter a gain factor < 1.0 to make the clip darker with less clamping.

Code:
-> see 1st post
EDIT: function is part of complete script in 1st post

Last edited by martin53; 13th October 2013 at 20:18. Reason: reference to updated script in 1st post
martin53 is offline   Reply With Quote
Old 23rd June 2013, 10:23   #10  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
More advanced versions of the simple estimators drafted above will restrict estimation to colors in the picture that might be biased grey.
Below is a mask generator that I have used lately and that will find its way into later versions of the AWB function itself.

It can be fed with this test clip, to show the functionality.

Luma statistics functions give much better results if their input is restricted to these pixels, i.e. the MYStats functions with this mask give much better results than the standard functions.

I'll be offline for several days from now.

Code:
function testClipYUV() {
    c = BlankClip(240,640,480,"YV12")
    c.mt_lutspa(yexpr="128", uexpr="x 255 *", vexpr="y 255 *", chroma="process")
}
Mask generator
Code:
function GrayGamutMask(clip c) {
#
# Function to create a mask that is only white for clip pixels within a defined color and luminance range.
# Expects clip in YUV format
# Returns mask clip in YV24 format
#
# Author		doom9.org/martin53
# Version		1.0
# Date			June 18, 2013
# Uses plugins	GRunT, MaskTools v2.0a48
# Tested with	Avisynth V2.6
#
# Changes		n.a.
#
    TOL_UP = 0.9
    TOL_DN = -0.4
    U_DN = 128-15	# -50 mired, i.e. color temperature change from D65 to 4900K warm day or 7800K to 5600K flash
    V_UP = 128+7	# -50 mired, i.e. color temperature change from D65 to 4900K warm day or 7800K to 5600K flash
    U_UP = 128+15	#  50 mired, i.e. color temperature change from 4900K light to D65
    V_DN = 128-7	#  50 mired, i.e. color temperature change from 4900K light to D65
    YMIN = 100
    YMAX = 235 - 1
    c24 = c.ConvertToYV24()
	#Make a mask according to experimentally found U and V change over color temperature
	#mt_lutxy is fed with the U and V components. x stands for the U component, y for the V component.
    expr = mt_polish("(x-128)/15+(y-128)/7 <= "+string(TOL_UP)
		 \+" & (x-128)/15+(y-128)/7 >= "+string(TOL_DN)
		 \+" & x <= "+string(U_UP)
		 \+" & y >= "+string(V_DN)
		 \+" & x >= "+string(U_DN)
		 \+" & y <= "+string(V_UP)
		 \+" ? 255 : 0")
    exprX = mt_polish("(x-128)/15+(y-128)/7 <= "+string(TOL_UP*2)
		 \+" & (x-128)/15+(y-128)/7 >= "+string(TOL_DN*2)
		 \+" & x <= "+string(U_UP*2)
		 \+" & y >= "+string(V_DN*2)
		 \+" & x >= "+string(U_DN*2)
		 \+" & y <= "+string(V_UP*2)
		 \+" ? 255 : 0")
	#subtitle(expr)
    mChroma1 = mt_lutxy(c24.UToY(), c24.VToY(), expr=expr, chroma="128")
	#Choose this mask if it is not all black, otherwise choose fallback mask with doubled tolerance values
	mChroma = GConditionalFilter(mChroma1, mChroma1, mt_lutxy(c24.UToY(), c24.VToY(), expr=exprX, chroma="128"), "AverageLuma()>0")
	#Complete the mask with minimum and maximum luma restrictions.
	#mt_lutxy is fed with the Y component and the previous mask. x stands for Y (luma), y for the previous mask.
    expr = mt_polish("y > 0"+" & x >= "+string(YMIN)+" & x <= "+string(YMAX)+" ? 255 : 0")
	#subtitle(expr)
    mLuma = mt_lutxy(c24, mChroma, expr=expr, chroma="128")
	#Choose mask mLuma if it is not all black, else choose fallback mask mChroma
	GConditionalFilter(mLuma, mLuma, mChroma, "AverageLuma()>0")
	#Choose this mask if it is not all black, else choose fallback all white mask
	GConditionalFilter(last, last, BlankClip(c24, color_yuv=$f08080), "AverageLuma()>0")
	#debug
	#GScriptClip("""RT_Debug("Mask",string(AverageLuma())) last""")	#"
}
Usage e.g.
Code:
TestClipYUV()
merge(last, GrayGamutMask().ConvertToYV12(), weight=0.1)
martin53 is offline   Reply With Quote
Old 6th July 2013, 14:05   #11  |  Link
Navarre66
Registered User
 
Join Date: Mar 2013
Posts: 6
error in MaxRGB1

I'm trying to use MaxRGB1, but I keep getting the error:
Quote:
Script error: ScriptClip does not have a named argument "args"
Apparently inferring it doesn't like the args="RGBinYUV"

I'm using AviSynth 2.6.0 Alpha4 because I get problems with dither.avsi if I use AviSynth 2.5.8 or SEt's version of Avisynth 2.6.

Navarre66 is offline   Reply With Quote
Old 6th July 2013, 14:41   #12  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,377
Quote:
Originally Posted by Navarre66 View Post
I keep getting the error:
Script error: ScriptClip does not have a named argument "args"
Apparently inferring it doesn't like the args="RGBinYUV"
You need to install the GRunT plugin (which - among other things - extends ScriptClip with the 'args' parameter).
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 6th July 2013, 18:45   #13  |  Link
Navarre66
Registered User
 
Join Date: Mar 2013
Posts: 6
Quote:
Originally Posted by Gavino View Post
You need to install the GRunT plugin (which - among other things - extends ScriptClip with the 'args' parameter).
Thanks. That did the trick.
Navarre66 is offline   Reply With Quote
Old 25th July 2013, 22:14   #14  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
Script is now attached to 1st post. Please read abstract before you try it.
Comments welcome.

Last edited by martin53; 26th July 2013 at 13:24.
martin53 is offline   Reply With Quote
Old 18th August 2013, 18:23   #15  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
This is a call for comments on expectations what an AWB script should serve for.

After I fiddled so much with my lots of images & scenes, I learned that it is not that difficult to average RGB in an image, but that slight adjustments of scenes require a script to restrict itself, while the correction of images taken with sunlight setting in bulb illuminated environment, or very shadowy environment, require strong impact.

So I may optimize it to do subtle enhancements, but sacrifice the ability to always return the same result, no matter how extreme the user may bias the input in advance. For that purpose, I shifted the 'correlated color temperature' of the input clip by +-1000C, and the output does not stay the same, how I earlier defined the goal for myself. Personally I feel now that it is important to retain the original mood of scenes in most cases, so the color of the result just seems 'right' in an unobstrusive way, i.e. just eliminate annoying color bias e.g. from incandescent light, but not overdo compensation. That might require manual pre-balancing in some cases, however. A script for manual pre-shifting of the correlated color temperature is provided in this thread.
martin53 is offline   Reply With Quote
Old 19th August 2013, 18:25   #16  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 852
Hi martin53!
I find Your script useful, thanks for Your time. My source mainly VHS capture indoor shooting with color distortion (a lot of yellow see https://www.dropbox.com/s/wyc1o6ygvxbuv8a/samplevhs.avi).
Code:
AVISource("tape1.avi")
AssumeTFF()
ConvertToYV12(interlaced=true)
Crop(16,12,-16,-12)
s=SeparateFields()
s.AutoGain(adjust_mode=1).ShadesOfGray1()
# OR s.AutoGain(adjust_mode=1).GrayWorld1()
StackVertical(s,last)
#AssumeFieldBased().Weave()
work fine shift yellow to blue color and video look good (I know real color because have photo and slide from this place), but script
Code:
AVISource("tape1.avi")
AssumeTFF()
ConvertToYV12(interlaced=true)
Crop(16,12,-16,-12)
s=SeparateFields()
s.AutoGain(adjust_mode=1).AWB()
StackVertical(s,last)
#AssumeFieldBased().Weave()
give more yellow than at source. At other source script give unstable output atr preview in VirtualDub.
Please advice parameter set for AWB for my source..
yup.
yup is offline   Reply With Quote
Old 19th August 2013, 19:58   #17  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
yup,
downloaded the example.
What codec should I use to decode lagarith LAGS video (W7 x86)? my (I admit, probably outdated) AviSource/ffdshow are helpless. VLC plays, but with *heavy* blue flicker.

(Forgive my ignorance, I of course know how to run google, but at the moment don't like to spend time learning about lagarith)

EDIT:[ don't bother, XMediaRecode->HuffYUV worked.

I checked with the current script on my HD. One problem is the date stamp. The "10/" shows good white. I will add xm and ym to the parameters list to allow masking of a rectangle, please have a bit patience.
In between, you might mask that rectangle with a Overlay() line.

But aside from automatic balancing, I found
Code:
ShiftCCT(5600)
quite appealing in this case.

If you allow, I'll use the short sequence as additional test case. It is interesting because if flickers more than my other test clips.
]/EDIT

Last edited by martin53; 19th August 2013 at 21:10.
martin53 is offline   Reply With Quote
Old 19th August 2013, 20:42   #18  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
M53, pretty sure I've found a bug in RT_Stats RT_RGBChanxxx filters, suspend usage for a few hours, I'll do a fix.
Found it when doing MRGBChanStats() plug.

Lagarith pretty common codec for AVI YV12
http://www.videohelp.com/tools/Lagar...ss-Video-Codec
Just install it, not much to learn.


But being overtaken by UT_Video (RGB,YUY2,YV12)
http://www.videohelp.com/tools/Ut-Video-Codec-Suite
I hardly ever use Lagarith or HuffYUV now.
Just install it, not much to learn.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 20th August 2013, 12:39   #19  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 852
martin53!

I try code
Code:
AVISource("tape1.avi")
AssumeTFF()
ConvertToYV12(interlaced=true)
Crop(16,12,-16,-12)
s=SeparateFields()
s.AutoGain(adjust_mode=1).ShiftCCT(5600)
StackVertical(s,last)
And see good result. Please confirm need I use Autogain before Your AWB functions?
yup.
yup is offline   Reply With Quote
Old 20th August 2013, 18:04   #20  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
Quote:
Originally Posted by yup View Post
Please confirm need I use Autogain before Your AWB functions?
I was not aware of any influence between the two. When I first checked some minutes ago, the clip with AutoGain was automatically balanced in a much better way than before, but then when I compared histograms I did not find an effect of Autogain on any single pixel of a frame - and I looked very closely, with a difference amplifier. And I can not reproduce the mentioned effect now.

At the moment, I can not identify any difference. Please describe the effect you observe with/-out it.

Last edited by martin53; 20th August 2013 at 18:45.
martin53 is offline   Reply With Quote
Reply

Tags
awb, color, colorbalance

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:11.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.