Show max / min luma - help needed from script wizard

[neily]

Hi,

I've been trying to write a script which will overlay a subtitle of a running minimum minimum and maximum maximum luma value for a clip, so that I can see whether the luminance range of the clip needs adjusting before passing to an encoder. I have tried looking at the help file and example scripts, but it is just plain beyond my comprehension of AVISynth! Can anyone help?

[Manao]

In the documentation, look for the 'ScriptClip' filter : you'll see that the following gives you the max luma value :

ScriptClip("subtitle(string(YPlaneMax))")

[Didée]

Same does ColorYUV(analyze=true).

Also possible: (nice visualization, but out-of-range values sometimes harder to see)

histogram(mode="levels")

Visualization *which* pixels are out-of-range: (needs MaskTools - there was an own filter for showing these, but I can't recall it ATM)

yv12lut("x 16 < x 235 > | 255 x - x ?",U=2,V=2)

(will invert out-of-range pixels, luma only)

[neily]

Thanks for the replies, but what I was hoping for was a script that when run, the min luma and max luma would be sticky, i.e. when the clip has been run through, the min and max luma values would be the values for the clip rather than the last frame.

[Cyberia]

So basically, you want an 'autoscan' feature just like VirtualDub's Levels?

[Karyudo]

This is not my thread (obviously), but I'll second the OP's interest in such a "sticky levels" feature, and in fact go one further:

I'd like to be able to see on which frame (or frames) the maximum (and/or minimum) levels occurred. Like maybe output a text file with all the frames ranked? Or just the frames that share the highest and lowest values?

So, for example, you could make a capture with deliberately low contrast and brightness settings, find where the max and min were, and then be sure to look at the histograms of those specific frames when setting the parameters for a final capture.

This would mean you could definitely be sure you're avoiding blowing out highlights or crushing blacks when making your capture.

(I hope I haven't hijacked the original idea of the thread too much...? If I have, I apologize.)

[Manao]

WriteFile / ConditionnalReader are your friends ( have a look at the documentation )

[tsp]

you can use this script it needs the function pop (http://forum.doom9.org/showthread.php?s=&threadid=83451&highlight=pop) to work

global s=AVISource("c:\test.avi").converttoyv12()
FrameEvaluate(s,"""\
global Minimum=YPlaneMin(s)<pop(255,"Minimum")?YPlaneMin(s): pop(255,"Minimum")""")
FrameEvaluate(last,"""\
global MinFrame=YPlaneMin(s)<pop(255,"Minimum")?current_frame: pop(0,"MinFrame")""")

scriptclip(last,"""\
subtitle("Global min:"+string(Minimum)+" Current min:"+string(YPlaneMin(s))+\
" MinFrame at frame: "+string(MinFrame)) """,after_frame=true)

[Didée]

SansGrip already has coded such a tool, back in AviSynth 2.0 time: RangeInfo.

Should still work through LoadPluginEx, but obviously needs YUY2 sources. Wouldn't be a bad idea to port it to AS 2.5 ...

- RangeInfo plugin (http://www.indeus.com/sansgrip/avisynth/RangeInfo-0.1.zip)
- Source (http://www.indeus.com/sansgrip/avisynth/Ghostbuster-0.1_src.zip)
- ReadMe (http://www.indeus.com/sansgrip/avisynth/Noise_Generators-readme.html)

[Karyudo]

One little error: The readme is at http://www.kvcd.net/sansgrip/avisynth/RangeInfo-readme.html, not at the link provided (which is for Noise Generators).

(It's SansGrip's fault for making too many filters! See them all at http://www.kvcd.net/sansgrip/avisynth/.)

[Karyudo]

...and of course that filter sounds like it does exactly what not only the OP wants, but what I'd like to see, as well!

I do hope someone with some coding ability (i.e. sadly not me) can pick up and run with your idea of porting to AviSynth 2.5. Whoever does so will have at least two big fans in me and neily.

[tsp]

YV12 version of the show part of RangeInfo, again it needs pop to function:
you can use yv12Lut for the missing highlighting of specific pixel



function rangeinfoYV12(clip s){
global s=s
FrameEvaluate(s,"""\
global MinimumY=YPlaneMin(s)<pop(255,"MinimumY")?YPlaneMin(s): pop(255,"MinimumY")""")
FrameEvaluate(last,"""\
global MinFrameY=YPlaneMin(s)<=pop(255,"MinimumY")?current_frame: pop(0,"MinFrameY")""")

FrameEvaluate(last,"""\
global MaximumY=YPlaneMax(s)>pop(0,"MaximumY")?YPlaneMax(s): pop(0,"MaximumY")""")
FrameEvaluate(last,"""\
global MaxFrameY=pop(0,"MaximumY")<=YPlaneMax(s)?current_frame: pop(0,"MaxFrameY")""")

FrameEvaluate(last,"""\
global MinimumU=UPlaneMin(s)<pop(255,"MinimumU")?UPlaneMin(s): pop(255,"MinimumU")""")
FrameEvaluate(last,"""\
global MinFrameU=UPlaneMin(s)<=pop(255,"MinimumU")?current_frame: pop(0,"MinFrameU")""")

FrameEvaluate(last,"""\
global MaximumU=UPlaneMax(s)>pop(0,"MaximumU")?UPlaneMax(s): pop(0,"MaximumU")""")
FrameEvaluate(last,"""\
global MaxFrameU=pop(0,"MaximumU")<=UPlaneMax(s)?current_frame: pop(0,"MaxFrameU")""")

FrameEvaluate(last,"""\
global MinimumV=VPlaneMin(s)<pop(255,"MinimumV")?VPlaneMin(s): pop(255,"MinimumV")""")
FrameEvaluate(last,"""\
global MinFrameV=VPlaneMin(s)<=pop(255,"MinimumV")?current_frame: pop(0,"MinFrameV")""")

FrameEvaluate(last,"""\
global MaximumV=VPlaneMax(s)>pop(0,"MaximumV")?VPlaneMax(s): pop(0,"MaximumV")""")
FrameEvaluate(last,"""\
global MaxFrameV=VPlaneMax(s)>=pop(0,"MaximumV")?current_frame: pop(0,"MaxFrameV")""")


return scriptclip(last,"""\
subtitle("Global Ymin:"+string(MinimumY)+" Current minY:"+string(YPlaneMin(s))+\
" MinFrameY at: "+string(MinFrameY),y=20).\
subtitle("Global Ymax:"+string(MaximumY)+" Current maxY:"+string(YPlaneMax(s))+\
" MaxFrameY at: "+string(MaxFrameY),y=40).\
subtitle("Global Umin:"+string(MinimumU)+" Current minU:"+string(UPlaneMin(s))+\
" MinFrameU at: "+string(MinFrameU),y=60).\
subtitle("Global Umax:"+string(MaximumU)+" Current maxU:"+string(UPlaneMax(s))+\
" MaxFrameU at: "+string(MaxFrameU),y=80).\
subtitle("Global Vmin:"+string(MinimumV)+" Current minV:"+string(VPlaneMin(s))+\
" MinFrameV at: "+string(MinFrameV),y=100).\
subtitle("Global Vmax:"+string(MaximumV)+" Current maxV:"+string(VPlaneMax(s))+\
" MaxFrameV at: "+string(MaxFrameV),y=120).\
subtitle("MeanY: "+string(AverageLuma(s))+" MeanU: "+string(AverageChromaU(s))+"MeanV: "+string(AverageChromaV(s)),y=160).\
subtitle("Current frame: "+string(current_frame),y=180)\
""",after_frame=true)
}

[neily]

Thanks for all the replies. As I mentioned, this kind of scripting is beyond my normal experience. Using various examples, I have come up with the following script which seems to work, but I don't necessarily understand everything that is going on in it. So please tell me what may be "wrong" with it. BTW, the threshold for YPlaneMax, which is supposed be in percent, doesn't seem to work as expected. The source for the value used in ColorYUV analyze implies 1/256 of pixels, but to get the same result in YPlaneMax required an input of 39.


AVISource("R:\Test.avi")
ConvertToYV12()
global YMax = 0
global YMaxFrame = 0
global YLooseMax = 0
global YLooseMaxFrame = 0
FrameEvaluate("YMax = YMax > YPlaneMax ? YMax :YPlaneMax" )
FrameEvaluate("YMaxFrame = YMax > YPlaneMax ?YMaxFrame : current_frame" )
FrameEvaluate("YLooseMax = YLooseMax > YPlaneMax(39) ? YLooseMax :YPlaneMax(39)" )
FrameEvaluate("YLooseMaxFrame = YLooseMax > YPlaneMax(39) ?YLooseMaxFrame : current_frame" )
ScriptClip( """ Subtitle("YMax: " + String(YMax),8,20) """)
ScriptClip( """ Subtitle("YMax Frame: " + String(YMaxFrame),8,40) """)
ScriptClip( """ Subtitle("YLooseMax: " + String(YLooseMax),8,60) """)
ScriptClip( """ Subtitle("YLooseMaxFrame: "+ String(YLooseMaxFrame),8,80) """)

[tsp]

neily: The only thing that I can see is wrong with the script is that you need to add after_frame=true to the scriptclip if want the values from the current frame and not the last frame. Neat trick with the global var=0. Removes the need for pop. The modifed script for rangeinfoYV12 looks like this:


function rangeinfoYV12(clip s){
global s=s
global MinimumY=255
global MinFrameY=0
global MaximumY=0
global MaxFrameY=0
global MinimumU=255
global MinFrameU=0
global MaximumU=0
global MaxFrameU=0
global MinimumV=255
global MinFrameV=0
global MaximumV=0
global MaxFrameV=0

FrameEvaluate(s,"""\
MinimumY=YPlaneMin(s)<MinimumY?YPlaneMin(s):MinimumY""")
FrameEvaluate(last,"""\
MinFrameY=YPlaneMin(s)<=MinimumY?current_frame:MinFrameY""")

FrameEvaluate(last,"""\
MaximumY=YPlaneMax(s)>MaximumY?YPlaneMax(s):MaximumY""")
FrameEvaluate(last,"""\
MaxFrameY=MaximumY<=YPlaneMax(s)?current_frame:MaxFrameY""")

FrameEvaluate(last,"""\
MinimumU=UPlaneMin(s)<MinimumU?UPlaneMin(s):MinimumU""")
FrameEvaluate(last,"""\
MinFrameU=UPlaneMin(s)<=MinimumU?current_frame:MinFrameU""")

FrameEvaluate(last,"""\
MaximumU=UPlaneMax(s)>MaximumU?UPlaneMax(s):MaximumU""")
FrameEvaluate(last,"""\
MaxFrameU=MaximumU<=UPlaneMax(s)?current_frame:MaxFrameU""")

FrameEvaluate(last,"""\
MinimumV=VPlaneMin(s)<MinimumV?VPlaneMin(s):MinimumV""")
FrameEvaluate(last,"""\
MinFrameV=VPlaneMin(s)<=MinimumV?current_frame:MinFrameV""")

FrameEvaluate(last,"""\
MaximumV=VPlaneMax(s)>MaximumV?VPlaneMax(s):MaximumV""")
FrameEvaluate(last,"""\
MaxFrameV=VPlaneMax(s)>=MaximumV?current_frame:MaxFrameV""")


return scriptclip(last,"""\
subtitle("Global Ymin:"+string(MinimumY)+" Current minY:"+string(YPlaneMin(s))+\
" MinFrameY at: "+string(MinFrameY),y=20).\
subtitle("Global Ymax:"+string(MaximumY)+" Current maxY:"+string(YPlaneMax(s))+\
" MaxFrameY at: "+string(MaxFrameY),y=40).\
subtitle("Global Umin:"+string(MinimumU)+" Current minU:"+string(UPlaneMin(s))+\
" MinFrameU at: "+string(MinFrameU),y=60).\
subtitle("Global Umax:"+string(MaximumU)+" Current maxU:"+string(UPlaneMax(s))+\
" MaxFrameU at: "+string(MaxFrameU),y=80).\
subtitle("Global Vmin:"+string(MinimumV)+" Current minV:"+string(VPlaneMin(s))+\
" MinFrameV at: "+string(MinFrameV),y=100).\
subtitle("Global Vmax:"+string(MaximumV)+" Current maxV:"+string(VPlaneMax(s))+\
" MaxFrameV at: "+string(MaxFrameV),y=120).\
subtitle("MeanY: "+string(AverageLuma(s))+" MeanU: "+string(AverageChromaU(s))+"MeanV: "+string(AverageChromaV(s)),y=160).\
subtitle("Current frame: "+string(current_frame),y=180)\
""",after_frame=true)
}

[neily]

Having sort of got the result I thought I wanted, I'm not sure how useful it is. Like others have expressed, all I am trying to do is best work out how to optimise luminance levels during conversions. My source is DV, and the target is DVD, but there are so many variables. The Matrox DV codec, which outputs to RGB, has the option to map YUV to either RGB 0-255, 16-255, or 16-235. AVISynth can output either RGB or YUY2. CCE can then map to 0-255 or 16-235. Previewing on a computer monitor is problematic, as one can't know how the raw MPEG2 files are handled by the DirectShow filter. Not a new problem, but it doesn't get any easier.

Anyway, having reviewed the comments and filters available, I have come up with this function, which takes a clip, and stacks vertically beneath a magnified luma portion of Histogram("levels"), with the option of displaying ColorYUV(analyze=true). It seems to help.


function ShowLumaStacked(clip c,int mag,bool showanalyze)
{
# mag - magnification of histogram height, showanalyze - add ColorYUV(analyze=true)
histoclip=ConvertToYV12(c).Histogram("levels").crop(c.width,0,256,68).\
Tweak(hue=0.0,sat=5.00,bright=48.0,cont=1.00).PointResize(c.width,68 * mag).ConvertToYUY2()
return showanalyze? StackVertical(ConvertToYUY2(c).ColorYUV(analyze=true),histoclip): \
StackVertical(ConvertToYUY2(c),histoclip)
}


Edited 2005-01-30: Works better with PointResize and a Tweak

[oo_void]

Good work on the select range function... I've written something similar myself, and in doing so, I've noticed that a simple plane eval might not give you an accurate levels measure. For example, one stray pixel due to wierdness within a clip, can skew the whole measurement.

Thankfully, the people who developed the the plane evaluation functions took this into account by adding a threshhold param. From the docs, "Threshold is a percentage, on how many percent of the pixels are allowed above or below minimum."

I've modified the funtion to accept an additional threshold param (defaults to 2%), along with fixing what I think was a very simple bug in the first iteration. Enjoy ...


function rangeinfoYV12(clip s, int "v") {
Default(v, 2)

global s = s
global v = v

global MinimumY = 255
global MinFrameY = 0
global MaximumY = 0
global MaxFrameY = 0
global MinimumU = 255
global MinFrameU = 0
global MaximumU = 0
global MaxFrameU = 0
global MinimumV = 255
global MinFrameV = 0
global MaximumV = 0
global MaxFrameV = 0


FrameEvaluate(s, """MinimumY = YPlaneMin(s, v) < MinimumY ? YPlaneMin(s, v) : MinimumY""")
FrameEvaluate(last, """MinFrameY = YPlaneMin(s, v) <= MinimumY ? current_frame : MinFrameY""")

FrameEvaluate(last,"""MaximumY = YPlaneMax(s, v) > MaximumY ? YPlaneMax(s, v) : MaximumY""")
FrameEvaluate(last,"""MaxFrameY = MaximumY <= YPlaneMax(s, v) ? current_frame : MaxFrameY""")

FrameEvaluate(last,"""MinimumU = UPlaneMin(s, v) < MinimumU ? UPlaneMin(s, v) : MinimumU""")
FrameEvaluate(last,"""MinFrameU = UPlaneMin(s, v) <= MinimumU ? current_frame : MinFrameU""")

FrameEvaluate(last,"""MaximumU = UPlaneMax(s, v) > MaximumU ? UPlaneMax(s, v) : MaximumU""")
FrameEvaluate(last,"""MaxFrameU = MaximumU <= UPlaneMax(s, v) ? current_frame:MaxFrameU""")

FrameEvaluate(last,"""MinimumV = VPlaneMin(s, v) < MinimumV ? VPlaneMin(s, v) : MinimumV""")
FrameEvaluate(last,"""MinFrameV = VPlaneMin(s, v) <= MinimumV ? current_frame:MinFrameV""")

FrameEvaluate(last,"""MaximumV = VPlaneMax(s, v) > MaximumV ? VPlaneMax(s, v) : MaximumV""")
# Buggy! FrameEvaluate(last,"""MaxFrameV = VPlaneMax(s) >= MaximumV ? current_frame : MaxFrameV""")
FrameEvaluate(last,"""MaxFrameV = VPlaneMax(s, v) <= MaximumV ? current_frame : MaxFrameV""")


return scriptclip(last,"""\
subtitle("Global Ymin:" + string(MinimumY) + " Current minY:" + string(YPlaneMin(s)) + \
" MinFrameY at: " + string(MinFrameY), y = 20).\
subtitle("Global Ymax:" + string(MaximumY) + " Current maxY:" + string(YPlaneMax(s)) + \
" MaxFrameY at: " + string(MaxFrameY), y = 40).\
subtitle("Global Umin:" + string(MinimumU) + " Current minU:" + string(UPlaneMin(s)) + \
" MinFrameU at: " + string(MinFrameU), y = 60).\
subtitle("Global Umax:" + string(MaximumU) + " Current maxU:" + string(UPlaneMax(s)) + \
" MaxFrameU at: " + string(MaxFrameU), y = 80).\
subtitle("Global Vmin:" + string(MinimumV) + " Current minV:" + string(VPlaneMin(s)) + \
" MinFrameV at: " + string(MinFrameV), y = 100).\
subtitle("Global Vmax:" + string(MaximumV) + " Current maxV:" + string(VPlaneMax(s)) + \
" MaxFrameV at: " + string(MaxFrameV), y = 120).\
subtitle("MeanY: " + string(AverageLuma(s)) + " MeanU: " + string(AverageChromaU(s)) + \
" MeanV: " + string(AverageChromaV(s)), y = 160).\
subtitle("Current frame: " + string(current_frame), y = 180) \
""", after_frame = true)
}

------
oo_void

[tsp]

oo_void as neily álso discovered there is a bug in the code. The threshold value goes from 0 to 10,000 mapping to the value 0% to 100%. Strange as if someone divided with 100 ones to many.

[neily]

OK,

Here's another couple of useful function, which shows areas below a certain luma threshold bright green and above a threshold bright red, and just the areas outside the thresholds against a grey background. With the first, there is the option just to show the overlay over the original clip or a blank black clip. With the second there is the option to show the light/dark areas against a uniform grey background or a green-tinged version of the original. They are dependent on stickboy's Threshold plugin. I didn't manage to come up with a way of doing it natively. Being old-fashioned, I still load plugins manually, and it doesn't seem to give any processing overhead, but you may prefer to move plugin loading out of the function.


function ShowColouredMinMax(clip c,int thresh1,int thresh2,bool overlayonly)
{
#thresh1 - colour pixels green BELOW this threshold
#thresh2 - colour pixels red ABOVE this threshold
#overlayonly - show against black background
LoadPlugin("E:\Video\AviSynth\AVISynth v2.5.5\plugins\Threshold.dll")
green=BlankClip(c,color=$00FF00)
red=BlankClip(c,color=$FF0000)
black=BlankClip(c,color=$000000)
lowmask = Threshold(c,threshold=thresh1,low=0,high=255).Invert()
highmask = Threshold(c,threshold=thresh2+1,low=0,high=255)
return overlayonly ? Overlay(Overlay(black,green,0,0,lowmask,mode="blend"),red,0,0,highmask,mode="blend"):\
Overlay(Overlay(c,green,0,0,lowmask,mode="blend"),red,0,0,highmask,mode="blend")
}



function ShowJustMinMax(clip c,int thresh1,int thresh2,bool showgreen)
{
#thresh1 - pixels BELOW this threshold are displayed as non-grey
#thresh2 - pixels ABOVE this threshold are displayed as non-grey
#showgreen - true:thresholds displayed against green-tinged original clip
#showgreen - false:thresholds displayed against uniform grey background
LoadPlugin("E:\Video\AviSynth\AVISynth v2.5.5\plugins\Threshold.dll")
grey=BlankClip(c,color=$808080)
greenclip=Overlay(c.Levels(0,1,255,100,156),green,0,0,opacity=0.6,mode="softlight")
lowmask = Threshold(c,threshold=thresh1,low=0,high=255).Invert()
highmask = Threshold(c,threshold=thresh2+1,low=0,high=255)
return showgreen ? Overlay(Overlay(greenclip,c,0,0,lowmask,mode="blend"),c,0,0,highmask,mode="blend"):\
Overlay(Overlay(grey,c,0,0,lowmask,mode="blend"),c,0,0,highmask,mode="blend")
}

stickboy's Threshold filter (http://www.avisynth.org/stickboy/Threshold.zip)