Measure grain level of clip? (how to use static type variable to calc avg?)

[bennynihon]

Is there a way, through an AVISynth script, to put some number (even if it's just some unit-less number with little meaning other than being a scale) on the amount of grain contained in a clip? I suppose it could be normalized in a range of 0.0 - 1.0, where the values indicate the amount of graininess. Numbers closer to 0 indicate less grain, while numbers closer to 1.0 indicate more grain.

The reason I ask, is it'd be nice to first run a small portion of a clip (N frames worth) through a script that measures graininess, and then I adjust whether or not to degrain the source (and with what strength) before fulling encoding it. Perhaps I would bucket the clips. Movies with grain between 0.0 - 0.2 get these settings, movies between 0.2 - 0.4 get these settings, etc.

Too much grain and the file size of an x264 encode is huge. Too little grain and a lot of banding can result in the x264 encode.

[bennynihon]

GrainEvaluate (http://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=805)
The grain strength values are not normalized in the range of 0.0~1.0 though. From the author's description, roughly Y strength values in 0.2~0.4 are weak grain, 0.4~0.7 are medium, 0.7~1.2 are strong, >1.2 should be quite heavy.

Cool. I'll give it a shot tomorrow. Thanks!

[bennynihon]

So the GrainEvaluate function is below. Is there a way to use a static variable (or mimic one) that accumulates the grain that's calculated for each frame and finds the average grain for the entire clip? I tried using a global variable (ydiff) in my main script and have the GrainEvaluate function accumulate the value to that global variable (the 'ydiff = ydiff + grainStrY' line below). However, it complains that it doesn't know what grainStrY is. Odd because it's assigned a value directly above in the FrameEvaluate line.

Function GrainEvaluate(clip c, string "file", bool "Y", bool "U", bool "V"){
file = Default( file, "GrainEvaluateFile.txt")
Y = Default( Y, true )
U = Default( U, false )
V = Default( V, U )

last = c
global GE_nr = RemoveGrain(Y?4:-1, U?4:-1, V?4:-1)

global sepF = "frame "
global sepY = Y ? "; Y grain strength = " : ""
global sepU = U ? "; U grain strength = " : ""
global sepV = V ? "; V grain strength = " : ""

Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)" ) : nop
U ? FrameEvaluate("global grainStrU = ChromaUDifference(GE_nr)") : nop
V ? FrameEvaluate("global grainStrV = ChromaVDifference(GE_nr)") : nop

ydiff = ydiff + grainStrY

WriteFile(file,
\ "sepF", "current_frame",
\ "sepY", Y ? "grainStrY" : "",
\ "sepU", U ? "grainStrU" : "",
\ "sepV", V ? "grainStrV" : "",
\ append=false)

return last
}

[johnmeyer]

So the GrainEvaluate function is below. Is there a way to use a static variable (or mimic one) that accumulates the grain that's calculated for each frame and finds the average grain for the entire clip?I don't have a direct answer to your question as asked, but I wonder if you really want to accumulate grain information for the entire clip. You might get better results instead if you used a moving average of perhaps half a dozen frames on either side of each frame. This is what I did when solving a completely different problem as I posted here:

function GenerateMask (http://forum.doom9.org/showthread.php?p=1510813#post1510813)

Scroll down in the code section to the last function in the code.

The reason I did it that way is that the luma differences (which is what I was trying to estimate, instead of grain) tend to vary over the course of any clip, and the only thing that matters is the behavior in the local vicinity of each frame. By using a moving average, the function adapts to local conditions and therefore produces better results. If this applies to your situation, perhaps you can adapt the idea (and the code) to your function.

If this IS useful to you, then perhaps you could add some code to adapt the moving average at scene changes, something I didn't bother to do.

[bennynihon]

@johnmeyer

I'm only using this to ascertain how "grainy" a clip is (I use SelectRangeEvery to analyze just a few minutes of a long movie), and then choose whether or not (and how strong) to degrain it using MDegrain with motion compensation.

The idea is to use something like the GrainEvaluate function (which currently prints out the grain level per frame to a file), but instead accumulate those per frame grain values to calculate an average over the frames I evaluate. It doesn't have to be perfect, but with enough of a sample size it gives a good relative metric for how grainy a movie is.

[StainlessS]

Look into what JM said, but this might be an immediate problem you have

Global ydiff = ydiff + grainStrY # need to use Global when assigning to a global,


you'de also have to divide by the number of sampled frames.


Also a little code that gives more accurate results over entire clip (float is quite limited)


# Requires GScript and ShowChannels plugins.
AviSource("D:\avs\avi\1.avi").trim(0,0999)

Sum = 0.0
SumHi=0.0 # Hi part Extra precision (need for long scans)
SumLo=0.0 # Lo part Extra precision
GScript("""
for(n=0,FrameCount()-1) {
Tmp = RT_AverageLuma(n)
RT_Debug("RT_Debug:","["+String(n)+"]","AveLuma = "+String(Tmp)) # Real Time Debug Info using DebugView
# Sum = Sum + Tmp # Were gonna use extra precision instead
SumLo = SumLo + Tmp
SumHi = SumHi + floor(SumLo / 256.0) # Hi part, Extra Precision
SumLo = frac(SumLo / 256.0) * 256.0 # Lo part, Extra Precision
}
""")
# Sum=Sum/Float(FrameCount()) # Were gonna use extra precision instead
Sum = (SumHi / Float(FrameCount()) * 256.0) + (SumLo / Float(FrameCount()))
s=ShowChannels(ver=true) # Show eg Ave Luma for frame and Accumulated Ave luma for all frames visited
s.Subtitle(String(Sum),Align=1)

# The sum variable shown on bottom left via Subtitle, is the accumulated Average Luma for entire clip, will be valid on the LAST FRAME.
# ShowChannels used for result comparison.


The relevant parts of above are the Sum, SumHi and SumLo functionality, keeping maximum precision.

EDIT:
However, it complains that it doesn't know what grainStrY is.Think you may need to use the After_Frame=true arg to FrameEvaluate()
ie

Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)" ,after_frame=true ) : nop

[bennynihon]

Look into what JM said, but this might be the immediate problem you had

Global ydiff = ydiff + grainStrY # need to use Global when assigning to a global,


you'de also have to divide by the number of sampled frames.

I have ydiff defined as a global in my main script (outside of this function) to initialize it. That's also where I divide it by the total number of frames.


EDIT:
Think you may need to use the After_Frame=true arg to FrameEvaluate()
ie

Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)" ,after_frame=true ) : nop


I'll have to try this out. Thanks.

[StainlessS]

Global ydiff = ydiff + grainStrY # need to use Global when assigning to a global,


From docs

To define and / or assign a value to a global variable you must precede its name with the keyword global at the left side of the assignment. The keyword is not needed (actually it is not allowed) in order to read the value of a global variable.

[Gavino]

... it complains that it doesn't know what grainStrY is. Odd because it's assigned a value directly above in the FrameEvaluate line.
That assignment happens (for each frame) at run-time, but your assignment to ydiff is done (once only) at compile-time, when grainStrY does not yet exist. You need to put that assignment inside a FrameEvaluate as well.
FrameEvaluate("global ydiff = ydiff + grainStrY", after_frame=true)

[StainlessS]

Aha, Big G to the rescue again. :)

EDIT: Does the After_frame=true, also hold or not ?

EDIT: Oops, you already included that, but is it true of the


Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)" ,after_frame=true ) : nop


line

See you replying, Thanx in advance.

[Gavino]

Does the After_frame=true, also hold or not ?

EDIT: Oops, you already included that, but is it true of the


Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)" ,after_frame=true ) : nop


line
It's not necessary there, since the first three (Y, U, V) assignments can be done in any order, and the only thing that matters is that the assignment to ydiff is done last.

@bennynihon: Another problem I foresee with your script is that you will need to use WriteFile() to get the final value of ydiff out of your script, since it is accumulated at run-time.
You could use the ScriptClip("WriteFile(..., append=false")) trick (see this post) so that only the final value is left in the file.

An alternative would be to loop over the frames at compile-time using GScript, as in StainlessS's approach in post #7.

[bennynihon]

@bennynihon: Another problem I foresee with your script is that you will need to use WriteFile() to get the final value of ydiff out of your script, since it is accumulated at run-time.
You could use the ScriptClip("WriteFile(..., append=false")) trick (see this post) so that only the final value is left in the file.


Thanks so much for the help. What about WriteFileEnd()? Couldn't that work to get the final value of ydiff?

[Gavino]

What about WriteFileEnd()? Couldn't that work to get the final value of ydiff?
Unfortunately not.
Although WriteFileEnd() doesn't actually write its output until script closure, the value itself is evaluated at compile-time, so is no good for what you want to do here.

[bennynihon]

Unfortunately not.
Although WriteFileEnd() doesn't actually write its output until script closure, the value itself is evaluated at compile-time, so is no good for what you want to do here.

Awesome. It all makes sense now.