Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th March 2013, 20:03   #1  |  Link
bennynihon
Registered User
 
Join Date: Oct 2001
Posts: 106
Measure grain level of clip? (how to use static type variable to calc avg?)

Is there a way, through an AVISynth script, to put some number (even if it's just some unit-less number with little meaning other than being a scale) on the amount of grain contained in a clip? I suppose it could be normalized in a range of 0.0 - 1.0, where the values indicate the amount of graininess. Numbers closer to 0 indicate less grain, while numbers closer to 1.0 indicate more grain.

The reason I ask, is it'd be nice to first run a small portion of a clip (N frames worth) through a script that measures graininess, and then I adjust whether or not to degrain the source (and with what strength) before fulling encoding it. Perhaps I would bucket the clips. Movies with grain between 0.0 - 0.2 get these settings, movies between 0.2 - 0.4 get these settings, etc.

Too much grain and the file size of an x264 encode is huge. Too little grain and a lot of banding can result in the x264 encode.

Last edited by bennynihon; 25th March 2013 at 18:12.
bennynihon is offline   Reply With Quote
Old 20th March 2013, 02:50   #2  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 597
GrainEvaluate
The grain strength values are not normalized in the range of 0.0~1.0 though. From the author's description, roughly Y strength values in 0.2~0.4 are weak grain, 0.4~0.7 are medium, 0.7~1.2 are strong, >1.2 should be quite heavy.
HolyWu is offline   Reply With Quote
Old 20th March 2013, 10:12   #3  |  Link
bennynihon
Registered User
 
Join Date: Oct 2001
Posts: 106
Quote:
Originally Posted by HolyWu View Post
GrainEvaluate
The grain strength values are not normalized in the range of 0.0~1.0 though. From the author's description, roughly Y strength values in 0.2~0.4 are weak grain, 0.4~0.7 are medium, 0.7~1.2 are strong, >1.2 should be quite heavy.
Cool. I'll give it a shot tomorrow. Thanks!
bennynihon is offline   Reply With Quote
Old 25th March 2013, 18:11   #4  |  Link
bennynihon
Registered User
 
Join Date: Oct 2001
Posts: 106
So the GrainEvaluate function is below. Is there a way to use a static variable (or mimic one) that accumulates the grain that's calculated for each frame and finds the average grain for the entire clip? I tried using a global variable (ydiff) in my main script and have the GrainEvaluate function accumulate the value to that global variable (the 'ydiff = ydiff + grainStrY' line below). However, it complains that it doesn't know what grainStrY is. Odd because it's assigned a value directly above in the FrameEvaluate line.

Code:
Function GrainEvaluate(clip c, string "file", bool "Y", bool "U", bool "V"){
  file = Default( file, "GrainEvaluateFile.txt")
  Y    = Default( Y,    true )
  U    = Default( U,   false )
  V    = Default( V,       U )

  last = c
  global GE_nr = RemoveGrain(Y?4:-1, U?4:-1, V?4:-1)

  global sepF  = "frame "
  global sepY  = Y ? "; Y grain strength = " : ""
  global sepU  = U ? "; U grain strength = " : ""
  global sepV  = V ? "; V grain strength = " : ""

  Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)"   ) : nop
  U ? FrameEvaluate("global grainStrU = ChromaUDifference(GE_nr)") : nop
  V ? FrameEvaluate("global grainStrV = ChromaVDifference(GE_nr)") : nop

  ydiff = ydiff + grainStrY

  WriteFile(file,
  \         "sepF", "current_frame",
  \         "sepY", Y ? "grainStrY" : "",
  \         "sepU", U ? "grainStrU" : "",
  \         "sepV", V ? "grainStrV" : "",
  \         append=false)

  return last
}
bennynihon is offline   Reply With Quote
Old 25th March 2013, 19:41   #5  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,166
Quote:
Originally Posted by bennynihon View Post
So the GrainEvaluate function is below. Is there a way to use a static variable (or mimic one) that accumulates the grain that's calculated for each frame and finds the average grain for the entire clip?
I don't have a direct answer to your question as asked, but I wonder if you really want to accumulate grain information for the entire clip. You might get better results instead if you used a moving average of perhaps half a dozen frames on either side of each frame. This is what I did when solving a completely different problem as I posted here:

function GenerateMask

Scroll down in the code section to the last function in the code.

The reason I did it that way is that the luma differences (which is what I was trying to estimate, instead of grain) tend to vary over the course of any clip, and the only thing that matters is the behavior in the local vicinity of each frame. By using a moving average, the function adapts to local conditions and therefore produces better results. If this applies to your situation, perhaps you can adapt the idea (and the code) to your function.

If this IS useful to you, then perhaps you could add some code to adapt the moving average at scene changes, something I didn't bother to do.
johnmeyer is offline   Reply With Quote
Old 25th March 2013, 19:47   #6  |  Link
bennynihon
Registered User
 
Join Date: Oct 2001
Posts: 106
@johnmeyer

I'm only using this to ascertain how "grainy" a clip is (I use SelectRangeEvery to analyze just a few minutes of a long movie), and then choose whether or not (and how strong) to degrain it using MDegrain with motion compensation.

The idea is to use something like the GrainEvaluate function (which currently prints out the grain level per frame to a file), but instead accumulate those per frame grain values to calculate an average over the frames I evaluate. It doesn't have to be perfect, but with enough of a sample size it gives a good relative metric for how grainy a movie is.
bennynihon is offline   Reply With Quote
Old 25th March 2013, 19:49   #7  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,952
Look into what JM said, but this might be an immediate problem you have
Code:
    Global ydiff = ydiff + grainStrY # need to use Global when assigning to a global,
you'de also have to divide by the number of sampled frames.


Also a little code that gives more accurate results over entire clip (float is quite limited)

Code:
# Requires GScript and ShowChannels plugins. 
AviSource("D:\avs\avi\1.avi").trim(0,0999)

   Sum = 0.0
   SumHi=0.0    # Hi part Extra precision (need for long scans)
   SumLo=0.0    # Lo part Extra precision
   GScript("""
     for(n=0,FrameCount()-1) {
       Tmp = RT_AverageLuma(n)      
       RT_Debug("RT_Debug:","["+String(n)+"]","AveLuma = "+String(Tmp)) # Real Time Debug Info using DebugView
 #       Sum = Sum + Tmp    # Were gonna use extra precision instead    
       SumLo = SumLo + Tmp
       SumHi = SumHi + floor(SumLo / 256.0)  # Hi part, Extra Precision
       SumLo = frac(SumLo / 256.0) * 256.0   # Lo part, Extra Precision
     }
   """)
 #   Sum=Sum/Float(FrameCount())     # Were gonna use extra precision instead
   Sum = (SumHi / Float(FrameCount()) * 256.0) + (SumLo / Float(FrameCount()))
   s=ShowChannels(ver=true)          # Show eg Ave Luma for frame and Accumulated Ave luma for all frames visited
   s.Subtitle(String(Sum),Align=1)
   
 # The sum variable shown on bottom left via Subtitle, is the accumulated Average Luma for entire clip, will be valid on the LAST FRAME.
 # ShowChannels used for result comparison.
The relevant parts of above are the Sum, SumHi and SumLo functionality, keeping maximum precision.

EDIT:
Quote:
Originally Posted by bennynihon View Post
However, it complains that it doesn't know what grainStrY is.
Think you may need to use the After_Frame=true arg to FrameEvaluate()
ie
Code:
  Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)" ,after_frame=true   ) : nop
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 25th March 2013 at 20:25.
StainlessS is offline   Reply With Quote
Old 25th March 2013, 20:03   #8  |  Link
bennynihon
Registered User
 
Join Date: Oct 2001
Posts: 106
Quote:
Originally Posted by StainlessS View Post
Look into what JM said, but this might be the immediate problem you had
Code:
    Global ydiff = ydiff + grainStrY # need to use Global when assigning to a global,
you'de also have to divide by the number of sampled frames.
I have ydiff defined as a global in my main script (outside of this function) to initialize it. That's also where I divide it by the total number of frames.

Quote:
EDIT:
Think you may need to use the After_Frame=true arg to FrameEvaluate()
ie
Code:
  Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)" ,after_frame=true   ) : nop
I'll have to try this out. Thanks.
bennynihon is offline   Reply With Quote
Old 25th March 2013, 20:11   #9  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,952
Quote:
Originally Posted by StainlessS View Post
Global ydiff = ydiff + grainStrY # need to use Global when assigning to a global,
From docs

Quote:
To define and / or assign a value to a global variable you must precede its name with the keyword global at the left side of the assignment. The keyword is not needed (actually it is not allowed) in order to read the value of a global variable.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 25th March 2013, 20:15   #10  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,380
Quote:
Originally Posted by bennynihon View Post
... it complains that it doesn't know what grainStrY is. Odd because it's assigned a value directly above in the FrameEvaluate line.
That assignment happens (for each frame) at run-time, but your assignment to ydiff is done (once only) at compile-time, when grainStrY does not yet exist. You need to put that assignment inside a FrameEvaluate as well.
Code:
FrameEvaluate("global ydiff = ydiff + grainStrY", after_frame=true)
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 25th March 2013, 20:17   #11  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,952
Aha, Big G to the rescue again.

EDIT: Does the After_frame=true, also hold or not ?

EDIT: Oops, you already included that, but is it true of the

Code:
Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)" ,after_frame=true   ) : nop
line

See you replying, Thanx in advance.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 25th March 2013 at 20:38.
StainlessS is offline   Reply With Quote
Old 25th March 2013, 20:41   #12  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,380
Quote:
Originally Posted by StainlessS View Post
Does the After_frame=true, also hold or not ?

EDIT: Oops, you already included that, but is it true of the

Code:
Y ? FrameEvaluate("global grainStrY = LumaDifference(GE_nr)" ,after_frame=true   ) : nop
line
It's not necessary there, since the first three (Y, U, V) assignments can be done in any order, and the only thing that matters is that the assignment to ydiff is done last.

@bennynihon: Another problem I foresee with your script is that you will need to use WriteFile() to get the final value of ydiff out of your script, since it is accumulated at run-time.
You could use the ScriptClip("WriteFile(..., append=false")) trick (see this post) so that only the final value is left in the file.

An alternative would be to loop over the frames at compile-time using GScript, as in StainlessS's approach in post #7.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 25th March 2013, 20:46   #13  |  Link
bennynihon
Registered User
 
Join Date: Oct 2001
Posts: 106
Quote:
Originally Posted by Gavino View Post
@bennynihon: Another problem I foresee with your script is that you will need to use WriteFile() to get the final value of ydiff out of your script, since it is accumulated at run-time.
You could use the ScriptClip("WriteFile(..., append=false")) trick (see this post) so that only the final value is left in the file.
Thanks so much for the help. What about WriteFileEnd()? Couldn't that work to get the final value of ydiff?
bennynihon is offline   Reply With Quote
Old 25th March 2013, 20:50   #14  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,380
Quote:
Originally Posted by bennynihon View Post
What about WriteFileEnd()? Couldn't that work to get the final value of ydiff?
Unfortunately not.
Although WriteFileEnd() doesn't actually write its output until script closure, the value itself is evaluated at compile-time, so is no good for what you want to do here.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 25th March 2013, 20:56   #15  |  Link
bennynihon
Registered User
 
Join Date: Oct 2001
Posts: 106
Quote:
Originally Posted by Gavino View Post
Unfortunately not.
Although WriteFileEnd() doesn't actually write its output until script closure, the value itself is evaluated at compile-time, so is no good for what you want to do here.
Awesome. It all makes sense now.
bennynihon is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:37.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.