Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
20th September 2021, 01:01 | #1 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
TweakYPlaneNoiseThresh() - script
Might come in useful.
TweakYPlaneNoiseThresh.avsi Code:
# TweakYPlaneNoiseThresh.avsi /* TweakYPlaneNoiseThresh() v0.04, by StainlessS @ Doom9. # https://forum.doom9.org/showthread.php?p=1952709#post1952709 Req AVS+, RT_Stats, Grunt. Any Planar Y/YUV/A. [depth 8 -> 32]. Shows results of some YPlane operation [YPlaneMin, YPlaneMax, YPlaneMinMaxDifference] on sampled clip, with multiple simultaneous Threshold settings. TweakYPlaneNoiseThresh(clip c,int "YPlaneOp"=0,Float "NoiseBase"=0.0,Float "NoiseStep"=0.005,Int "Rows"=21) eg YPlaneMin(Threshold=NoiseTh) Threshold is a percentage, the maximum percent population of the pixels that are allowed to be ignored below minimum (ignore extreme pixels ie noise). YPlaneMin first creates a pixel population count array of the current frame, and then it creates a running total of pixel population stepping from 0 upwards. When the running total pixel population % is greater or equal to threshold arg, it returns the index of the array element that it has so far reached. If the lowest populated array elements total less than Threshold, they are ignored. If the lowest populated element % population is greater or equal to Threshold, then nothing is ignored and that lowest poplated pixel element index is returned. YPlaneMin(threshold=0.0) ignores nothing and returns the index of the lowest populated element in the count array. You might use eg YPlaneMin(Threshold=0.1) to ignore (at most) the darkest 0.1% of pixels when finding minimum pixel luma. YplaneMax(Threshold=0.2), ignores lightest (at most) 0.2% of pixels. YplaneMinMaxDifference(Threshold=0.3), ignores (at most) the darkest 0.3% and lightest 0.3% of pixels. YPlaneMedian(), is implemented as YPlaneMin(Threshold=50.0). Metrics: (only 5 lines of results out of 21) 0] TweakYPlaneNoiseThresh: YPlaneMin @ NoiseTh % (1280x720P10) Ignore up to Ignore single NoiseTh OneInXPixels Square of Pixels YPlaneMin(NoiseTh) 0.000 % 0 0x0 152 0.050 % 2000 21x21 168 0.100 % 1000 30x30 172 0.150 % 666 37x37 172 0.200 % 500 43x43 172 To specify to ignore at most 1 in every 500 darkest pixels in a frame, you can use YPlaneMin(Threshold=100.0/500.0) [as the 5th result line above]. The "Ignore single Square of pixels" entry shows the approx max size of a non noise object that could potentially be ignored by mistake [when no other lower level noise is on frame]. "Ignore single Square of pixels" size shown will vary with frame size. The rightmost "YPlaneMin(NoiseTh)" column shows the actual result from (in this case) YPlaneMin(). Args: YPlaneOp, default 0, 0->2, 0=YPlaneMin(), 1=YPlaneMax(), 2=YPlaneMinMaxDifference(). NoiseBase, default 0.00, 0.0->1.0, Lowest Noise Threshold tested. NoiseStep default 0.005, 0.0<NoiseStep<=1.0, Noise Theshold step between 21 different Noise Thresholds, relative to NoiseBase. Rows, default 21, 1 -> max allowed by frame height (silent limit). Number of rows displayed. [the more rows, the slower] */ Function TweakYPlaneNoiseThresh(clip c,int "YPlaneOp",Float "NoiseBase",Float "NoiseStep",Int "Rows") { c myName="TweakYPlaneNoiseThresh: " YPlaneOp = Default(YPlaneOp,0) NoiseBase = Default(NoiseBase,0.0) NoiseStep = Default(NoiseStep,0.005) Assert(0 <= YPlaneOp <= 2,myName+"0 <= YPlaneOp <= 2") Assert(0.0 <= NoiseBase <= 1.0,myName+"0.0 <= NoiseBase <= 1.0") Assert(0.0 < NoiseStep <= 1.0,myName+"0.0 < NoiseStep <= 1.0") W=Width H=Height RowMax=(H/18)-3 bpc=BitsPerComponent Rows=Default(Rows,21).Max(1).Min(RowMax) Assert(!IsRGB&&IsPlanar,myName+"Planar with Y only") YOP=Select(YPLaneOp,"YPlaneMin","YPLaneMax","YPlaneMinMaxDifference") evS="T%d=NoiseBase+(i*NoiseStep) I%d=(T%d>0.0?100.0/T%d:0).Int S%d=(T%d>0.0?SqRt(P*T%d/100).Round:0).Int Y%d="+YOP+"(T%d)" # Newlines are a PITA, gotta leave "\\n" for Scriptclip Eval("S=RT_string..."), so using here "\\\\n" to leave "\\n" in Scriptclip. [Same for "%%%%"="%%"] dhS=RT_String("""RT_String("%%d] TweakYPlaneNoiseThresh: %s @ NoiseTh %%%% (%dx%dP%d)\\\\n Ignore up to Ignore single\\\\n""" + \ "NoiseTh%%%% OneInXPixels Square of pixels %s(NoiseTh)\\\\n",YOP,W,H,bpc,YOP) drS="%.6f %10d %5dx%-5d " + (bpc==32?"%f":"%5d") dvS="\ T%d,I%d,S%d,S%d,Y%d" vS=","+Chr(10)+"\ n,"+chr(10) for(i=0,Rows-1) { dhS=RT_String("%s%s%s",dhS,drS,(i==Rows-1?Chr(34):"\\n")) vS=RT_String("%s%s%s",vS,RT_String(dvS,i,i,i,i,i),(i==Rows-1)?")":","+Chr(10)) } rtS=dhS+vS # DbgS=rts.RT_StrReplace("\\n",Chr(10)) RT_WriteFile(".\TweakYPlaneNoiseThresh_DEBUG.txt","%s",DbgS) RT_Debug(DbgS) # Output Debugging to file and DebugView SSS=""" n=current_frame W=Width H=Height P=W*H For(i=0,Rows-1) { Eval(RT_String(evS,i,i,i,i,i,i,i,i,i)) } Return Subtitle(Eval(rtS),lsp=0,Font="Courier New") """ Return GScriptClip(SSS,Args="NoiseBase,NoiseStep,Rows,evS,rtS",Local=True) } Code:
##### CONFIG ####### #Avisource(".\OUT\TEST1.avi") #FfVideoSource(".\OUT\TEST1.avi") AviSource(".\BP-KTL.AVI") #ConvertBits(10) YPlaneOp = 0 # 0) 0->2, 0=YPlaneMin(), 1=YPlaneMax(), 2=YPlaneMinMaxDifference(). NoiseBase = 0.00 # 0.0) 0.0->1.0, Lowest Noise Threshold shown. NoiseStep = 0.005 # 0.005) 0.0<NoiseStep<=1.0, Noise Theshold stepping. Rows = 9999 # 9999, All of em' (silent limited to frame height). ##### END CONFIG ###### Return TweakYPlaneNoiseThresh(YPlaneOp,NoiseBase,NoiseStep,Rows)
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 10th October 2021 at 13:46. Reason: update |
20th September 2021, 07:56 | #2 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
I don't see the wives! 😤
But usefull script 😁
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database Last edited by ChaosKing; 20th September 2021 at 07:59. |
20th September 2021, 11:33 | #3 | Link |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
|
maybe you will say "why not both?" but more wives will destroy your life
anyway, StainlessS, "Req AVS+" but you still use old grun instead of http://avisynth.nl/index.php/Function_objects see here https://github.com/Dogway/Avisynth-S...3.0d.avsi#L173 and https://github.com/Dogway/Avisynth-S...0c5f4747704439
__________________
See My Avisynth Stuff Last edited by real.finder; 20th September 2021 at 11:40. |
20th September 2021, 14:13 | #4 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
I had to puzzle over that for quite a time, A picture is worth a thousand words.
Old Dog, Old Tricks. (Dogway [EDIT: And Real.Finder] must be just a puppy). AVS+ req, for native HBD YPlane...(), and builtin GScript stuff. I should really show bit depth on frame too. Currently modding to make user selectable number of rows [maybe 21 rows too much for some, have not tested on slow machine, only i7-8700]. Will repost original script [maybe showing bit depth] later in thread, so it can be seen what the update is doing. [The update will be very cryptic gobble-de-gook (to make the variable rows stuff work)]. EDIT: Original script, updates will be more cryptic to understand. Code:
# TweakYPlaneNoiseThresh.avsi /* TweakYPlaneNoiseThresh(), by StainlessS @ Doom9. # https://forum.doom9.org/showthread.php?p=1952709#post1952709 Req AVS+, RT_Stats, Grunt. Any PLanar Y/YUV/A. [depth 8 -> 16]. Shows results of some YPlane operation [YPlaneMin, YPlaneMax, YPlaneMinMaxDifference] on sampled clip, with 21 simultaneous Threshold settings. TweakYPlaneNoiseThresh(clip c,int "YPlaneOp"=0,Float "NoiseBase"=0.0,Float "NoiseStep"=0.05) eg YPlaneMin(Threshold=NoiseTh) Threshold is a percentage, the maximum percent population of the pixels that are allowed to be ignored below minimum (ignore extreme pixels ie noise). YPlaneMin first creates a pixel population count array of the current frame, and then it creates a running total of pixel population stepping from 0 upwards. When the running total pixel population % is greater or equal to threshold arg, it returns the index of the array element that it has so far reached. If the lowest populated array elements total less than Threshold, they are ignored. If the lowest populated element % population is greater or equal to Threshold, then nothing is ignored and that lowest poplated pixel element index is returned. YPlaneMin(threshold=0.0) ignores nothing and returns the index of the lowest populated element in the count array. You might use eg YPlaneMin(Threshold=0.1) to ignore (at most) the darkest 0.1% of pixels when finding minimum pixel luma. YplaneMax(Threshold=0.2), ignores lightest (at most) 0.2% of pixels. YplaneMinMaxDifference(Threshold=0.3), ignores (at most) the darkest and lightest 0.3% of pixels. YPlaneMedian, is implemented as YPlaneMin(Threshold=50.0). Metrics: (only 5 lines of results out of 21) 0] TweakYPlaneNoiseThresh: YPlaneMin @ NoiseTh % (1280x720P10) Ignore up to Ignore single NoiseTh OneInXPixels Square of Pixels YPlaneMin(NoiseTh) 0.000 % 0 0x0 152 0.050 % 2000 21x21 168 0.100 % 1000 30x30 172 0.150 % 666 37x37 172 0.200 % 500 43x43 172 To specify to ignore at most 1 in every 500 darkest pixels in a frame, you can use YPlaneMin(Threshold=100.0/500.0) [as the 5th result line above]. The "Ignore single Square of pixels" entry shows the approx max size of a non noise object that could potentially be ignored by mistake [when no other noise is on frame]. The rightmost "YPlaneMin(NoiseTh)" column shows the actual result from (in this case) YPlaneMin(). Args: YPlaneOp, default 0, 0->2, 0=YPlaneMin(), 1=YPlaneMax(), 2=YPlaneMinMaxDifference(). NoiseBase, default 0.00, 0.0->1.0, Lowest Noise Threshold tested. NoiseStep default 0.05, 0.0<NoiseStep<=1.0, Noise Theshold step between 21 different Noise Thresholds, relative to NoiseBase. */ Function TweakYPlaneNoiseThresh(clip c,int "YPlaneOp",Float "NoiseBase",Float "NoiseStep") { c myName = "TweakYPlaneNoiseThresh: " YPlaneOp = Default(YPlaneOp,0) NoiseBase = Default(NoiseBase,0.0) NoiseStep = Default(NoiseStep,0.05) Assert(0 <= YPlaneOp <= 2,myName+"0 <= YPlaneOp <= 2") Assert(0.0 <= NoiseBase <= 1.0,myName+"0.0 <= NoiseBase <= 1.0") Assert(0.0 < NoiseStep <= 1.0,myName+"0.0 < NoiseStep <= 1.0") YOP = Select(YPLaneOp,"YPlaneMin","YPLaneMax","YPlaneMinMaxDifference") evS="T%d=NoiseBase+(i*NoiseStep) I%d=(T%d>0.0?100.0/T%d:0).Int S%d=(T%d>0.0?SqRt(P*T%d/100).Round:0).Int Y%d="+YOP+"(T%d)" SSS=(""" n = current_frame W=Width H=Height P=W*H bpc=BitsPerComponent For(i=0,20) { Eval(RT_String(evS,i,i,i,i,i,i,i,i,i)) } S=RT_String( \ "%d] TweakYPlaneNoiseThresh: %s @ NoiseTh %% (%dx%dP%d)\\n" + \ " Ignore up to Ignore single\\n" + \ "NoiseTh OneInXPixels Square of pixels %s(NoiseTh)\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" + \ "%.3f %% %6d %4dx%-4d %5d\\n" \ ,n,YOP,W,H,bpc,YOP, \ T0,I0,S0,S0,Y0,T1,I1,S1,S1,Y1,T2,I2,S2,S2,Y2,T3,I3,S3,S3,Y3,T4,I4,S4,S4,Y4,T5,I5,S5,S5, \ Y5,T6,I6,S6,S6,Y6,T7,I7,S7,S7,Y7,T8,I8,S8,S8,Y8,T9,I9,S9,S9,Y9,T10,I10,S10,S10,Y10, \ T11,I11,S11,S11,Y11,T12,I12,S12,S12,Y12,T13,I13,S13,S13,Y13,T14,I14,S14,S14,Y14, \ T15,I15,S15,S15,Y15,T16,I16,S16,S16,Y16,T17,I17,S17,S17,Y17,T18,I18,S18,S18,Y18, \ T19,I19,S19,S19,Y19,T20,I20,S20,S20,Y20) Return Subtitle(S,lsp=0,Font="Courier New") """) Return GScriptClip(SSS,Args="YOP,NoiseBase,NoiseStep,evS",Local=True) } Code:
##### CONFIG ####### #Avisource(".\OUT\TEST1.avi") FfVideoSource(".\OUT\TEST1.avi") YPlaneOp = 0 # 0) 0->2, 0=YPlaneMin(), 1=YPlaneMax(), 2=YPlaneMinMaxDifference(). NoiseBase = 0.00 # 0.0) 0.0->1.0, Lowest Noise Threshold shown. NoiseStep = 0.05 # 0.05) 0.0<NoiseStep<=1.0, Noise Theshold stepping. ##### END CONFIG ###### Return TweakYPlaneNoiseThresh(YPlaneOp,NoiseBase,NoiseStep)
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 20th September 2021 at 15:42. |
20th September 2021, 15:44 | #5 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
First post updated with display of bit depth on frame, [ Just shows eg (1280x720P10) ]
Above post added this update and will leave there when further updated as is simpler to see what it does.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
20th September 2021, 19:37 | #6 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Update to post 1.
Now can select Rows (default 21), 1 -> height of frame (silent limit). Where possible, template strings are now constructed outside of scriptclip, should use less memory [although I dont see anybody scanning a 200,000 frame clip with it]. Code:
FfVideoSource(".\OUT\TEST1.avi") ConvertBits(16) TweakYPlaneNoiseThresh(Rows=999999999) EDIT: Above, 16 bit. EDIT: Zooming in a bit, changing NoiseStep from 0.05 to 0.01.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 20th September 2021 at 20:22. |
21st September 2021, 01:54 | #7 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
OK, here v0.03, supports 32 bit Float, see 1st post.
Was Code:
drS="%.3f %% %6d %4dx%-4d %5d" Code:
drS="%.3f %% %6d %4dx%-4d " + (bpc==32?"%f":"%5d") EDIT: Not tested much, [not much to test ] EDIT: Above with NoiseStep 0.01 instead of default 0.05 (same as prev image), [thought it was borked for a few seconds] Code:
FfVideoSource(".\OUT\TEST1.avi") ConvertBits(32) TweakYPlaneNoiseThresh(NoiseStep=0.01,Rows=999999999)
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 21st September 2021 at 02:45. |
29th September 2021, 21:02 | #8 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
1st post updeted, v0.04.
Allowed for smaller thresholds [more digits], changed default NoiseStep from 0.05 to 0.005. Fixed RowMax=(H/18)-3 [was RowMax=H/20] used client Code:
##### CONFIG ####### #Avisource(".\OUT\TEST1.avi") #FfVideoSource(".\OUT\TEST1.avi") AviSource(".\BP-KTL.AVI") #ConvertBits(10) YPlaneOp = 0 # 0) 0->2, 0=YPlaneMin(), 1=YPlaneMax(), 2=YPlaneMinMaxDifference(). NoiseBase = 0.00 # 0.0) 0.0->1.0, Lowest Noise Threshold shown. NoiseStep = 0.005 # 0.005) 0.0<NoiseStep<=1.0, Noise Theshold stepping. Rows = 9999 # 9999, All of em' (silent limited to frame height). ##### END CONFIG ###### Return TweakYPlaneNoiseThresh(YPlaneOp,NoiseBase,NoiseStep,Rows)
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 29th September 2021 at 21:23. |
4th October 2021, 20:10 | #10 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
However, this was the thread that prompted script function:- https://forum.doom9.org/showthread.p...21#post1952621
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
|
14th July 2022, 22:55 | #11 | Link |
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 702
|
Maybe I will sound dumb here but the pixels it is allowed to ignore is considered noise. So would this be suitable to use for realtime (ScriptClip) to estimate how noisy the frame is? I guess the output value is rather how much "noise free" the image is?
Would be interested to try this in my script where I dynamically adjust Thsad and Tradius with temporal denoisers. As for now I use a combo of averageLuma and rgbDifference to get some kind of noise estimation value but it has some problems. |
15th July 2022, 12:39 | #12 | Link | |||
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
Quote:
FranceBB in thread prompting the script, wanted to detect black sequence at end of clip, Quote:
Here is the script function that it was intended to help find best NoiseTh noise threshold for [I did not have any idea if was digital Black, or had some/much noise] From here:- https://forum.doom9.org/showthread.p...26#post1952526 Code:
Function FindNonBlackFramesAtEnd(clip c,Int YMaxTh,Float "NoiseTh",bool "Debug") { myName="FindNonBlackFramesAtEnd: " c Debug=Default(debug,False) FC = FrameCount result = 0 NoiseTh=Float(Default(NoiseTh,0.4)) for(n=FC-1,0,-1) { current_frame = n Y = YPlaneMax(Last,Threshold=NoiseTh) (Debug) ? RT_DebugF("%d] Y=%f",n,Y,name=myName) : NOP if(Y>YMaxTh) {result = n+1 n = -1 } } return result } EDIT: We did not want to trim off black at end of clip that was actually part of the video, (maybe with audio), only the extra artificially added frames that should be stripped off. EDIT: That function FindNonBlackFramesAtEnd() actually returns the frame number of the first artificial black frame, OR, the length of the NON Black clip. [probably a misleading function name - I'm shit at thinking of function names, they all start out as Fred(), or Bill() or similar]
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 16th July 2022 at 13:37. |
|||
Tags |
yplanemax, yplanemin, yplaneminmaxdifference |
Thread Tools | Search this Thread |
Display Modes | |
|
|