Runtime filtering

[Dogway]

Im trying to execute the next code, aimed to feed denoising values upon motion, but Im running in some syntax issues about input types, quotes, etc

Ideally I would write this in a single line, but spliced it to get things working first and then optimizing.

The error I get is "h had the wrong type":

v=TemporalSoften(3,255,255,0,2)
h=scriptclip("""string(YDifferenceFromPrevious(v)/10+0.5,"%1.2f")""")
s=string(tnlmeans(ax=3,ay=3,az=1,sx=2,sy=2,bx=0,by=0,h=h,a=2.0,sse=false))
filtered1=MT("Dither1Pre(flt="+s+",stacked=true)",2,2,true)

[cretindesalpes]

You have to put everything depending on h in the ScriptClip part. Also, from the doc (http://avisynth.org/mediawiki/ScriptClip):
"Restrictions" - the output of the script MUST be exactly like the clip delivered to ScriptClip (same colorspace, width and height).
Dither1Pre() doubles the clip height so it won't work. This one should be OK:
v = TemporalSoften(3,255,255,0,2)
filtered1 = Dither_convert_8_to_16 ().ScriptClip ("""
Dither_get_msb ()
h = string(YDifferenceFromPrevious(v)/10+0.5,"%1.2f")
s = "tnlmeans(ax=3,ay=3,az=1,sx=2,sy=2,bx=0,by=0,h="+h+",a=2.0,sse=false)"
MT("Dither1Pre(flt=s,stacked=true)",2,2,true)
""")

[Gavino]

Using MT() inside ScriptClip might not give any speed advantage since its startup overheads (thread creation, etc) will be incurred on every frame. Try it with and without MT() and see what difference you get.

[Dogway]

neuron2: Its hard to understand the behaviour, I know that the output of scriptclip is not a float, so you are just rephrasing my question. Any way to output float using YDifferenceFromPrevious? Do you know?

Thanks a lot cretindesalpes, my logic didn't think in that way, I thought that getting values was a separate thing and stored in a variable in float changing at every frame. Not able to use MT() is going to slow down things a lot! Regarding input=output maybe I can switch Dither_convert_8_to_16 to stackvertical(last,last), looking at the functions it seems to do the same. Same for Dither_get_msb. Well, its only to dont mix concepts...

[Gavino]

Any way to output float using YDifferenceFromPrevious?
...
I thought that getting values was a separate thing and stored in a variable in float changing at every frame.The only thing that (potentially) changes on every frame are variables in the code executed by ScriptClip (or whatever run-time filter). Variables in the main script are set just once, at compile-time.

YDifferenceFromPrevious does return a float, but its value can only be used inside the run-time environment. The only way of getting it 'out' is to write it to a file with WriteFile or encode it into the output clip in some way.

You should read the following pages to understand more:
http://avisynth.org/mediawiki/Runtime_environment
http://avisynth.org/mediawiki/The_script_execution_model

[Dogway]

Thanks for the links, overall I have the theory, but glue it all together and make it work is what I fail at. Getting it out with WriteFile isn't going to help me in anything (related to MT) if in the end I need to feed it inside a runtime function.

[Gavino]

You have to read all the relevant documentation several times before you get the full picture. Slowly it will become clearer. :)

I wasn't actually suggesting Writefile as a solution to your problem - just pointing out it's one of the few ways to get data out of the run-time environment.

[Dogway]

I wasn't actually suggesting Writefile as a solution...
It only was a redundant question to neuron2.

I have read those pages around 10 or 15 times each. I just haven't done a thesis but learned very good tips and tricks. Im not going to take your smiley as a hint for a possible solution because I dont think there is one : D

Still one of my main questions is unresolved: can cretindesalpes code "h" and "s" variables be nested inside the Dither1pre function? or isn't there a way in avisynth to do this that doesn't affect performance?

[Gavino]

can cretindesalpes code "h" and "s" variables be nested inside the Dither1pre function? or isn't there a way in avisynth to do this that doesn't affect performance?
Since "h" is derived from YDifferenceFromPrevious(), it varies from one frame to another, and so can only be evaluated inside ScriptClip (or one of its friends). Therefore (since, as I said, you can't get values out of the run-time environment) anything that uses "h" must also be inside ScriptClip.

However, this might work:
v=TemporalSoften(3,255,255,0,2)
s="""scriptclip("tnlmeans(ax=3,ay=3,az=1,sx=2,sy=2,bx=0,by=0,h=YDifferenceFromPrevious(v)/10+0.5,a=2.0,sse=false)")"""
filtered1=MT("Dither1Pre(flt=s,stacked=true)",2,2,true)

I'm not sure if ScriptClip works within MT (I suspect not), so you might have to remove the MT bit. But at least Dither1Pre would then be outside ScriptClip.

[Dogway]

Indeed MT didn't work, but I see that the only workaround for (nested) quotes is to take them out as variables. Since it was going to be so damn slow I chose dfttest with mdegrain temporal, both biased upon motion, very neat. I hope one day tritical makes a more optimized tnlmeans, because still I don't consider NLmeans reliable enough...
Also you can see how I "prepared" the source for motion evaluation, I dont know if there is a more proper way probably with scene detection, only asking just in case...

...
v=TemporalSoften(3,255,255,0,2)
d=stackvertical(last,last).scriptclip("""
h1 = round(300/(YDifferenceFromPrevious(v)/0.5+0.5)+250)
h2 = YDifferenceFromPrevious(v)/1+1
d=crop(0,0,0,-478).MDeGrain3(asuper, b1v, f1v, b2v, f2v, b3v, f3v, thSAD=h1,plane=0,lsb=true)
d.dfttest(sigma=h2,tbsize=1,lsb_in=true,lsb=true)""")
...

edit: actually this fails miserably in an encoding test. Memory issues I think

[Gavino]

...
d=stackvertical(last,last).scriptclip("""
...
d=crop(0,0,0,-478).MDeGrain3(asuper, b1v, f1v, b2v, f2v, b3v, f3v, thSAD=h1,plane=0,lsb=true)
d.dfttest(sigma=h2,tbsize=1,lsb_in=true,lsb=true)""")
...
That script won't work as ScriptClip must return a clip with the same dimensions as its input (see post #2), and the crop is changing the dimensions.

Also, the use of 'd' both inside and outside the ScriptClip is confusing. Is that intentional?

[Dogway]

No it wasn't. I can fix it.
Im aware that Im cropping input by half, but mdegrain3 is doubling the output height again. Let me test

Tested but it crashes again, maybe its too much for scriptclip?

[Gavino]

mdegrain3 is doubling the output height again.
Are you sure? I wasn't aware that mdegrain3 could change the height.
What does asuper refer to? (not shown in your post)

[Dogway]

I wasn't aware that mdegrain3 could change the height.

it's the mod version of mvtools2 from the Dither pack.

I post the relevant code, I also tested without mrecalculate which is not sctrictly necessary, but note this code works when previewing single frames in avspmod.

blksize=8
overlap=4
asuper=MSuper(pel=2,sharp=2,hpad=blksize, vpad=blksize,chroma=false)
rsuper=MSuper(pel=2,sharp=2,hpad=blksize, vpad=blksize,levels=1,chroma=false)
b3v = MAnalyse(asuper, isb=true, delta=3, blksize=blksize, overlap=overlap,chroma=false)
b2v = MAnalyse(asuper, isb=true, delta=2, blksize=blksize, overlap=overlap,chroma=false)
b1v = MAnalyse(asuper, isb=true, delta=1, blksize=blksize, overlap=overlap,chroma=false)
f1v = MAnalyse(asuper, delta=1, blksize=blksize, overlap=overlap,chroma=false)
f2v = MAnalyse(asuper, delta=2, blksize=blksize, overlap=overlap,chroma=false)
f3v = MAnalyse(asuper, delta=3, blksize=blksize, overlap=overlap,chroma=false)

v=TemporalSoften(3,255,255,0,2)
d=stackvertical(last,last).scriptclip("""
h=YDifferenceFromPrevious(v)
h1 = 300/(h/0.5+0.5)+250
h2 = h+1
b3v = MRecalculate(rsuper, b3v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
b2v = MRecalculate(rsuper, b2v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
b1v = MRecalculate(rsuper, b1v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
f1v = MRecalculate(rsuper, f1v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
f2v = MRecalculate(rsuper, f2v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
f3v = MRecalculate(rsuper, f3v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
t=crop(0,0,0,-478).MDeGrain3(asuper, b1v, f1v, b2v, f2v, b3v, f3v, thSAD=round(h1),plane=0,lsb=true)
t.dfttest(sigma=h2,tbsize=1,lsb_in=true,lsb=true)""")

[cretindesalpes]

You could also do almost all the processing out of ScriptClip() with a limited set of values (say 3 per filter: low, medium, high) and use the runtime functions only to select/interpolate the results (or maybe with just ConditionalFilter()).

[Dogway]

Yes, I saw the examples in the help... if prefer this gradual way, but if it can't be I will try that out, using a thresholding for low/medium/high (not very enthusiastic but...). Do you think there is a reason using Dither's mdegrain3+dfttest is too much for scriptclip?

edit: Also this is a dumb question but as you see I switched the "dither_*" lines with stackvertical and crop, Im not sure at all if this has any impact on the performance because maybe it's only processed at the parsing phase(?)

[Gavino]

Do you think there is a reason using Dither's mdegrain3+dfttest is too much for scriptclip?
Do you get problems immediately or only after rendering a lot of frames?
cretindesalpes may have more inside knowledge.

Im not sure at all if this has any impact on the performance because maybe it's only processed at the parsing phase(?)
At the parsing stage, everything inside ScriptClip is just a string, so basically no processing takes place.
At run-time, this string is parsed afresh for every frame served, instantiating filters to create an inner filter chain from which a frame is fetched.

[Dogway]

maybe after rendering 20 frames. Ram usage starts to go up and up, and crashes around 2.2Gb. If I remove Mrecalculate it crashes faster with ram steady at around 1.3Gb
Vdub log:


Crash reason: Unhandled Microsoft C++ Exception

Thread call stack:
7c812afb: kernel32!RaiseException [7c800000+12aa9+52]
7c812afb: kernel32!RaiseException [7c800000+12aa9+52]
7c921028: ntdll!wcsncpy [7c910000+1057f+aa9]
7c921086: ntdll!wcsncpy [7c910000+1057f+b07]
7c9201db: ntdll!RtlAllocateHeap [7c910000+100c4+117]
7c812afb: kernel32!RaiseException [7c800000+12aa9+52]
7c359aed: MSVCR71!_CxxThrowException [7c340000+19ab9+34]
10001972: AviSynth!00001972
100038e2: AviSynth!000038e2
10009b33: AviSynth!avs_release_value [10000000+75d0+2563]
014a1f45: MT!00001f45
7c80bb64: kernel32!lstrcmpi [7c800000+bb41+23]
01605541: TNLMeans!00005541
7c920041: ntdll!RtlFreeHeap [7c910000+ff2d+114]
7c92005d: ntdll!RtlFreeHeap [7c910000+ff2d+130]
10005e4f: AviSynth!00005e4f
7c9200b8: ntdll!RtlFreeHeap [7c910000+ff2d+18b]
7c920041: ntdll!RtlFreeHeap [7c910000+ff2d+114]
7c92005d: ntdll!RtlFreeHeap [7c910000+ff2d+130]
01608cfb: TNLMeans!_AvisynthPluginInit2@4 [01600000+8bb0+14b]
10009037: AviSynth!avs_release_value [10000000+75d0+1a67]
10009443: AviSynth!avs_release_value [10000000+75d0+1e73]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
014a1e53: MT!00001e53
7c91da1a: ntdll!NtRegisterThreadTerminatePort [7c910000+da0e+c]
7c80b729: kernel32!GetModuleFileNameA [7c800000+b56f+1ba]

Tested without the setmtmode(2,2) line, just left setmtmode(5,2) at the very first and it worked...

[cretindesalpes]

I just did a quick test and observed that MRecalculate inside ScriptClip makes the memory usage increase linearly until it crashes (-> 2 GB memory barrier); the destructor must have a memory leak or something. Dfttest too, but much slower. MDegrain3 looks OK.

Maybe other functions have the same issue.

[Dogway]

Yes, another test without setmtmode(2,2) but with mrecalculate also crashed, just later than using multithreading. Maybe my test with no multithreading no mrecalculate worked because I only tested a small section of 400 frames, not long enough to let scriptclip crash.