Apply filters with local gradient

[zorr]

I posted my version here (https://forum.doom9.org/showthread.php?t=177365).

Zorr,
Would your code work with all filters, eg could you for instance using a gray ramp, blur max 1.58 at frame left, null in middle and sharpen max 1.0 at frame right.
(or eg temporal smooth or sharpen, or MC filtering, or frame replacement filtering, or is it just a LUT).

Yes, it applies the filter for the whole frame and then extracts the parts with correct mask value and composites all the variations with different mask values to a final result. The applied function could do anything and be as complex as needed.

[StainlessS]

Thanks Zorr, I cant spend any time examining your code right now, busy trying to recover some accidentally deleted files.
Got thousands (10,000) of text files to look through, busy making script to show first x number of lines of text files on video frame.

EDIT: I did consider trying out Expr() but shied off as I've never used it before. [I presume it will take Masktools expressions without change].

[nji]

nji
I think that you mentioned OpenGL somewhere. IIRC, that stuff has some kind of pixel filtering operations, so it is likely exactly what you want
(dont know much at all about OpenGL, and not relally feeling inclined to duplicate OpenGL as some kind of Avisynth filter).

I've never been in that "OpenGL corner" to know about its pixel filtering ops.
Maybe it exists.
Surely I will be able to implement even in plain C++ etc.

But the point is I'm needing it on app level of movie manipulation.
And - as I tried to point out in this thread - it seems quite interesting
to see about the difference of exact pixel filtering vs. mishmash that
is proposed all the time.

The only thing I knew about AviSynth so far that was it's just that
inter-filter manipulation which makes its strength.
Sorry this thread convinced me of the contrary.
Thanks anyway! :)

[StainlessS]

Avisynth was never meant to be abused in the way that I've been trying to pervert it, trouble is I dont know how to abuse it well enough [nor know if it is possible to to do it without error].

Anyways, I've did repeat of script in post #44, but with the trim line chaged to "trim(60,-800) # trim off static start, 800 frames only".
[try to ensure script completes]
Also force freed destframe in dll destructor(not really necessary but just to be sure) , enabled debug stuff in dll and capped output to file in Debugview.

00000087 47.04938507 SelectPixelByMask: Calling AddFunction SelectPixelByMask # Add dll function to avisynth (pre script execution)
00000088 47.04946518 SelectPixelByMask: returned from AddFunction

00000198 0.14413995 SelectPixelByMask: Creator IN
00000199 0.14419755 SelectPixelByMask: Creator Calling Constructor # Creating filter
00000200 0.14424321 SelectPixelByMask: Constructor IN/OUT
00000201 0.14428234 SelectPixelByMask: Creator returned from Constructor, OUT
00000202 0.14447002 SelectPixelByMask: GetFrame IN
00000203 0.44192630 SelectPixelByMask: GetFrame OUT
00000204 0.52262574 SelPix3: 0] 0.125000% 2.63FPS # in script
00000205 0.52272862 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT # destframe force freed, filter destroyed (called when out=0 in script)
00000206 0.52439749 SelectPixelByMask: Creator IN
00000207 0.52443445 SelectPixelByMask: Creator Calling Constructor
00000208 0.52446741 SelectPixelByMask: Constructor IN/OUT
00000209 0.52449894 SelectPixelByMask: Creator returned from Constructor, OUT
00000210 0.52462900 SelectPixelByMask: GetFrame IN
00000211 0.80134016 SelectPixelByMask: GetFrame OUT
00000212 0.84081995 SelPix3: 1] 0.250000% 3.14FPS
00000213 0.84093916 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT
00000214 0.84261560 SelectPixelByMask: Creator IN
00000215 0.84266162 SelectPixelByMask: Creator Calling Constructor
00000216 0.84270108 SelectPixelByMask: Constructor IN/OUT
00000217 0.84273803 SelectPixelByMask: Creator returned from Constructor, OUT
00000218 0.84287393 SelectPixelByMask: GetFrame IN
00000219 1.11795390 SelectPixelByMask: GetFrame OUT
00000220 1.15778732 SelPix3: 2] 0.375000% 3.15FPS

# ...

00006558 356.57012939 SelPix3: 795] 99.500000% 2.63FPS
00006559 356.57028198 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT
00006560 356.57205200 SelectPixelByMask: Creator IN
00006561 356.57211304 SelectPixelByMask: Creator Calling Constructor
00006562 356.57214355 SelectPixelByMask: Constructor IN/OUT
00006563 356.57217407 SelectPixelByMask: Creator returned from Constructor, OUT
00006564 356.57232666 SelectPixelByMask: GetFrame IN
00006565 356.99841309 SelectPixelByMask: GetFrame OUT
00006566 357.03823853 SelPix3: 796] 99.625000% 2.14FPS
00006567 357.03839111 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT
00006568 357.04010010 SelectPixelByMask: Creator IN
00006569 357.04016113 SelectPixelByMask: Creator Calling Constructor
00006570 357.04019165 SelectPixelByMask: Constructor IN/OUT
00006571 357.04022217 SelectPixelByMask: Creator returned from Constructor, OUT
00006572 357.04037476 SelectPixelByMask: GetFrame IN
00006573 357.52993774 SelectPixelByMask: GetFrame OUT
00006574 357.57003784 SelPix3: 797] 99.750000% 1.88FPS
00006575 357.57019043 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT
00006576 357.57192993 SelectPixelByMask: Creator IN
00006577 357.57199097 SelectPixelByMask: Creator Calling Constructor
00006578 357.57202148 SelectPixelByMask: Constructor IN/OUT
00006579 357.57205200 SelectPixelByMask: Creator returned from Constructor, OUT
00006580 357.57220459 SelectPixelByMask: GetFrame IN
00006581 357.93356323 SelectPixelByMask: GetFrame OUT
00006582 357.97360229 SelPix3: 798] 99.875000% 2.48FPS
00006583 357.97375488 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT
00006584 359.09152222 SelectPixelByMask: Creator IN
00006585 359.09158325 SelectPixelByMask: Creator Calling Constructor
00006586 359.09161377 SelectPixelByMask: Constructor IN/OUT
00006587 359.09164429 SelectPixelByMask: Creator returned from Constructor, OUT
00006588 359.09185791 SelectPixelByMask: GetFrame IN
00006589 359.38601685 SelectPixelByMask: GetFrame OUT
00006590 359.42672729 SelPix3: 799] 100.000000% 0.69FPS
00006591 359.42684937 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT


1) Rising mem usage
https://i.postimg.cc/ykVCZkVf/img-1.jpg (https://postimg.cc/ykVCZkVf)

2) Script shows "Done" messageclip, finished. Pause then close VDub (window closes, remains in memory.)
https://i.postimg.cc/sBwvmL85/img-2.jpg (https://postimg.cc/sBwvmL85)

3) Remains in memory, maybe 3 stages of avisynth freeing memory
https://i.postimg.cc/qtM7NLRq/img-3.jpg (https://postimg.cc/qtM7NLRq)

4) End of 3rd stage of freeing memory, Vdub process gone. Little bit extra mem freed.
https://i.postimg.cc/SnmSPrcd/img-4.jpg (https://postimg.cc/SnmSPrcd)

Note Up time and Commit(GB) [EDIT: Pagefile expanding, I saw it in earlier run at 35GB]

Looks like memory cannot be freed until brought from pagefile into physical memory, I guess thats to do with linked lists.

[nji]

... thanks again! :-)

Well - as told above - I'm completely new to AviSynth and by that
am not able to assess if the task of gradient driven filter application
is only possible by "abusing" AviSynth.

However if you look at the Vaporsynth script that was provided by zorr:
It's very small and neat.

Again ... if you're sure the error is by AviSynth (and not by your script)
maybe strip it to the minimum and post it to AviSynth development subforum?

[zorr]

EDIT: I did consider trying out Expr() but shied off as I've never used it before. [I presume it will take Masktools expressions without change].

Yes Expr is a lot like MaskTools.

I tried to do a pretty much direct and simplified conversion of my VapourSynth version. It's using your GreyRamp() and limited 256x256 resolution and only processing the luma channel... yet it seems to run out of memory or stack space. I can get it to process 20-30 mask values but anything more and it craps out in multitude of ways, yet the memory usage is actually very low.

The version below is processing a modest number of mask values, you can try more by changing the mask_value of the applyFunc call or the recursion test value on the last line of applyFunc. Some of these issues may be because I only have an ancient x86 version of Avisynth installed. The applyFunc should be run inside ScriptClip in order to use animated masks but that made it fail as well.

src = AVISource("source.avi")
src = ConvertToYV16(src)
src = Crop(src, 0,0,256,256)

mask = GreyRamp()
mask = ConvertToY8(mask)

final = BlankClip(src)
result = applyFunc(mask, src, final, 128)

return result

Function applyFunc(mask, src, final, int mask_value) {
m = mask.mt_lut("x "+String(mask_value)+" = 255 0 ?")
gradient_clip = blurx(src, mask_value/255.0)
final = mt_lutxyz(final, gradient_clip, m, "z 255 = y x ?")
return mask_value < 140 ? applyFunc(mask, src, final, mask_value+1) : final
}

Function blurx(clip, float ratio) {
return FastBlur(clip, ratio*5.0)
}