Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
12th January 2020, 10:18 | #41 | Link |
Suspended for forum rule violations
Join Date: Mar 2018
Location: Germany
Posts: 216
|
Oh man - this looks far more complex than I had thought.
The less I'm able to contribute. A thing I know about is that the collaboration between VD and AviSynth is problematic sometimes when the script is very complex. For example using MSU script for interpolating ... when doing fastward/-backward in VD it crashes regulary. Have to jump to timeline. About the code in general. If I'm about to encircle strange effects I usually strip the code more and more to see what's the cause. So maybe disable multithread first. If it still has unexpected effects (mem, time) maybe strip (comment) lines of code one by one? Especially mem usage should be really low for the alg ... I think. The 256 masks can be handled not all at once but one after the other, and for each iteration there are only 3 to 4 additional masks. (==> There must be some strange thing happening). Sorry I cannot tribute more than silly thoughts. |
12th January 2020, 14:02 | #42 | Link | |||
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
Quote:
Quote:
I think problem in Avisynth, is hanging on to mem somewhere.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
|||
12th January 2020, 14:24 | #43 | Link | |
Suspended for forum rule violations
Join Date: Mar 2018
Location: Germany
Posts: 216
|
Quote:
So maybe the mem problem of AviSynth is the cause for the "run"time? (Getting/ Letting mem usually takes long) And then ... maybe worthwile to dig further to get a potential bug Avisynth? |
|
12th January 2020, 18:34 | #44 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Removed script functions [ GreyRamp(), MyFunc() ], embed code directly into main level script.
Not much difference. (craps out about 1440 frames) Code:
/* Req: Avs+ SelectPixelByMask RT_Stats. requires OUT\ directory in current dir SelectPixelByMask(clip Data, clip Mask) Data 256 frames, Y8 or YV24, or RGB24, or RGB32. Mask Y8 */ # CONTRAST StartCont = 0.5 # Mask 0 EndCont = 2.0 # Mask 255 BaseName = ".\OUT\MyPic_" Matrix = "Rec601" myName = "SelPix3: " DEBUG = True ################### AviSource("D:\SELECTPIXELBYMASK\Parade.avi") trim(60,0) # trim off static start BilinearResize(256,256) # Try not to swamp memory too much ConvertToYV24() # Sat will not work on Y8, And not RGB either (Tweak YUV only) ORG=Last FC=ORG.Framecount Msg=MessageClip("Done") # Create now when mem avilable ################### black = BlankClip(Length=1,color=$000000, width=1, height=256,Pixel_type="Y8") white = BlankClip(Length=1,color=$ffffff, width=1, height=256,Pixel_type="Y8") Mask=StackHorizontal(black,white).BilinearResize(512, 256).Crop(128,0,-128,-0).Loop(FC) #Mask=Mask.BilinearResize(Width,Height) # Rough ramp, single frame, same size as clip, just for testing #return Mask.info BaseName=RT_GetFullPathName(BaseName) StartTime=RT_TimerHP For(n=0,FC-1) { current_frame=n Data=ORG.Trim(0,-1).Loop(256) # 256 identical frames Data=Data.Animate (0, 255, "Tweak", \ [* START Args *] \ 0.0, [* Hue default 0.0 *] \ 1.0, [* Sat default 1.0 *] \ 0.0, [* Bright default 0.0 *] \ StartCont, [* Cont default 1.0 *] \ False, [* Coring Default True *] \ [* END Args *] \ 0.0, [* Hue default 0.0 *] \ 1.0, [* Sat default 1.0 *] \ 0.0, [* Bright default 0.0 *] \ EndCont, [* Cont default 1.0 *] \ False [* Coring Default True *] \ ) Out=Data.SelectPixelByMask(Mask).Loop(FC) Out=Out.ConvertToRGB32(matrix=Matrix).ImageWriter(file=BaseName, start=n,end=n, type="png") Out.RT_AverageLuma(w=1,h=1) # Force write image file if(DEBUG) { Tim = RT_TimerHP RT_DebugF("%d] %f%% %.2fFPS",n,(n+1)*100.0/FC,1/(Tim-StartTime),name=myName) StartTime=Tim } Out=0 Data=0 } Return Msg
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 12th January 2020 at 18:49. |
12th January 2020, 20:18 | #45 | Link |
Suspended for forum rule violations
Join Date: Mar 2018
Location: Germany
Posts: 216
|
Please excuse: I made a severe typo in my post before.
What I meant was: If traced down to the line of AviSynth code which causes the mem problem, and it should not grap/ keep as much, then maybe it's an error of AviSynth? (Which would make this investigation even more valuable) |
12th January 2020, 20:54 | #46 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
|
12th January 2020, 21:18 | #47 | Link |
Suspended for forum rule violations
Join Date: Mar 2018
Location: Germany
Posts: 216
|
No, of course.
That would be an issue for the AviSynth development team then. But, if I get you right, it is not as far, that it is known if it's an error in your script or in a specific AviSynth statement? EDIT: A third option (that this alg can't be done without huge ressources of space and time) cannot be true. That's what I do know for sure. Last edited by nji; 12th January 2020 at 22:28. Reason: see EDIT |
13th January 2020, 00:36 | #48 | Link |
Registered User
Join Date: Mar 2018
Posts: 447
|
This piqued my interest. Unfortunately my code-fu is not very strong when it comes to Avisynth, but I was able to implement this in a VapourSynth script.
I haven't done any extensive testing but it seems to run ok, memory consumption is about 2,4GB when running with a 1080x576 source video. The 8-bit mask in this case had all 256 values in it, the script will skip calculating the effect for the missing mask values and will be more efficient when there are less mask values. Speed was about 1fps with 256 mask values and about 5-6 fps with 16 mask values, using a horizontal blur effect as the test function (I had another heavy task running in the background so these measurements are not very accurate). If you're interested I can post the script (to VapourSynth section of this forum). |
16th January 2020, 23:00 | #50 | Link | ||
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
Quote:
EDIT: I do create a new frame on filter construction, but should be destroyed when destructing [when setting the relevant clips to 0 in script]. EDIT: Zorr, Would your code work with all filters, eg could you for instance using a gray ramp, blur max 1.58 at frame left, null in middle and sharpen max 1.0 at frame right. (or eg temporal smooth or sharpen, or MC filtering, or frame replacement filtering, or is it just a LUT). EDIT: nji I think that you mentioned OpenGL somewhere. IIRC, that stuff has some kind of pixel filtering operations, so it is likely exactly what you want (dont know much at all about OpenGL, and not relally feeling inclined to duplicate OpenGL as some kind of Avisynth filter).
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 16th January 2020 at 23:23. |
||
17th January 2020, 00:12 | #51 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
I posted my version here.
Quote:
|
|
17th January 2020, 01:11 | #52 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Thanks Zorr, I cant spend any time examining your code right now, busy trying to recover some accidentally deleted files.
Got thousands (10,000) of text files to look through, busy making script to show first x number of lines of text files on video frame. EDIT: I did consider trying out Expr() but shied off as I've never used it before. [I presume it will take Masktools expressions without change].
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 17th January 2020 at 01:24. |
17th January 2020, 10:21 | #53 | Link | |
Suspended for forum rule violations
Join Date: Mar 2018
Location: Germany
Posts: 216
|
Quote:
Maybe it exists. Surely I will be able to implement even in plain C++ etc. But the point is I'm needing it on app level of movie manipulation. And - as I tried to point out in this thread - it seems quite interesting to see about the difference of exact pixel filtering vs. mishmash that is proposed all the time. The only thing I knew about AviSynth so far that was it's just that inter-filter manipulation which makes its strength. Sorry this thread convinced me of the contrary. Thanks anyway! |
|
17th January 2020, 14:54 | #54 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Avisynth was never meant to be abused in the way that I've been trying to pervert it, trouble is I dont know how to abuse it well enough [nor know if it is possible to to do it without error].
Anyways, I've did repeat of script in post #44, but with the trim line chaged to "trim(60,-800) # trim off static start, 800 frames only". [try to ensure script completes] Also force freed destframe in dll destructor(not really necessary but just to be sure) , enabled debug stuff in dll and capped output to file in Debugview. Code:
00000087 47.04938507 SelectPixelByMask: Calling AddFunction SelectPixelByMask # Add dll function to avisynth (pre script execution) 00000088 47.04946518 SelectPixelByMask: returned from AddFunction 00000198 0.14413995 SelectPixelByMask: Creator IN 00000199 0.14419755 SelectPixelByMask: Creator Calling Constructor # Creating filter 00000200 0.14424321 SelectPixelByMask: Constructor IN/OUT 00000201 0.14428234 SelectPixelByMask: Creator returned from Constructor, OUT 00000202 0.14447002 SelectPixelByMask: GetFrame IN 00000203 0.44192630 SelectPixelByMask: GetFrame OUT 00000204 0.52262574 SelPix3: 0] 0.125000% 2.63FPS # in script 00000205 0.52272862 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT # destframe force freed, filter destroyed (called when out=0 in script) 00000206 0.52439749 SelectPixelByMask: Creator IN 00000207 0.52443445 SelectPixelByMask: Creator Calling Constructor 00000208 0.52446741 SelectPixelByMask: Constructor IN/OUT 00000209 0.52449894 SelectPixelByMask: Creator returned from Constructor, OUT 00000210 0.52462900 SelectPixelByMask: GetFrame IN 00000211 0.80134016 SelectPixelByMask: GetFrame OUT 00000212 0.84081995 SelPix3: 1] 0.250000% 3.14FPS 00000213 0.84093916 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT 00000214 0.84261560 SelectPixelByMask: Creator IN 00000215 0.84266162 SelectPixelByMask: Creator Calling Constructor 00000216 0.84270108 SelectPixelByMask: Constructor IN/OUT 00000217 0.84273803 SelectPixelByMask: Creator returned from Constructor, OUT 00000218 0.84287393 SelectPixelByMask: GetFrame IN 00000219 1.11795390 SelectPixelByMask: GetFrame OUT 00000220 1.15778732 SelPix3: 2] 0.375000% 3.15FPS # ... 00006558 356.57012939 SelPix3: 795] 99.500000% 2.63FPS 00006559 356.57028198 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT 00006560 356.57205200 SelectPixelByMask: Creator IN 00006561 356.57211304 SelectPixelByMask: Creator Calling Constructor 00006562 356.57214355 SelectPixelByMask: Constructor IN/OUT 00006563 356.57217407 SelectPixelByMask: Creator returned from Constructor, OUT 00006564 356.57232666 SelectPixelByMask: GetFrame IN 00006565 356.99841309 SelectPixelByMask: GetFrame OUT 00006566 357.03823853 SelPix3: 796] 99.625000% 2.14FPS 00006567 357.03839111 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT 00006568 357.04010010 SelectPixelByMask: Creator IN 00006569 357.04016113 SelectPixelByMask: Creator Calling Constructor 00006570 357.04019165 SelectPixelByMask: Constructor IN/OUT 00006571 357.04022217 SelectPixelByMask: Creator returned from Constructor, OUT 00006572 357.04037476 SelectPixelByMask: GetFrame IN 00006573 357.52993774 SelectPixelByMask: GetFrame OUT 00006574 357.57003784 SelPix3: 797] 99.750000% 1.88FPS 00006575 357.57019043 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT 00006576 357.57192993 SelectPixelByMask: Creator IN 00006577 357.57199097 SelectPixelByMask: Creator Calling Constructor 00006578 357.57202148 SelectPixelByMask: Constructor IN/OUT 00006579 357.57205200 SelectPixelByMask: Creator returned from Constructor, OUT 00006580 357.57220459 SelectPixelByMask: GetFrame IN 00006581 357.93356323 SelectPixelByMask: GetFrame OUT 00006582 357.97360229 SelPix3: 798] 99.875000% 2.48FPS 00006583 357.97375488 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT 00006584 359.09152222 SelectPixelByMask: Creator IN 00006585 359.09158325 SelectPixelByMask: Creator Calling Constructor 00006586 359.09161377 SelectPixelByMask: Constructor IN/OUT 00006587 359.09164429 SelectPixelByMask: Creator returned from Constructor, OUT 00006588 359.09185791 SelectPixelByMask: GetFrame IN 00006589 359.38601685 SelectPixelByMask: GetFrame OUT 00006590 359.42672729 SelPix3: 799] 100.000000% 0.69FPS 00006591 359.42684937 SelectPixelByMask: Destructor IN, setting destframe=NULL Destructor OUT 2) Script shows "Done" messageclip, finished. Pause then close VDub (window closes, remains in memory.) 3) Remains in memory, maybe 3 stages of avisynth freeing memory 4) End of 3rd stage of freeing memory, Vdub process gone. Little bit extra mem freed. Note Up time and Commit(GB) [EDIT: Pagefile expanding, I saw it in earlier run at 35GB] Looks like memory cannot be freed until brought from pagefile into physical memory, I guess thats to do with linked lists.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 17th January 2020 at 15:15. |
17th January 2020, 17:47 | #55 | Link |
Suspended for forum rule violations
Join Date: Mar 2018
Location: Germany
Posts: 216
|
... thanks again! :-)
Well - as told above - I'm completely new to AviSynth and by that am not able to assess if the task of gradient driven filter application is only possible by "abusing" AviSynth. However if you look at the Vaporsynth script that was provided by zorr: It's very small and neat. Again ... if you're sure the error is by AviSynth (and not by your script) maybe strip it to the minimum and post it to AviSynth development subforum? |
17th January 2020, 23:29 | #56 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
I tried to do a pretty much direct and simplified conversion of my VapourSynth version. It's using your GreyRamp() and limited 256x256 resolution and only processing the luma channel... yet it seems to run out of memory or stack space. I can get it to process 20-30 mask values but anything more and it craps out in multitude of ways, yet the memory usage is actually very low. The version below is processing a modest number of mask values, you can try more by changing the mask_value of the applyFunc call or the recursion test value on the last line of applyFunc. Some of these issues may be because I only have an ancient x86 version of Avisynth installed. The applyFunc should be run inside ScriptClip in order to use animated masks but that made it fail as well. Code:
src = AVISource("source.avi") src = ConvertToYV16(src) src = Crop(src, 0,0,256,256) mask = GreyRamp() mask = ConvertToY8(mask) final = BlankClip(src) result = applyFunc(mask, src, final, 128) return result Function applyFunc(mask, src, final, int mask_value) { m = mask.mt_lut("x "+String(mask_value)+" = 255 0 ?") gradient_clip = blurx(src, mask_value/255.0) final = mt_lutxyz(final, gradient_clip, m, "z 255 = y x ?") return mask_value < 140 ? applyFunc(mask, src, final, mask_value+1) : final } Function blurx(clip, float ratio) { return FastBlur(clip, ratio*5.0) } |
|
|
|