Alternative to ApplyEvery plugin

[Andouille]

Hi people,

Is there an alternative to the ApplyEvery plugin ?
It is very slow to initialize and eats a ton of ram.

Thank you.

[creaothceann]

Try this:

function ApplyEvery(clip c, int Period, string Thunk) {
c.KillAudio
last + last.BlankClip(Period - (FrameCount % Period)).KillAudio
ApplyEvery_Main(Period, Thunk).Trim(0, -c.FrameCount)
(c.HasAudio) ? last.AudioDub(c) : last
}


function ApplyEvery_Main(clip c, int Period, string Thunk) {
c.Trim(0, -Period)
Eval(Thunk) # can't connect this with the previous line for some reason
(c.FrameCount > Period) ? last + c.Trim(Period, 0).ApplyEvery_Main(Period, Thunk) : last
}
It even doesn't leave black frames at the end if FrameCount isn't a multiple of Period.

[Gavino]

Depending on what you are trying to do, you may be able to replace ApplyEvery() with JDL_SimpleApplyEvery() (from jdl-range.avsi (http://avisynth.org/stickboy/jdl-range.avsi)) or a combination of SelectEvery(), DeleteEvery() and InterleaveEvery() (the last two being from the same plugin as ApplyEvery).

Note - creaothceann's version is just as slow as (or even slower than) ApplyEvery(), as it recursively iterates over all frames of the clip (like the original JDL_ApplyEvery() which ApplyEvery() superceded).

[Andouille]

Thanks guys, I will test all your suggestions and report back.

( I'm trying to test compressibility of grain's pattern )

[IanB]

As always script the decision not the function! Words to live by.
Period=8
BlankClip()
Thunk=ShowFrameNumber() # Something to do
ConditionalFilter (Thunk, Last, "Current_Frame % Period", "==", "Period-1")

Both the ApplyEvery plugin and creaothceann's script create FrameCount()/Period instances of Thunk, Trim and Splice. Not a very smart thing to do, especially if Thunk is non-trivial.

[Gavino]

By contrast, JDL_SimpleApplyEvery() creates only a single instance of thunk.

IanB's example could be written as
JDL_SimpleApplyEvery("ShowFrameNumber", 8, 7)

[Andouille]

Sorry guys but nothing does work.
Creaothceann script, applyevery plugin, script in the jdl package do the same thing (obscene ram usage and looong initialization)
IanB script does not do what I want. The filtering has to be recalled every n frames.
I do not want to interleave the filtered clip with the source clip.

[creaothceann]

Would something like this work?

global c = Version.Trim(0, -100) # DSS2 / DirectShowSource / AVISource / ...
global Index = 0
global Period = 3
global Thunk = "Info"


FramesToAdd = Period - (c.FrameCount % Period)
c = c + c.BlankClip(FramesToAdd)
ApplyEvery
c
Trim(0, FrameCount - FramesToAdd - 1)



function ApplyEvery {
c.Trim(Index, Period)
# Apply(Thunk)
global Index = Index + Period
global c = last + c.Trim(Index, 0)
(Index < FrameCount) ? ApplyEvery : nop
}
Unfortunately the Apply call doesn't want to cooperate...

[Andouille]

Nope, that doesn't work :(

[IanB]

@Andouille,script does not do what I want. The filtering has to be recalled every n frames.
I do not want to interleave the filtered clip with the source clip.
Then please specify the pattern you do want.

My example applied ShowFrameNumber to every 8th frame.0 1 2 3 4 5 6 7 8 9 ...
x x x x x x x S x x ...
Say for a 10 frame clip, with Period=3 write a script using trim and + that does what you want and we will show you how to do it for any clip and any period.

[Gavino]

function ApplyEvery {
c.Trim(Index, Period)
# Apply(Thunk)
global Index = Index + Period
global c = last + c.Trim(Index, 0)
(Index < FrameCount) ? ApplyEvery : nop
}
This code has a number of errors (apart from the ugly use of global variables). But even if it was correct, the underlying method would still not solve the basic performance problem, which is that an instance of Thunk is created for each group of processed frames (Framecount/Period instances).

@Andouille: what was in your original call to ApplyEvery()?
From that we could probably work out a suitable replacement.

[Andouille]

Something like that :
blankclip()

filter(7)

function filter(clip c, int n)
{
c
trim(0,n-1).addgrain(seed=0) +\
trim(n,2*n-1).addgrain(seed=0) +\
trim(2*n,3*n-1).addgrain(seed=0) +\
trim(3*n,4*n-1).addgrain(seed=0) +\
trim(4*n,5*n-1).addgrain(seed=0)
#............and so on............
}
#with the applyevery plugin :
#applyevery(7,"addgrain(seed=0)")

[Gavino]

applyevery(7,"addgrain(seed=0)")
Right, I see the problem here is, because of the random nature of the function, you really want to create many different instances of addgrain().

Try this:
ScriptClip("""
res = (current_frame%7 == 0 ? addgrain(seed=0) : res)
return res
""")
This will still create as many instances of addgrain, but it will do it at runtime (so no initialisation slowdown) and more importantly, should free the previous instance each time it creates a new one.

[Andouille]

Seems to does not work either :(
Sorry Gavino...

[Gavino]

What happens? Do you get an error message?
You will have to render the clip linearly from start to finish.

[Andouille]

No error. It seems to only apply addgrain "normaly".

[Gavino]

Try changing it to this:
ScriptClip("""
res = (current_frame%7 == 0 ? Trim(current_frame, 0).addgrain(seed=0) : res)
return res
""")
That should in principle replicate the same pattern of addgrain calls as your original ApplyEvery code.

Does the perfomance improve compared to ApplyEvery?

[Andouille]

The performance improved.... certainly because it still does not work :(
Thanks for your efforts though

[Gavino]

Just to be sure - did the original (ApplyEvery) code 'work' (even if slow)?

[Andouille]

Sure it does work.
The biggest problem is the ram usage.

[IanB]

Ouch! I see, you want every group of 7 frames to hit AddGrain(seed=0) with frames 0 to 6 then reset and continue for the next group of 7.

Period=7

BlankClip(Width=320, Height=240)
StackHorizontal(ShowFrameNumber(True), Last)

Slave=ScriptClip("Return Thunk")

ScriptClip("""
Trim(Current_Frame, 0)
ShowFrameNumber(True) # AddGrain(seed=0)
Trim(0, Length=Current_Frame) + Last
Global Thunk=Last
Return Thunk
""")

ConditionalFilter (Last, Slave, "Current_Frame % Period", "==", "0")

[Gavino]

Yes, that's basically what I was trying to do with my second version, but I see now it wasn't adjusting the frame numbers correctly. Instead of
res = (current_frame%7 == 0 ? Trim(current_frame, 0).addgrain(seed=0) : res)
it should have been
res = (current_frame%7 == 0 ? Trim(current_frame, 0).addgrain(seed=0).SelectEvery(1, -current_frame) : res)

IanB's version is effectively doing the same thing, but (though longer) is slightly more efficient as it reduces run-time script parsing time by splitting up the code into two different ScriptClip instances, of which only one is used on each individual frame.

(Incidentally Ian, Thunk doesn't need to be global. ;))

[Andouille]

Well done IanB, it does work :) . Thank you.

I already made the cleaning suggested by Gavino and changed the trim "length" because I am still on Avisynth 2.5.

Now I'm trying to make it work in a function but ScriptClip and ConditionalFilter do not want my variables "Filter" and "Period".



function FilterEvery (clip c, string Filter, int Period)
{
c

Slave = ScriptClip("""Filter""")

ScriptClip("""
Trim( current_frame, 0)
Filter
Trim( 0, -current_frame) + Last
""")

ConditionalFilter (Last, Slave, "Current_Frame % Period", "==", "0")
}

[IanB]

The conditional filters run at the scope of the FilterEvery call, so the variables need to be at that scope. A hack around is to pass in the name of the variables to the function.function FilterEvery (clip c, string FilterName, string PeriodName) {
c

Slave=ScriptClip("Return Thunk")

ScriptClip("""
Trim( current_frame, 0)
Eval("""+FilterName+""")
Current_Frame==0 ? Last : Trim( 0, -current_frame) + Last
Thunk=Last
Return Thunk
""")

ConditionalFilter (Last, Slave, "Current_Frame % "+PeriodName, "==", "0")
}

Filter="ShowFrameNumber(True)"
Period=7

BlankClip(Width=320, Height=240)
StackHorizontal(ShowFrameNumber(True), Last)

FilterEvery("Filter", "Period")Note! Thunk is also at the callers scope, so if you want more than 1 call at the same scope you would need to pass in a unique variable name for it as well.

[Gavino]

A hack around is to pass in the name of the variables to the function.
A less hacky alternative is to use the run-time filters from GRunT, which were designed principally to solve this problem of 'variable binding'.
function FilterEvery (clip c, string Filter, int Period) {
c

Slave=GScriptClip("Return Thunk")

GScriptClip("""
Trim( current_frame, 0)
Eval(Filter)
Current_Frame==0 ? Last : Trim( 0, -current_frame) + Last
Thunk=Last
Return Thunk
""", args="Filter", local=false) #local=false to preserve Thunk

GConditionalFilter (Last, Slave, "Current_Frame % Period", "==", "0", args="Period", local=false)
}

BlankClip(Width=320, Height=240)
StackHorizontal(ShowFrameNumber(True), Last)

FilterEvery("ShowFrameNumber(True)", 7)

[IanB]

Gavino implementation using a single ScriptClip :-function FilterEvery (clip C, string FilterName, string PeriodName) {
C.ScriptClip("""
Trim(current_frame, 0)
res = current_frame%"""+PeriodName+""" == 0 ?
\ Eval("""+FilterName+""").SelectEvery(1, -current_frame)
\ :
\ res
Return res
""")
}Note that I had to move the Trim because Eval does not take a clip argument and the eval'd expression uses Last for it's input clip.

Hmmm need to add an Eval(Clip C, String S) alias to the language.

[Gavino]

Gavino implementation using a single ScriptClip
As before, this could also be changed to use GRunT, allowing the filter and period to be passed as values rather than indirectly as names.
function FilterEvery (clip C, string Filter, int Period) {
C.GScriptClip("""
Trim(current_frame, 0)
res = current_frame%Period == 0 ?
\ Eval(Filter).SelectEvery(1, -current_frame)
\ :
\ res
Return res
""", args="Filter, Period", local=false)
}

Hmmm need to add an Eval(Clip C, String S) alias to the language.
Yes, good idea.
The obvious implementation would be to set 'last' to C and then invoke eval(s), but it would have to ensure that 'last' is unchanged for something like
c = c.Eval(s)
so it probably needs to save and restore any existing 'last'.

[Andouille]

Argh... It is not working for me :(


With the IanB's last one :
blankclip()
filterevery("addgrain(1000,seed=0","7")

function FilterEvery (clip C, string FilterName, string PeriodName) {
C.ScriptClip("""
Trim(current_frame, 0)
res = current_frame%"""+PeriodName+""" == 0 ?
\ Eval("""+FilterName+""").SelectEvery(1, -current_frame)
\ :
\ res
Return res
""")
}

For all frames, I got this ScriptClip error :
Script error: expected a , or )
([ScriptClip], line 5, column 5)



With the Gavino's GRunT one :
blankclip()
FilterEvery("addgrain(1000,seed=0",7)

function FilterEvery (clip C, string Filter, int Period) {
C.GScriptClip("""
Trim(current_frame, 0)
res = current_frame%Period == 0 ?
\ Eval(Filter).SelectEvery(1, -current_frame)
\ :
\ res
Return res
""", args="Filter, Period", local=false)
}

For frames % Period, I got this ScriptClip error :
Script error: expected a , or )
((null), line 1, column 20)
([ScriptClip], line 6)

Other frames return this :
I don't know what "res" means
([ScriptClip], line 6)



Maybe I'm missing something obvious ?
My AviSynth version is 2.5.8.5. Can it be the culprit ?

[Gavino]

With the IanB's last one :
filterevery("addgrain(1000,seed=0","7")
Should be:
filterevery(""" "addgrain(1000,seed=0)" ""","7")

With the Gavino's GRunT one :
FilterEvery("addgrain(1000,seed=0",7)
Should be:
filterevery("addgrain(1000,seed=0)",7)

[Andouille]

I feel stupid :o

[Andouille]

Just found this to make the call to the function less "quoted" when using only AviSynth core :
function FilterEvery (clip C, string FilterName, int Period) {

FilterName = """""""+string(FilterName)+"""""""
Period=string(Period)

C.ScriptClip("""
Trim(current_frame, 0)
res = current_frame%"""+Period+""" == 0 ?
\ Eval("""+FilterName+""").SelectEvery(1, -current_frame)
\ :
\ res
Return res
""")
}

Will test speed tomorrow and see if it does work fine with heavy filtering (grain with masks) AND multithreading.

In all cases, thx guys.

[IanB]

Almost certainly guaranteed that this will not work with multithreading. :devil:

[Gavino]

FilterName = """""""+string(FilterName)+"""""""
Use of string() is redundant here as Filtername is already a string.
Slightly easier to read is to use chr(34) to produce a quote character:
FilterName = chr(34)+FilterName+chr(34)
However (just like your version), this still doesn't work if Filtername itself contains internal quotes, eg ShowFrameNumber(true, font="Courier").
So you need to add triple quotes instead of single quotes:
q = chr(34)
q3 = q+q+q
Filtername = q3+Filtername+q3
(Simpler just to use GRunT :))

Almost certainly guaranteed that this will not work with multithreading. :devil:
That's right, as the frames need to be rendered in the correct order.
I'm not sure that run-time filters like ScriptClip work with MT anyway.
Do you know, IanB?
In particular, how is access to variables (especially current_frame) controlled?

[Andouille]

Sorry for the laaaate reply...

So yes, multithreading does not work.
Speed is 1/4 lower but it wont never crash because the ram issue.