Animate() question

[maxxon]

I was using Animate() to smoothly move a clip box from one point of a video to another and noticed something which I can't explain.

Here's my code:
output_file("output.txt")
version()
Animate(last, 0, 10, "test", 0, 10)

function test(clip clip, float x)
{
output(string(x))
return clip
}

To run this requires my AviSynthLib, which implements the functions output_file() and output().

Anyway, my understanding of animate is shaky, but according to the documentation:

Animate is a meta-filter which evaluates its parameter filter with continuously varying arguments. At frame start_frame and earlier, the filter is evaluated with the arguments given by start_args. At frame end_frame and later, the filter is evaluated with the arguments given by end_args. In between, the arguments are linearly interpolated for a smooth transition.

Which implies to me that it somehow calls the filter function 3 or more times, once for a clip that is at and before start_frame, once for a clip at and after end_frame, and a bunch of times for each frame in the clip between start_frame and end_frame (depending on how many frames there are between start_frame and end_frame).

However, that is not what's happening. It appears to be called only twice. And according to the output.txt file, it is being called with the parameter 0 and then again with parameter 10. Where is the interpolation happening?

Obviously my understanding is flawed, so could someone tell me what is actually happening?

Thanks,


|\/|x

[maxxon]

BTW, when using this for a real filter, such as drawing a bounding box or creating a cropping box, this results in the correctly smooth transition of interpolating the values, so I just don't understand what's going on.


|\/|x

[Gavino]

Which implies to me that it somehow calls the filter function 3 or more times, once for a clip that is at and before start_frame, once for a clip at and after end_frame, and a bunch of times for each frame in the clip between start_frame and end_frame (depending on how many frames there are between start_frame and end_frame).
That's correct.
The 'start' and 'end' filters are invoked at load-time, while the filters for the intermediate values are invoked at run-time (during frame serving).

I would expect the output to be
0.0, 10.0, 1.0, 2.0, ..., 9.0
Are you actually rendering frames 1-9?

Perhaps there is a problem with your output() function at run-time?
Perhaps the file has not yet been closed when you look at it?

[maxxon]

That's correct.
The 'start' and 'end' filters are invoked at load-time, while the filters for the intermediate values are invoked at run-time (during frame serving).

Really? Interesting. So you are saying that if the frame isn't requested, the intermediate frames are not generated?
That's correct.
I would expect the output to be
0.0, 10.0, 1.0, 2.0, ..., 9.0
Are you actually rendering frames 1-9?

Well, the example is contrived, but the actual animation showed all the frames with the expected transforms.

Perhaps there is a problem with your output() function at run-time?

Dunno. Not sure what the problem could be.

Perhaps the file has not yet been closed when you look at it?
No, I can write to the file directly after the animation and will see the final file write.

Just for reference, here is what the full code would look like, somewhat simplified:

global output__junk = BlankClip(height=0, width=0, length=0)

output__junk.WriteFileStart("output.txt", """"BEGIN"""", append=true)
x = version().Animate(1, 10, "test", 0, 10)

output__junk.WriteFileStart("output.txt", """"END"""", append=true)
return x

function test(clip clip, int x)
{
output__junk.WriteFileStart("output.txt", "x", append=true)
return clip
}

This will output the following:

BEGIN
0
10
END
Now I must admit, I was never very comfortable with the WriteFile* commands, at least in how the documentation explains them, and is why I wrote the output() function in the first place.

While experimenting, I wrote this:

w = BlankClip(height=0, width=0, length=0)
w.WriteFile("output.txt", """"BEGIN"""", append=true)
x = version().WriteFile("output.txt", """ "current_frame: "+current_frame.string""", append=true)
w.WriteFile("output.txt", """"END"""", append=true)
return x

Which is an eye opener for me. Every time a particular frame is requested, that frame number is outputted. Testing this with AvsPMOD, it shows that if I go forward, back, or seek someplace randomly, it will output exactly what frame is being shown. *sigh* I'm really having trouble with this paradigm. If the code is just a representation of filters, it would appear that somehow the code stream is somewhat separate from the filter stream.

Oh, wait, WriteFile IS a filter, just a NULL one that outputs text to a file. And the code is a representation of PARALLEL filters, hence why BEGIN and END are not output at all since a frame from w was never requested. Changing w.WriteFile to w.WriteFileStart makes the BEGIN and END output first, but until a frame is request of x, current_frame is not displayed. WriteFileEnd is only executed when the .avs file is closed then. Yes, using AvsPMOD, by refreshing the preview, it outputs the string.

Sorry for all my ramblings, I'm having a really hard time trying to wrap my head around this. And if I write it down, others who are having the similar problems may benefit... if they're not bored to tears first. :D

So moving back to the original question as to why 1-9 are not outputted, I'm still not sure. Perhaps I'm not understanding how the filter chain is working?

[TheFluff]

You seem to have discovered that Avisynth script isn't an imperative programming language. Congratulations, most people never get this far.

Avisynth's "clip" variable type really doesn't represent a video clip in any meaningful way. It actually represents (the end of) a chain of function pointers. Any code that runs inside a function that operates on a clip is only executed when this function chain itself is executed, and that only happens when the client application (i.e. the program that loads the Avisynth script) requests a frame. The client application is free to request frames in any order, or to not request any frames at all. A lot of people don't realize this, and most of the time that's not really a problem, but as soon as you start adding side effects (such as writing to a file) to your GetFrame() calls, the ghosts of a hundred long-dead Lisp programmers will come to haunt you in your sleep.

Basically, there are two stages to Avisynth script execution, namely 1) the compilation stage, and 2) the execution stage (this is a simplified model, but it'll do for now).

During the compilation stage, Avisynth parses the script file, executes filter constructors (or, more specifically, all the AddFunction() hooks), runs all non-filter functions (once) and builds the function pointer chain. Animate() calls the supplied function twice here, once for the first frame of the input clip and once for the last. When this phase is done, Avisynth will execute no more code on its own; the rest is all up to the client application.

During the execution stage, the client application pulls frames through the function pointer chain in any order it feels like. If no frames are requested, no code is executed. Frames are only processed when the application requests them, which means that if you have code that runs when frame 100 is requested and the client application only requests frames 0-90, you're shit out of luck. During this stage, when a frame is requested, Animate() does its linear interpolation here, by simply multiplying with the requested frame number, and calls the given function with the generated parameters.


All of this stuff boils down to one important rule: don't create filter functions that have side-effects. (WriteFile() is one such function, and it's a very good example of Things You Should Not Do.) At least not until you've programmed Haskell for a few years and are fully aware of the caveats of doing such things. If you do insist on doing it anyway, keep in mind that you probably want your functions to be idempotent, i.e. requesting the same frame twice will only run the side-effect code once. That still doesn't help you with the ordering problem, however, and there's no guarantee that the client application will actually request all frames.

If you want a silly workaround for this, keep in mind that Avisynth doesn't actually forbid you from running arbitrary code in AddFunction() hooks, and that you have access to all function parameters there. That is, you can actually request all frames of a clip yourself, in order, during the script compilation phase. (Don't do this, it's a terrible idea.) If you want more questionable ideas regarding gratuitously abusing the compilation phase and/or ScriptClip and its runtime friends, you should go talk to StainlessS, since he seems to be fascinated with the stuff.



tl;dr: Functions that return a clip are called "lazily" when the client application requests frames. Functions that return things that aren't clips are called during the script compilation phase, before the first frame has been delivered. Both of these rules are rules of thumb only; they can be broken by runtime functions and by abusing your freedom to run arbitrary code during the compilation phase.

[maxxon]

Wow! Mind blown! Will have to sit and think on this for a bit, but it certainly clears up a lot of questions and oddities I've seen.

Thanks!


|\/|x

[maxxon]

Oh wait! This actually may answer another question I have. If I have a clip that is x frames and I break it into 3, running the middle clip through 1 or more filters, put it back together again and then do it again and again, breaking it at different points at non-overlapping sub-clips, this won't actually cause a severe performance hit will it? Only the filter code for that particular frame requested will be executed.

Is this correct?


|\/|x

[TheFluff]

I'm not sure if I understand the question. But here's an example:

c = somesource()
c1 = c.selectevery(3,0).really_slow_filter()
c2 = c.selectevery(3,1)
c3 = c.selectevery(3,2).another_slow_filter()

return c1.trim(1,1) + c2

In this case, really_slow_filter()'s constructor is executed once (during the compilation phase), and then its GetFrame() function may run once (for frame 1, when frame 0 of the output clip is requested). another_slow_filter()'s constructor is executed during the compilation phase but the filter itself is never invoked. The filter graph model means that only frames that are actually required to construct the output clip are requested from upstream filters. This means, among other things, that clip.slow_filter().trim(short_segment) runs at the same speed as clip.trim(short_segment).slow_filter() (at least with spatial filters).

Keep in mind though that most temporal filters request "surrounding" frames when they are requested to deliver one frame.

[StainlessS]

@TheFluff,
Ho Ho Ho. Ain't nowt wrong with a little flexibility. :)

When the client program requests a frame it will be pulled through the filter graph chain via your
trims/splices (with little overhead) and only those frames with additional extra filtering will acquire
additional expense. ClipClop(), might make such a requirement easier to manage, where you could
create up to 255 variously processed clips and replace ranges/frames from any of the processed
clips back into the source clip. With ClipClop 'NickNames', you can also use named clips rather than
clip indexes eg "HardDenoised" or "LightDenoised" or whatever making it easier to figure out which clip
you want to use (in practice Nicknames might likely be somewhat shorter than given).

Perhaps TheFluff would like to come back on that, his posts are always entertaining. :)

EDIT: Beaten to it.

[Gavino]

Wow! Mind blown! Will have to sit and think on this for a bit, but it certainly clears up a lot of questions and oddities I've seen.
You should read this from start to finish:
The script execution model (http://avisynth.org/mediawiki/The_script_execution_model)