Runtime filter evaluation sequence rather more complex than documented

[fvisagie]

Hi All,

There seems to be rather more to the evaluation sequence of runtime filters than the "bottom to top" mentioned at The script execution model/Scope and lifetime of variables (http://avisynth.nl/index.php/The_script_execution_model/Scope_and_lifetime_of_variables).

Here,

sequence = ""
BlankClip(length=2, width=160, height=60, color=$000000)
FrameEvaluate("""sequence = sequence + "d"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "c"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "e"""", show=false, after_frame=true)
FrameEvaluate("""sequence = sequence + "f"""", show=false, after_frame=true)
clip1 = last
BlankClip(length=2, width=160, height=60, color=$FFFFFF)
FrameEvaluate("""sequence = sequence + "h"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "g"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "i"""", show=false, after_frame=true)
FrameEvaluate("""sequence = sequence + "j"""", show=false, after_frame=true)
clip2 = last

clip1.Dissolve(clip2, 1)

FrameEvaluate("""sequence = sequence + "b"""", show=false, after_frame=false)
FrameEvaluate("""sequence = "a"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "k"""", show=false, after_frame=true)
ScriptClip("""
sequence = sequence + "l"
Subtitle(sequence)
""", show=false, after_frame=true)

the sequence of events for e.g. the second frame is abcdefghij.
http://forum.doom9.org/attachment.php?attachmentid=14250&stc=1&d=1403448205

That seems to indicate that:

for runtime filters with after_frame = false, the runtime filters of a particular filter graph node are executed in the opposite sequence to which they appear in the script (i.e. bottom to top as stated),
for runtime filters with after_frame = true, the runtime filters of a particular filter graph node are executed in the sequence in which they appear in the script,
any runtime filters of the final output node with after_frame = false are executed before those of intermediate nodes,
any runtime filters of the final output node with after_frame = true are executed after those of intermediate nodes,
intermediate nodes have their runtime filters executed in the sequence in which the nodes appear in the script filter graph.

EDIT: incorrect - see subsequent posts.
A subtle implication of the last point is that all runtime filters for a particular intermediate node are evaluated before other nodes later in the script filter graph have their runtime filters evaluated. One unwelcome consequence of that is that runtime code of intermediate nodes cannot determine the nodes' positions in the filter graph - using the nodes' after_frame=false code to increment a node counter for their after_frame=true code to query won't produce the desired result.

This is what I think is going on here, but any corrections and improvements are welcome!

Cheers,
Francois

[Gavino]

That's all as I would expect.

Frames are fetched from the filter graph from bottom (final output) to top (source filters).
For an instance of ScriptClip() or FrameEvaluate() with after_frame=false (the default), the corresponding run-time script is evaluated before fetching a frame from the filter above, so such run-time scripts are also evaluated from bottom to top. When after_frame=true, evaluation of the corresponding run-time script is postponed until after the frame from the filter above has been retrieved, so such scripts are evaluated on the 'way back' down the chain.

A subtle implication of the last point is that all runtime filters for a particular intermediate node are evaluated before other nodes later in the script filter graph have their runtime filters evaluated.
Perhaps I misunderstand you, but that seems wrong.
For example, if a later node has after_frame=false, it will have its run-time script evaluated before those of any run-time filters occurring earlier in the script.

Note that the presence of run-time functions (eg AverageLuma()) can also affect execution order, as these will require immediate fetching of a frame of their input. See the second detailed filter graph example on this page (http://avisynth.nl/index.php/ScriptClip#Advanced_conditional_filtering:_part_I). As it says there,
Notice how the addition of run-time filters and run-time functions makes the interactions between different parts of the filter chain more complex. This added complexity is managed internally by Avisynth, so you needn't worry about it. However, care is required when setting and using variables, as the order of events can be less obvious to the script writer (you!).
And, as we saw in your recent thread, caching can affect whether run-time scripts are evaluated at all on particular frames.

(EDIT) Further note: WriteFile() and WriteFileIf() evaluate their scripts after fetching an input frame, so althought they don't have an after_frame parameter, they behave like ScriptClip() or FrameEvaluate() with after_frame=true. That means the output is produced in the 'natural' script order.

[TheFluff]

The fact that the script "executes backwards" might be easier to grasp if you consider that the script statements describe a chain. If you have a chain rolled up on a wheel, you can unwind it one link at a time by grabbing one end and pulling. In the Avisynth case, when the program you open the script in requests a frame, it asks the tail end of the filter chain "get this frame for me", which causes the last filter in the chain to ask the next filter above it to deliver the frames it needs to produce an output frame, and that filter in turn asks the filter above it to deliver frames, and so on all the way up to the source filter.

Avisynth script looks like an imperative language, probably because that's what people are used to, but it's a completely different creature "under the hood".

[fvisagie]

Perhaps I misunderstand you, but that seems wrong.
For example, if a later node has after_frame=false, it will have its run-time script evaluated before those of any run-time filters occurring earlier in the script.

I'd incorrectly interpreted an incomplete test, thanks for challenging that. Here's a more complete example that proves your claim:

sequence = ""
BlankClip(length=2, width=180, height=30, color=$000000)
FrameEvaluate("""sequence = sequence + "f"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "e"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "g"""", show=false, after_frame=true)
FrameEvaluate("""sequence = sequence + "h"""", show=false, after_frame=true)
Invert()
FrameEvaluate("""sequence = sequence + "d"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "c"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "i"""", show=false, after_frame=true)
FrameEvaluate("""sequence = sequence + "j"""", show=false, after_frame=true)
clip1 = last
BlankClip(length=2, width=180, height=30, color=$FFFFFF)
FrameEvaluate("""sequence = sequence + "n"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "m"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "o"""", show=false, after_frame=true)
FrameEvaluate("""sequence = sequence + "p"""", show=false, after_frame=true)
Invert()
FrameEvaluate("""sequence = sequence + "l"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "k"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "q"""", show=false, after_frame=true)
FrameEvaluate("""sequence = sequence + "r"""", show=false, after_frame=true)
clip2 = last

clip1.Dissolve(clip2, 1)

FrameEvaluate("""sequence = sequence + "b"""", show=false, after_frame=false)
FrameEvaluate("""sequence = "a"""", show=false, after_frame=false)
FrameEvaluate("""sequence = sequence + "s"""", show=false, after_frame=true)
ScriptClip("""
sequence = sequence + "t"
Subtitle(sequence)
""", show=false, after_frame=true)

http://forum.doom9.org/attachment.php?attachmentid=14251&stc=1&d=1403506535

EDIT: Let me restate this more completely:
Is there a way for runtime code at the level of nodes incident to the final node to determine the number of such incident nodes, before the last such node has been evaluated (preferably the ordering of those nodes also)? I.e. for runtime code at the level of clip1 to determine that it isn't the last node to be evaluated? As far as I can see that isn't possible (it's only possible for runtime code at the level of the final node to determine that). The reason I'm looking for this capability is to consolidate (possibly expensive) processing of all incident nodes' runtime information before the final node is reached. That would make it possible to leave caching enabled and to still obtain valid results (within reason) when scrubbing the timeline during editing. Hope things make sense now ;).

[TheFluff]

EDIT: Let me restate this more completely:
Is there a way for runtime code at the level of nodes incident to the final node to determine the number of such incident nodes, before the last such node has been evaluated (preferably the ordering of those nodes also)? I.e. for runtime code at the level of clip1 to determine that it isn't the last node to be evaluated? As far as I can see that isn't possible (it's only possible for runtime code at the level of the final node to determine that).

No, I'm pretty sure that's not possible from within Avisynth script or an Avisynth filter. At the point where GetFrame() runs you can examine the upstream filter and traverse the filter chain upwards basically as far as you like if you are so inclined, but you cannot interact with your downstream filter, you can only return a frame to it (or throw an exception). Thus, you can determine how many filters are above you in the chain, but you know nothing about anything below you and you can't interact with it in any way.

Now, if you're an application that uses the Avisynth API directly and builds filter chains programmatically in a real programming language, instead of trying to do all of this silliness in babby's first graph description language with a bunch of weirdo tricks hacked in, all of this becomes trivial. Unfortunately, that's not what you asked for help with.

edit: actually, no, you can't traverse the graph upwards either, you can only interact with the filter immediately above you in the chain. For some reason I thought IClip.child was public, but it's not.

edit edit: really, stop trying to do complex things at runtime in Avisynth script. It's a terrible idea. The language kinda sorta works for simple graphs, but that's really all it was designed for. As soon as you start doing more complex things you immediately run into limitations and subtle gotchas like these. If you want to do more complex things, you need to start writing C++, either an Avisynth filter or an application that interacts with the Avisynth API.

[fvisagie]

If you want to do more complex things, you need to start writing C++

Unfortunately it would take more time to overcome all the necessary hurdles than I have available at the moment. I'd need to upskill to C++ from very rusty C, obtain and set up some development environment, learn about AVS' structure and its APIs etc. I do dream of doing that one day, though :), and appreciate your pointing out the limitations of runtime scripting vs. interacting from C++ code.