Avisynth+

[ultim]

If there's room for feature requests, can we skip some arguments when calling functions like this:
some_function(a,b,,d,e)
if I don't want to change the default value for the third argument, and don't want to explicitly use the names of the 4th and 5th arguments.

Sure, there is always room for feature requests! So please post everything on your mind here on the forums, or as a GitHub issue. It allows us to discuss your ideas. In this particular case, I agree with Gavino, that the syntax you demonstrated would let many typos go undiscovered, but Gavino proposed a nice way to get what you'd like without its dangers. Cheers to both of you.

[raffriff42]

Sure, there is always room for feature requests!How about this syntax change...for better self-documenting function declarations:

(EDIT or would this be out because it breaks compatibility?)## function declare examples, current syntax:

function SAA(clip C, int "ss", bool "cp", int "aa")

function FunkyDeblock(clip c, int "quant", int "th",
\ int "radius", bool "deblock", bool "depump")

function Detwinkle(clip C, float "grainmask",
\ int "grainmode", int "tstren",
\ float "hicont", float "hisharp",
\ float "unblur", float "mosens",
\ int "hismooth", String "show")

## add syntax support for DEFAULT VALUE:

## * defaults are optional of course
## * compiler assigns default values automatically if specified
## * string literals only - no variables or Eval statements!
## * strings are not quoted (certain chars can't be supported - unless escaped?)
## * empty String defaults accepted, meaning empty string, as opposed to Undefined.
## * other empty defaults are syntax errors
## * specify explicit undefined defaults with "Undefined", "Nop" or "null"

function SAA(clip C, int "ss=2", bool "cp=true", int "aa=48")

function FunkyDeblock(clip c, int "quant=4", int "th=3",
\ int "radius=4", bool "deblock=true", bool "depump=true")

function Detwinkle(clip C, float "grainmask=1.3",
\ int "grainmode=2", int "tstren=25",
\ float "hicont=1.3", float "hisharp=0",
\ float "unblur=3", float "mosens=3",
\ int "hismooth=2", String "show=")


## add syntax support for DEFAULT VALUE and VALID INPUTS:

## * compiler assigns default values and enforces valid inputs
## * space optional before start of "[...]" range specifier
## * ranges ".." delimited? Dunno.
## * valid alternate values "|" delimited
## * valid ranges: "[min..max]" "[min..]" "[..max]" or "[alt1|alt2|alt3]"
## * consider enforcing range specifier if default is used

function SAA(clip C, int "ss=2 [1..8]", bool "cp=true", int "aa=48 [4..256]")

function FunkyDeblock(clip c, int "quant=4 [0..51]", int "th=3 [0..128]",
\ int "radius=4 [1..60]", bool "deblock=true", bool "depump=true")

function Detwinkle[clip C, float "grainmask=1.3 [0.7..18]",
\ int "grainmode=2 [0..18]", int "tstren=25 [1..255]",
\ float "hicont=1.3 [0.1..10]", float "hisharp=0 [-1..1]",
\ float "unblur=3 [0..99]", float "mosens=3 [0.1..5]",
\ int "hismooth=2 [1..5]", String "show= [blur|mask|luma|cyan|]")

[ultim]

(EDIT or would this be out because it breaks compatibility?)

That's not a problem. As long as old Avisynth scripts can continue to run on Avs+, it is fine. I personally don't require that Avs+ scripts must also run on Avisynth, because otherwise a lot of improvements wouldn't be possible.

As for the feature at hand, I like the ideas, though I'd rather have some syntax details different. For example, the specification of the default values as per you wouldn't let the complier know the value's type. Only putting quotes around the var's name solves this, or in case a default values is supplied, not putting any quotes at all, because parameters that have a default are optional by definition.

[ultim]

Finally, some time to write down things on my mind. Well, not all the things, but it's a start ;)

Everybody hates MT-modes, me included. They shouldn't be required, but unfortunately we cannot dispose of them, because there are a lot of old plugins out there that were written without any consideration for multithreading. Not supporting MT modes would mean throwing out a lot of great plugins, keeping only those that perfectly adhere to the points in my previous "Plugin Writing Tips" post (http://forum.doom9.org/showthread.php?p=1649886#post1649886). So I've been wondering, what MT-modes should be supported? We know Avisynth-MT supports at least 5, but which of those are really usefull? I have recently recognized that the Avisynth-MT OP post has been updated, and now it only lists modes 1-3 as "usefull". I've set out to compile my own list, while in the process answering questions such as "When exactly do we need those modes?", "What modes are needed if no plugins are fixed?", "What modes are needed if all plugins are fixed (as per my plugin writing tips)?" and also "Which plugins cannot be used with the recommended set of MT-modes (1-3) of Avisynth-MT"?

I started to think analytically, and compiled a set of three tables found here (https://docs.google.com/spreadsheet/ccc?key=0AqVT4Ywc6rzYdHNoSm9mUWVDMWM4YTJvTTdEZFNlN1E&usp=sharing) to summarize my findings. I might have overlooked some things or made a mistake or two, so let me know if you find anything. Each table lists four negative plugin properties as their columns, basically violations against multithreading support. The last column contains the protection that Avisynth+ would need to implement to keep that kind of plugin working correctly despite its violations.

Table 1. has all the different combinations of properties, irrespective of if that combination is even realistic at all or not. For example, there can be no plugin such as its 9. row, because a plugin can only possibly require ordered frame retrieval if it stores at least some kind of internal state. There are also cases of inherently broken plugins that cannot be guarded against in Avisynth, because they'd require limiting the plugin simultaneously to both single AND to multiple instances. So, let's see, how many MT-modes do we need? Six MT-modes, ouch. More precisely, six MT-modes if we do want to support all possible kinds of "broken" plugins, without fixing any of them. This is where I've said "no way I'm gonna implement all those MT-modes"! So what can we do? Fix the plugins of course!

One of the easiest plugin fixes is getting rid of storing the environment pointer in classes or globals. Fixing these violations has some nice advantages: First of all, this is a trivial fix in any plugin, so as long as we have the sources, this kind of fixing will go fast and easy. Also, quite importantly, getting rid of this single violation in every plugin will effectively ensure that there is at least a way to use all plugins in multithreaded modes (as long as the required protections are implemented in the core). So, never store the env.ptr. in classes and globals, and we are down to only 7 cases and 4 MT-modes (3 protections). Welcome to table 2. The protections in table 2. are what we'd need to have if we could perform only a trivial fix in the few plugins that need it.

Getting to table 3. from here is actually quite simple, only two simple facts have been used for this transformation. One is to get rid of using global variables. This is also a simple fix, as basically all that needs to be done, is to make class variables out of global variables. This will not effect plugin behavior at all, but it will remove the need to serialize these plugins, which is practically the same as letting them run fully parallel. Believe me, this fix is almost just copy-paste work for many plugins (not for those whose algorithm requires it, but those can be fixed too in every case, except that more work needs to be done). The main obstacle will (again) be having the sources to the offending plugins. OK, so far so good. But to get to table 3., I've also done something else: I've assumed that there are no plugins that must serially access all frames up to frame n, to calculate frame n. Why? Because such a plugin would hardly be usable in Avisynth (as it would make it impossible to seek inside or trim the video), and thus I actually doubt that one like that exists at all. (EDIT: I've been let to know that there are such plugins in existence. But all Avs+ could do to protect them is to play back all frames of the video internally until the requested frame is reached, which is performance-wise not practical at all - think of having to wait for multiple hours just to seek into the middle of the video clip).

So, we're at table 3., and IMHO have don't need any complex plugin fixes. Table 3. summarizes the MT modes that we need to implement if we assume we can fix all plugins until at least none of them uses global vars. Is that realistic? Yes, as long as we have the sources. Surprisingly to me, the conclusion in this table is that we either need no protection at all, or a single protection for unfixed plugins that store state across frames, but that still allows for full performance/parallelization, only with memory overhead (fully fixed plugins never require any protection from Avisynth). A second protection (3rd mode) is added to optimize source filters further, and while tbh I don't think it is strictly needed for correct source filters, it should still help their performance by reducing disk seeks. Another (important but sad) advantage of having this special mode for source filters, is that the same protection also lets us use unfixed plugins that use global or static variables. Note that while this special mode allows you to use such plugins, you should not rely on this mode (except for source filters), because it will cripple your multicore performance and flush it down the toilet, instead it is better to work on the plugin to not require a single serialized instance.

Avisynth+ will likely implement the modes listed in table 3., and I will take time to try and fix/modify as many plugins as possible, as well as motivate others to do so. In contrast to Avisynth-MT though, Avisynth+ will also allow plugins to register themself for the correct MT-mode they need, so users won't have to know by heart for all plugins if they have been fixed, or if they are unfixed, which mode they need.


Congrats for reading this rant of mine. What's to take home ?
- Fix plugins that store and reuse the env. ptr. It is often neglected but very important.
- Fix plugins that use global or static variables. We can guard against them to ensure they will work correctly, but they cannot be executed in parallel, and will seriously limit the multithreaded performance for your whole filterchain (even for correct plugins, due to frame dependencies).
- You can violate multiple rules for multithreading, and not all are equally important. Most importantly, ensure that the env.ptr. is never stored and reused. If you still have energy left after that, fix the plugin further so that no global/static state is used. Then if you still have energy left, try to turn all class members into read-only.
- The three MT-modes listed as "usefull" for Avisynth-MT really are the three most usefull MT-modes.
- The MT-modes of Avisynth-MT are not enough to use every single plugin when MT-mode is enabled. You might get crashes or corrupted frames if you use an unsupported plugin in an MT-mode, but don't blame Avisynth-MT, it is really not his fault. Blame the author of the offending plugin.
- Always fix plugins instead of relying on MT-protections. Most fixes are very easy and you will get much better performance than using a built-in protection.
- If you have the sources to a plugin, please don't keep it to yourself.

[TurboPascal7]

Okay, here are my thoughts on the issue. Everything runs on the assumption that most filters will get updated. There are just too many things to add and I want to hope that with the help of VSynth team we will be able to make the most used plugins less bad quite fast.

Common things:

In any case the core should include a static thread pool with number of threads configured on script initialization. Additional function like SetThreads(4) should be provided with all calls other than the first one ignored. This is exactly how Avstp works.
Nothing should be threaded by default. If the core does not have any information on the filter it will run singlethreaded. Always.

The question is: how does the core get the required info if the filter does not register itself?

1. Single-line SetMtMode
First option is to make the user put this info in his script but unlike the original SetMtMode apply threading only to the filter inside the call. The result script would look like:
SetThreads(4)
Source(...)
SomeRegisteredFilter() # registers itself as threaded, runs with 4 threads
SomeBrokenFilter() # does not register itself, singlethreaded
ForceThreading(SomeFilter(), 2) # SomeFilter runs in mode 2, forced threading

This assumes that avisynth evaluator would allow making SomeFilter() threaded and you don't have to use like SetMtMode(2).SomeFilter().SetMtMode(0) which is quite ugly.
Compared to original mt-mode solutions, this makes immediately obvious where and which mt-mode is used. Of course it runs on assumption that most filers will get updated, otherwise you'll end up writing ForceThreading call on every line or separating all your stuff into functions just to thread said function in one call.

However, just like the vanilla SetMtMode solution, this has some serious disadvantages.
First of all, it requires users to know implementation details of every plugin and script they're using, in other words - making the user learn a lot of stuff he doesn't care about.
Second, it makes your script less readable by simply adding garbage calls to it.
Third, imagine you're using some dither.avsi function with 50 calls to perfectly threadable masktools and a single call to hopelessly broken removegrain. You'll end up using the slowest mt mode of all (if any) unless you're willing to modify this script. No one will do that.

So I think we should face it: SetMtMode solution is terrible. I'd rather continue running scripts singlethreaded than spending hours and hours understanding what I can and cannot thread just to later end up with a broken encode because I used a wrong mt mode.

2. Embed this information into the core
Do pretty much the same thing VSynth is doing (https://github.com/vapoursynth/vapoursynth/blob/master/src/avisynth/avisynth_compat.cpp#L207): embed all the required info into the core. Here is how it'd work:

The core has predefined dictionary of mt modes for every plugin known. {"mt_lut": 0, "removegrain": 2} and so on. User has no ways of providing this info. No SetMtMode function exists.
When a threaded plugin is loaded, its registration info overrides the default dictionary values. This is important because we'll never be able to keep the list up to date.
All filters that do not register themselves and don't have an entry in the core dictionary run singlethreaded.
When a filter gets updated, we simply remove its entry from the core. This makes it singlethreaded for users that updated the core but didn't update the filter. Can't be helped.

Yes, this solution is harder for the devs since one would need to gather the info required to embed it in the core. Yes, this solution is not optimal from performance point of view - even if the filter can be easily threaded, unless it registers itself or has an entry in the core dictionary - it won't.
But it does not require the user to know anything about filters he's using. Plus it doesn't add anything to your scripts. Plus you automatically get threading even inside complicated scripts that no one will ever modify. User updates avisynth.dll and it just works. Then when plugins get updated, he replaced the files in his plugins folder and it just works.

I definitely think this is the way to go. MAXIMUM PERFORMANCE is cool and everything, but I really want things to just work without me doing anything. In the end, 20 minutes of my time is much more important for me than those four additional hours my PC will spend on filtering running some more stuff singlethreaded.

[ajp_anton]

Bug?
subtitle doesn't accept an argument named "text". The text has to be the first unnamed argument for it to work. I was trying to re-organize the arguments when calling subtitle for easier readability.

Feature request: allow comments on split lines, like this:
filter(argument 1 #comment
\, argument 2)

[ultim]

Bug?
subtitle doesn't accept an argument named "text". The text has to be the first unnamed argument for it to work. I was trying to re-organize the arguments when calling subtitle for easier readability.
No, not a bug. The "text" argument is mandatory, meaning it is not a named argument. Because it is not a named argument, you cannot reorder it and refer to it by name.


Feature request: allow comments on split lines, like this:
filter(argument 1 #comment
\, argument 2)
Noted. Thx.

[Gavino]

Feature request: allow comments on split lines, like this:
filter(argument 1 #comment
\, argument 2)
You can use the /* ... */ style of comment to do this (in both Avisynth and Avisynth+)
filter(argument 1 /* comment */
\, argument 2)

[LaTo]

Surprisingly to me, the conclusion in this table is that we either need no protection at all, or a single protection for unfixed plugins that store state across frames, but that still allows for full performance/parallelization, only with memory overhead (fully fixed plugins never require any protection from Avisynth).

If we really need some writable class member, what is a good method?

(I was thinking about a wrapper which dispatch data between threads using a hashmap of thread id)

[TurboPascal7]

If we really need some writable class member, what is a good method?

(I was thinking about a wrapper which dispatch data between threads using a hashmap of thread id)
There is also an option of letting the core create a few instances of your filter class. This way you won't have to implement any kind of synchronization or complicate your plugin.

Using a hashmap in this case is quite hard. You will either have to implement a thread-safe hashmap or know all possible filter ids during hashmap creation in filter constructor. First way will complicate your implementation a lot, the other one will bind your plugin to avs+, plus it is somewhat a bad api design.

[ultim]

If we really need some writable class member, what is a good method?

(I was thinking about a wrapper which dispatch data between threads using a hashmap of thread id)

You don't need to do that kind of dispatching on your own.

The "best" (ahm) way depends on how you'll use the class member. If you don't mind missing out some frames here and there, the easiest way for you as a filter author is to not implement any synchronization, and let Avisynth create multiple instances of your filter. This will of course come with some overhead, mostly some memory, but it's the quick'n dirty method, because from your filter's point of view there will always be just one thread.

On the other hand, if you'd want to share your state between all your instances (and you don't want to miss out on any of the frames), then you'll want to implement your own synchronization, and ask for the 1st MT mode where Avisynth does not provide any "workaround"/protection. In this case Avs+ will create just a single instance of your filter (which will be the same one called from every thread), and you'll need to synchronize access to the state yourself. Note though that from your point of view, the frames might (probably) still not be accessed in sequential order.

So to sum up, the first method is useful if you only need to store some temporal state, but you don't mind if your filter's instance will not see some frames. Otherwise, use the second method. Jugding by the fact that you wanted to dispatch data based on the thread ID, you probably want to use the solution involving multiple instances.

(And for future reference, if you really do want to dispatch data yourself, you don't need to use a hashmap. Avs+ will sequentially number the threads for you, so you'll be able to use a simple array lookup. You'll tie your plugin to Avs+ by that though.)

[SEt]

Don't bother with MT modes if you want future-proof solution. It's a ok hack for doing work right now, but it's not scalable. I have much better idea how efficient threading should be done, but never got to implement it (mostly because I'm not pressed since MT modes do work for now).

That have been said, I'm busy with other areas atm. (And you will likely see something useful for Avisynth in several days. ;))

[TurboPascal7]

I have much better idea how efficient threading should be done
You could tell us about this idea of yours instead of just saying you have it, you know.

[TurboPascal7]

Okay, this projects needs some user input again.

A new issue got created on github (https://github.com/pylorak/avisynth/issues/10) but unfortunately we couldn't decide whether it should be implemented or not. This is not a critical issue of any sort but it is quite annoying.

How to reproduce
The minimal example as posted on github is:
blankclip(1000,720,480, "yv12",color_yuv=$808080)
anotherfilter = blankclip(1000,720,480, "yv12",color_yuv=$FF8080)
StackHorizontal(last, anotherfilter)
It works fine. Now if you remove the last line, making the script
blankclip(1000,720,480, "yv12",color_yuv=$808080)
anotherfilter = blankclip(1000,720,480, "yv12",color_yuv=$FF8080)
you will get a "Not a clip" error.

Why
Assignment operator returns uninitialized avisynth value, basically nothing. This "nothing" does not override the last variable so you still can request that one simply putting last on the next line.

Proposed solution
Instead of returning nothing, return the last clip value if it exists. Same effect could be achieved right now by adding last to the line after every assignment in your script.

Drawbacks
I don't see how this could break any existing script. It might reduce performance of script initialization a bit since it requires looking up the last variable on every assignment, but you won't notice this. Of course I might be missing something.

Why bother
You write a script and then you decide to try it without a few lines at the end, so you comment them out and BAM - "not a clip". This could be solved by always adding "last" to the last line of your scripts but it's kinda hacky.

What's needed
Your input. Basically, something like "oh I had this problem before, would be nice to get this fixed" or "nah, this will break my scripts because I found a way to rely on this behavior", or even "omg no, this will reduce initialization performance by 0.00000001%, can't have that" would work.
IanB's opinion if we could fix this in the vanilla avisynth would be really appreciated.

Thanks in advance.

[wOxxOm]

It might reduce performance of script initialization
There won't be any perceivable penalty at all in case avisynth+ would implicitly add 'last' only to the end of a scope (function/eval snippet/entire script).

[Gavino]

If I understand correctly, the proposal is that a script that ends with an assignment should return 'last', instead of a 'void' value as currently. I assume the same would apply to functions ending with an assignment (and also to Eval() contents).

I don't expect any currently working scripts (or functions) would break as a result.
However, a script that ends with an assignment is probably wrong. Why assign to a variable that will never be used?
Who knows what the user intended in such a case?
Perhaps he meant to return the value being assigned, rather than a 'last' that may have been set many lines earlier.
So I think it is better to keep the existing scheme, which forces the user to make the intention clear.

There won't be any perceivable penalty at all in case avisynth+ would implicitly add 'last' only to the end of a scope (function/eval snippet/entire script).
However, it must only do this for assignments. With the existing data structures, at the time the end of the scope is being evaluated, you no longer know whether you had an assignment - and when evaluating an assignment, you don't know whether it's the last thing in its scope. So the simplest implementation, if going ahead with this, would be for all assignments to return 'last' as their value.

[innocenat]

I think implicitly return last variable on every script is a good idea. Sure, it might not seems resonable at first, but if you need temporary storage in your script you probably aren't beginner and should know what you are doing and what last variable refer to anyway.

As for multithreading, I dont think creating multiple instances of filter is a good idea. I think avs interfacd should provide filter thread local storage and thread-safe global storage. On filter that require linear frame access, filter should be registered as such, but avs interface should provide a way for filter to tell the core that the process that require linear access has done and the core is allowed to request next frame while this frame is still processing.

I know this is not a light work and may break compatibility, but it's how I imagine it right now.

[TurboPascal7]

However, a script that ends with an assignment is probably wrong. Why assign to a variable that will never be used?
Who knows what the user intended in such a case?
Perhaps he meant to return the value being assigned, rather than a 'last' that may have been set many lines earlier.
So I think it is better to keep the existing scheme, which forces the user to make the intention clear.
This is a job for an analysis tool and not for the core itself, imho. I don't see how assignment on the last line is wrong while having any number of unused variables in other parts of the script is okay.

As for multithreading, I dont think creating multiple instances of filter is a good idea. I think avs interfacd should provide filter thread local storage and thread-safe global storage.
There is a "small" problem with this solution - it ties the plugin to avs+. Maybe it would be okay if we were starting from scratch, but with o9k avisynth forks around I think it would be reasonable to avoid coupling the plugins to avs+ interface as much as possible. Not to mention that the core will need to implement the multiple class instances solution anyway just to thread some existing plugins without modifications.

[cwk]

Can we assume this only attempts to solve the case where the last variable received a valid video clip, just fails to return it?

Consider the case where a script is calling functions that are under construction.


mySource = avisource("blah")
a = mySource.SomeCompletelyAwesomeFunction()
return a


Here 'not a clip' is useful, because it tells me my function is not returning a valid clip.

Another approach would be to make the existing 'not a clip' message more informative. It could tell the user what the current return value is, or point out the line providing the return.

[TurboPascal7]

Can we assume this only attempts to solve the case where the last variable received a valid video clip, just fails to return it?
This proposal would change exactly nothing in your example if this is what you're asking about.

[Reel.Deel]

As a novice AviSynth user I had this problem a few times, and when someone is new to AVS this could be kind of a turn off. Now that I know better and I'm used to this behavior would I like to see a change? Not really, but a more comprehensive error message would definitely be a good idea.

@TurboPascal7
On a side note, any plans to integrate FTurn into AviSynth+?

[wOxxOm]

On the 'last':

a typical mix of filter calls and assignment statements:mainfilter1()
b = auxfilter()
# last = mainfilter1() as expected
mainfilter2()
-----
now someone commented out just the last line knowing how 'last' clip is usually preserved over assignment statements, thus expecting the 'last' clip to carry result of the last filter applied in - what should we call it? - the main flow of the filter graph.mainfilter1()
b = auxfilter()
# mainfilter2()last = nothing????

So, irregardless of user experience, there seems to be no logic *syntax-wise* behind current behavior (but of course if one looks into the source code it's clear that filters() explicitly try to use the 'last' clip in rather an unsightly manner, which doesn't compensate the lack of syntax logic anyway).

[TurboPascal7]

On a side note, any plans to integrate FTurn into AviSynth+?
Will be in the next version or the one after that.

[Daiz]

Not a clip

I've run into this so many times I've lost count, usually in the exact situation described (testing filtering and commenting out a line), so here's a very large "yes, please make assignment return last" from here.

[Groucho2004]

However, a script that ends with an assignment is probably wrong. Why assign to a variable that will never be used?
Who knows what the user intended in such a case?
Perhaps he meant to return the value being assigned, rather than a 'last' that may have been set many lines earlier.
So I think it is better to keep the existing scheme, which forces the user to make the intention clear.
Exactly my thoughts. Also, less experienced users would probably in many cases not know what the script returns with this modification.
When you write a function in C for example, the compiler forces you to return from that function with the correct type and that makes sense. You don't dumb down the language because of convenience.

[TurboPascal7]

Exactly my thoughts. Also, less experienced users would probably in many cases not know what the script returns with this modification.
When you write a function in C for example, the compiler forces you to return from that function with the correct type and that makes sense. You don't dumb down the language because of convenience.
Avisynth is not C. Stop applying rules of normal languages to it. You don't want to write "return derp" in the end of every function/script, do you?

As for assignment on the last line being wrong argument, what about scripts like this:

super=MSuper(last, pel=4, planar=true)
pp_super=fft3dgpu().MSuper(pel=4, planar=true)

b1vec = MAnalyse(super, delta=1, isb=true, blksize=8, overlap=4,search=5)
f1vec = MAnalyse(super, delta=1, blksize=8, overlap=4,search=5)

b1clip = MCompensate(orig, super, b1vec, thSAD=400)
f1clip = MCompensate(orig, super, f1vec, thSAD=400)
Whoops, variable pp_super is unused, the user ended up calculating vectors on the wrong clip. This is a bug. And we did exactly nothing to prevent it. Now please explain me the reason why we should do something to prevent this kind of behavior if this assignment happens on the last line. Because it's easier to implement? Because it's always been like this? I think we either go full way preventing users from shooting in their own feet and forbid unused variables altogether (this is a bad idea please don't even consider this) or drop this kind of argument.

As for users not knowing what script returns: if this behavior got silently changed, how many people do you think would notice? :)

P.S. I'm sorry if I sound too "rude" and I should probably stop posting on the issue. The question was posted to get opinions, not arguing about them. Sorry.

[Groucho2004]

Avisynth is not C.
Thank you for your wise words. I was using it as an example, not to be taken literally. The logic still applies. Your logic is just different.
You don't want to write "return derp" in the end of every function/script, do you?
I kinda do...

[Sapo84]

Whoops, variable pp_super is unused, the user ended up calculating vectors on the wrong clip. This is a bug.

It depends, maybe the user was using pp_super before, didn't like the result and stopped using it without removing the variable.

Anyway, I'm against a fix that will allow sloppy coding (commenting lines and leaving an assignment as the last line sounds like sloppy coding to me), instead of commenting just use a return last in the place you need to test the output.

[Gavino]

Now please explain me the reason why we should do something to prevent this kind of behavior if this assignment happens on the last line.
Because in that the case the problem is more than just the unused variable (that's just an aspect I mentioned to illustrate that the script was incomplete). The issue is that the intended return value is not clear.

instead of commenting just use a return last in the place you need to test the output.
Yes, I was about to make that point myself.

[wOxxOm]

Commenting was just an example, there's no need to focus on that per se.

The real problem is the quirky behavior of an assignment statement in avisynth at the end of a scope.
It is inconsistent.

Either make all assignment statements kill the 'last' clip (nonsense), or make the last assignment behave exactly the same as the others.

[TurboPascal7]

Because in that the case the problem is more than just the unused variable (that's just an aspect I mentioned to illustrate that the script was incomplete). The issue is that the intended return value is not clear.
function test() {
colorbars(640, 480)
clp = blur(1)
}

colorbars(320, 240)
test()
blur(1.0)

What did the user intend to return here? This script actually works, with function basically being a nop() call. Talk about preventing users errors. =)

Anyway, this was actually pointed out as an example where this proposal would break stuff, which for me is a showstopper. So I'm gonna quit here. But I (and not only I) would still love to see your ideas on multithreading. ;)

[Gavino]

make the last assignment behave exactly the same as the others.
It does behave the same as the others, there is no inconsistency.

Note that the commonly held belief that 'last' is returned by default is wrong. See this post.

[wOxxOm]

It does behave the same as the others, there is no inconsistency.Internally, yes. But not from a user's point of view.
Note that the commonly held belief that 'last' is returned by default is wrong.It's 'wrong' only because of the issue we're talking about.

[Gavino]

Internally, yes. But not from a user's point of view.
What is the inconsistency from the user's point of view?

It's 'wrong' only because of the issue we're talking about.
Not just this issue (assignment) - if the last thing is a non-clip expression (eg an int), 'last' is not returned either, even if set earlier.

The Fluff had an excellent post just now describing how it works in detail, but it seems to have disappeared. :(

[wOxxOm]

What is the inconsistency from the user's point of view?Well, I thought it's shown in my posts above...
Not just this issue (assignment) - if the last thing is a non-clip expression (eg an int), 'last' is not returned either, even if set earlier.If it's not an assignment then from user's point of view it's the same as any other filter expression, so the 'last' is returned as expected.

[TheFluff]

I deleted the post because I thought the discussion had already covered what I was talking about but here's the relevant part again:

Avisynth has four types of expressions:
1) explicit returns ("return <expression>")
2) explicit assignments ("variable = <expression>")
3) implicit assignments to last (all expressions that aren't one of the two first types and evaluate to a clip)
4) other expressions where the return value is thrown away (everything else)

When the last expression in a block (either a function or a script) is not an explicit return (type 1 above), Avisynth will evaluate the expression and implicitly return the value it evaluates to, regardless of what the type of that value is (clip, int, string, undefined, whatever). The quirky thing here is that while expressions of type 3 and 4 above return the value of the expression, an explicit assignment returns an undefined value. I think this is a lazy fix to make sure an explicit assignment of a clip to an arbitrary variable doesn't overwrite last, which I suspect would happen if the assignment returned the assigned value like every other assignment operator on the planet does. The inconsistency is that an explicit assignment returns undefined while an implicit assignment to last returns the assigned value.

What TurboPascal7 proposed to change is simply the implicit return mechanism: if you attempt to implicitly return an explicit assignment expression, last should be returned instead of the undefined value, if last is defined at that point. The problem with this is that there's nothing about the scripting language that forces you to use last, and thus there's really nothing that says that at the end of a block the contents of last would be a better or more correct value to return than any other arbitrary value. I can accept an operator returning an undefined value, even if I think that's ugly, but having an operator return a defined, arbitrary value that may be completely unrelated to any of its operands is beyond ugly.

What I think you should do is promote the assignment operator to full citizen status like all of Avisynth's other operators, give it a place in the precedence table and make it return the assigned value like assignment operators in basically every other language do. Then fix the script parser so this doesn't overwrite last when the assigned value is a clip. This will probably break things though.

[ultim]

Yesterday we discussed this issue extensively with a couple of folks on IRC, and because there was no consensus, I encouraged them to bring this to the forums, to see what others think. I take Gavino's, Fluff's, Sapo's and Groucho's side here, and their arguments are the same why I wouldn't like to make this modification. To reiterate my points (which practically already have been said here by others), IMHO integrating this fix is not a good idea because:
- It will lead to confusion with non-expert users. If they expect anything to be returned after an assignment, they will expect the assigned value to be returned, not some hidden variable ('last') from a couple of lines earlier.
- More importantly, it can (and probably will) lead to hard-to-discover bugs, even with more experienced users. If the last statement is an assignment, did the user want the assigned value to be returned, or did he forget a line with 'last'? Changing the behavior to which basically equals to adding an implicit 'last' statement will hide one of these errors, while the current behavior makes the user spell out his expectations exactly (which avoids both error possibilities).

Wishful thinking on my part, but if we really wanted to make return values more logical and easier to use, we should either (1) return the last assigned value whatever it is even if not 'last', or (2, preferred) get rid of implicit returns completely and always require a 'return' statement in any block for a variable to be returned. Of course, both of these solutions would seriously break a lot of scripts, so obviously none of these two are gonna happen.

[Gavino]

I don't want to prolong the debate unnecessarily, as it seems we have now reached a conclusion, but just to address a couple of points raised...

If it's not an assignment then from user's point of view it's the same as any other filter expression, so the 'last' is returned as expected.
Consider
function f(clip c) {
c
...
width()
}
Here, we don't have an assignment but an int expression.
The value of that expression is returned rather than the value of 'last' (which is 'c').
(Note that only clip values are ever implicitly assigned to 'last'.)

The inconsistency is that an explicit assignment returns undefined while an implicit assignment to last returns the assigned value.
Strictly speaking, there is no implicit assignment to 'last' on the last statement. Instead, it is treated as an implicit 'return' (and defined this way in the docs).
If you look at it this way, then there is no inconsistency in the treatment of assignments.

[Lenchik]

"nah, this will break my scripts because I found a way to rely on this behavior" or something like that. I got used to current behaviour. And moreover i am often creating chains of scripts by importing them into one biggest. I think that new modification can make more difficulties than making life easier.

[SEt]

You could tell us about this idea of yours instead of just saying you have it, you know.
I doubt anyone it going to implement it, but as you requested, general outline:


In one word I can describe it as "superscalar" architecture. The same ideas as used in modern superscalar CPUs:

1) Work is separated from data flow. (-> denotes function call)
How it goes now: Render->FilterB->FilterA->Source.
How it should be: Scheduler->Source, Scheduler->FilterA, Scheduler->FilterB.
2) Filter has to be able to tell the scheduler which frame numbers from which its sources it needs to produce each destination frame number. Fast, without actual frames, but maybe imprecise (can fail its processing call and request more).
3) There are several kinds of filters: one instance multicall (like MTMode 1), one instance singlecall (MTMode 3), multi instance one call per instance (MTMode 2). Type is provided by the filter itself, always.
4) Based on 1-3, number of available hardware threads and data flow graph Scheduler builds filter call plan in superscalar way: several frames in flight on different stages.
5) Refcounting and no cache. Based on call plan each frame has its own reference count and will be kept exactly until all references are used. All produced frames are read-only, but could be forwarded (for filters like Trim).

As could be seen, such architecture not only would allow efficient and scalable utilization of hardware threads but will be able to incorporate even extremely threading-unfriendly filters with sequential frame processing without any correctness or speed drawbacks given enough other work in script.

Also it would be very memory-efficient: only actually needed data is kept and memory is consumed as much as needed for no redundant work in the scheduling window.

Cons: hard to implement, scheduling is active and will consume some resources, how filters work needs to be modified: source frames are provided to filter, not requested from inside.


Legacy Avisynth 2.5 filters work through separate filter "wrapper". Threading information is provided to it in textual user-editable file config.

[ultim]

"nah, this will break my scripts because I found a way to rely on this behavior" or something like that. I got used to current behaviour. And moreover i am often creating chains of scripts by importing them into one biggest. I think that new modification can make more difficulties than making life easier.

actually, i'm pretty confident that the change wouldn't break any existing scripts. from that point of view it should be totally harmless. i have other concerns, see my previous post.

[ultim]

I doubt anyone it going to implement it, but as you requested, general outline:


In one word I can describe it as "superscalar" architecture. The same ideas as used in modern superscalar CPUs:

1) Work is separated from data flow. (-> denotes function call)
How it goes now: Render->FilterB->FilterA->Source.
How it should be: Scheduler->Source, Scheduler->FilterA, Scheduler->FilterB.
2) Filter has to be able to tell the scheduler which frame numbers from which its sources it needs to produce each destination frame number. Fast, without actual frames, but maybe imprecise (can fail its processing call and request more).
3) There are several kinds of filters: one instance multicall (like MTMode 1), one instance singlecall (MTMode 3), multi instance one call per instance (MTMode 2). Type is provided by the filter itself, always.
4) Based on 1-3, number of available hardware threads and data flow graph Scheduler builds filter call plan in superscalar way: several frames in flight on different stages.
5) Refcounting and no cache. Based on call plan each frame has its own reference count and will be kept exactly until all references are used. All produced frames are read-only, but could be forwarded (for filters like Trim).

As could be seen, such architecture not only would allow efficient and scalable utilization of hardware threads but will be able to incorporate even extremely threading-unfriendly filters with sequential frame processing without any correctness or speed drawbacks given enough other work in script.

Also it would be very memory-efficient: only actually needed data is kept and memory is consumed as much as needed for no redundant work in the scheduling window.

Cons: hard to implement, scheduling is active and will consume some resources, how filters work needs to be modified: source frames are provided to filter, not requested from inside.


Legacy Avisynth 2.5 filters work through separate filter "wrapper". Threading information is provided to it in textual user-editable file config.

1) As you said, Avisynth right now is "Render->FilterB->FilterA->Source". Multithreading in Avisynth+ is being implemented by letting multiple such (temporally different) chains be evaluated in parallel. I can't say yet how good it works though, as it needs more work I don't have results yet. Can you describe in more detail what advantage your scheduler would provide compared to a traditional threadpool?
2) That is not possible with current Avisynth. To make this work, all existing plugins would need to be modified (though rather trivially). Anyway, if you do that, you can throw out all exisitng plugins until they are changed, or you can use a wrapper layer which throws away the advantages of your system. Btw, VapourSynth does exactly lthis. It lets plugins specify precisely which frames they need for the current job without the thread having to wait for their availability. This is what allows VapourSynth to implement pipelining, which has its advantages, but also drawbacks IMHO.
3) This is not really a new thing. MT-mode is allowed to be provided by filter itself in both VapourSynth and Aisynth+ (I'll write about that soon in another post), and the modes are already established in all projects.
4) This is similar to what Avisynth+ will allow: to have several frames being processed in parallel, all potentially from different stages of the filterchain. And I think this is possible in VapourSynth too, though I don't know enough of it in this respect to be sure.
5) No cache and strict refcounting: The first is bad for performance, the second is not possible without breaking plugins. I'm in the middle of adapting the cache and memory management of Avisynth+ for multithreading right now actually. Out of curiosity I have made various tests and measurements along the way here, like seeing how the filter chain performs without caching. Absolutely miserably. Caching is so important, that without it performance drops to only a fraction of what it was. Strict refcounting (throwing out frames the moment their refcount drops to zero) is not posisble without breaking all 2.5 plugins, because they have the relevant piece of code baked in. So you can modify the refcounting code to drop frames when the count reaches zero, but that part will not execute in 2.5 plugins. The result is that you will leak every single video frame (possibly multiple times, once for each stage), unless you only use 2.6 plugins, otherwise you'll run out of memory in 100-200 frames using a 720p video. I have found that out the hard way, trying to debug said memory leak for more than a day. FYI, this is exactly why I need to rethink my concept of the new caches in Avs+, which leads to MT being delayed somewhat. A possible "solution" to this refcount problem is to recompile all plugins for the 2.6 interface. But there are a lot of plugins out there, and just because you publish recompiles does not mean the users will not keep using old 2.5 plugins. And we don't even have the source for all plugins (even though we do for the most). All in all, I'm not gonna pursue this, and will instead just implement a different architecture that will work with existing plugins.

[SEt]

1) Scheduler is "advanced" threadpool. Instead of throwing many threads with random caching on hardware and hoping it would work faster, scheduler do exactly the needed work.
2) Like I said, "wrapper" plugin can provide the required additional information, no problem for most filters. In worst case it'll do fail-rerequest route.
5) Cache is absolutely meaningless waste of space if you already can provide all required source frames for the filter call. General refcounting is done by scheduler based on 2, wrapper can do additional refs in response to 2.5 frame refs, but I guess it won't be used much.

[ultim]

Is my understanding correct that you'd let each single filter work on multiple frames in parallel in its isngle thread? And multiple of such superscalar filters would be executing on each HW thread.

[Myrsloik]

I doubt anyone it going to implement it, but as you requested, general outline:
...

Since there may be some interest in comparing approaches and ideas...

VapourSynth works kinda 80% like what you describe.

1. There's scheduling, all frame requests are basically put in a queue and processed in order using a thread pool. If a request can't be processed at the moment (the filter instance is already busy) then it simply looks at the next request in line.

2. VapourSynth filters basically work like this. On the first call they request all (or at least most) frames necessary. Then the filter gets called again when they are ready. The procedure can be repeated if necessary.

3. Exactly

4. You get kinda an implicit, cheap version of this in VapourSynth. It combines requests for the same frame (even before looking in caches) so in many cases it ends up working very well. Even temporal filters normally have a very low number of requests to the actual caches.

5. If filters can do unexpected things you need some kind of cache. Of course 80% of the ones automatically inserted will just be wasting their time doing nothing much at all. And remember that users can also do unexpected things such as seek around in vdub. I do however think avisynth wastes obscene amounts of ram for its caching. Frames have strict reference counting.


Avisynth compatibility works like you say, only difference is that the magic "prefetch list" is compiled in. The method of threading for avisynth filters is a single instance running in its own avisynth environment wrapper. So there are always as many "avisynth environments" as there are filters. In combination with the prefetching each filter instance can at most fully use one cpu core (which usually turns out to be enough since stuff like mvtools is a gazillion medium complexity filters stuck together).

[SEt]

ultim
No, superscalar is scheduler. Its execution pipelines are hardware threads, executed "instructions" are filter instances (according to their threading capability) that work on "registers" – frames.
The key points are breaking the call chain and refcounting instead of cache.

Myrsloik
I'm not interested in VapourSynth.

[TurboPascal7]

So in the end it all comes to the "we need to modify all existing plugins" problem. "Nice".
I guess we'll have to set up a huge repo for all the (useful) plugins ever created and start making them less bad some time soon. I just really want to hope I won't be the only one working on this.

[ultim]

So in the end it all comes to the "we need to modify all existing plugins" problem. "Nice".
I guess we'll have to set up a huge repo for all the (useful) plugins ever created and start making them less bad some time soon. I just really want to hope I won't be the only one working on this.

Well this wouldn't be the first thing we could use such a repository for. Making plugins MT-compatible, recompiling for 64-bit, and checking ASM for calling convention violations would also all require a plugin source repository anyway. Not to mention, it would allow us to get rid of plugins with baked-in code, legacy code in Avisynth's core could be removed, and I could probably also implement a more efficient (and less leaky?) caching.

So there's a real bunch of reasons for setting up a repo for plugins, but there are some basic questions to be answered:

How would plugin authors react? If they are not interested in commiting to the repos in the future, will we have to port every single change they make?
How do we assign plugin maintainership? Obviously we must give authors "VIP" access, and divide the rest of the work between us.
What's the destiny of closed-source plugins?
Who decides which plugins are "useful"? "Useful" depends on who you ask, and what the plugin is used for. And just because some use cases are rare, it doesn't mean that the corresponding plugins aren't useful.
Is it realistic to work on most of the plugins and make all the changes? The changes necessary per plugin are small (unless you have to rewrite asm), but multiplied by the number of plugins...
How do we make sure that a user doesn't use legacy plugins mistakenly after that in Avisynth?
What happens if we fail for whatever reason? Can we depend on plugins being updated in a reasonable amount of time? Should development directions depending on the outcome be halted or not?
Do we have the manpower/time/resolve?


IMHO these are important questions that we should try to answer appropriately before doing anything rash. On the other hand, if we do can handle these question correctly, I'm all for it.

[TurboPascal7]

Wow, stop ruining my dreams so fast...

How would plugin authors react? If they are not interested in commiting to the repos in the future, will we have to port every single change they make?
How do we assign plugin maintainership? Obviously we must give authors "VIP" access, and divide the rest of the work between us.

We do not touch actively maintained plugins. If the author wants to continue using zip packages attached to doom9 posts as his version control system - it's "fine". Our problem is making him "want" to update his plugin for better integration with avs+. The only way we could do it is making sure any updates do not break compatibility with other versions of avisynth. For example asking the author to provide another registration function like AvisynthPluginInit is okay. Asking him to use internal avs+ threadpool is not unless he's really fond of the idea or prepared to maintain a few versions of the plugin.
What's the destiny of closed-source plugins?
Are there many useful of those? I never actually bothered to check what's closed and what's not.
But well, reverse-engineering is pretty much the only thing we could do about these. Not that you don't have to do the same with some open "source" plugins... Of course anything large and complicated is out of questions.

Who decides which plugins are "useful"? "Useful" depends on who you ask, and what the plugin is used for. And just because some use cases are rare, it doesn't mean that the corresponding plugins aren't useful.

What users request. I have maybe 15-20 I use the most, other people use other etc. Yes we'll probably end up with basically all plugins ever created. Can't be helped.

Is it realistic to work on most of the plugins and make all the changes? The changes necessary per plugin are small, but multiplied by the number of plugins...
Simple stuff like recompilation and maybe updating to a newer version of avisynth interface - yes, sure. Rewriting the asm, handling multithreading and making stuff cross-platform - basically impossible unless you find a large dedicated team for that. Hint: you won't.

How do we make sure that a user doesn't use legacy plugins mistakenly after that in Avisynth?

We can't and we don't. Centralizing plugin distribution is pretty much the only thing we can do, but even then no one can stops the user from installing broken plugins from other sources.

What happens if we fail for whatever reason? Can we depend on plugins being updated in a reasonable amount of time? Should development directions depending on the outcome be halted or not?
No, most plugins will never be updated or ported, especially in reasonable amount of time. No, development directions should not depend on this and we should find a way to make things work with what we get right now, at the same time trying to make the most use of plugins that do get updated. Yes it won't be efficient "for now" but it's the only choice we have. I might be able to update some useful plugins which I already worked with and a few more, but I'm a human too.

What we can and should do is making the most popular plugins used in literally 100% of scrips better, so most users get the most benefit from our work. But we cannot save the whole avisynth ecosystem.

[SEt]

So in the end it all comes to the "we need to modify all existing plugins" problem. "Nice".
Of course interfaces must be changed to support new things. It's absolutely normal considering the state of Avisynth interface today.
Binary compatibility with old 2.5 plugins would still be maintained – wrapper is exactly the thing for that. It would cost some efficiency and will require handwriting threading information, but nothing like recompilation required.

Even with other design plans I still suggest the "wrapper" idea: separate from the core plugin that provides compatibility layer. No need to integrate legacy into the new core.