Avisynth+

[ultim]

Just fixed in current git.

The $InternalFunctions$ and $PluginFunctions$ global vars are gone since the plugin management rewrite. Was that by mistake?

Ah cool, I just saw your post after posting my own. Would you be okay with modifications to AvsPmod? That means you might need a small path to recognize you are using Avisynth+, but I think it might even be posisble to find a unified way that works with both Avisynth and Avisynth+. I'll give some thougth to it if you are interested. Ofc I'll make the necessary modifications too.

[ultim]

Regarding porting to x64:

Taking code from Avxsynth and Avisynth64 could indeed be very helpful, but one must be careful becuase these have been forked long ago and are not up to date with Avisynth 2.6. Meaning the funcationality and semnatics of the filters might have diverged.

x64 VC++ does not support inline asm at all, which means handoptimized pieces of code will either need to be compiled separately using an external assembler, or they need to be converted to intrinsics. IMHO, intrinsics are way better. First of all many/most intel-style intrinsics are portable between compilers (even between GCC, and MSVC), and second, having to compile external .asm files using a dedicated assembler would complicate the build process a lot. Not to mention CMake ATM doesn't even have support for that when generating VS project files. So, TurboPascal7, if you con really help porting to intrinsics, that would be more than great.

Porting to x64 could even be made step-by-step. Most ASM resides in builtin filters, so as long as the core without filters can be compiled in x64, we can be adding more and more builtin filters back to an already working x64 version as the porting continues.

As a separate note, I see a potential problem of maintaining a unified public avisynth.h header without breaking 32-bit plugins. We could ofc always maintain two totally separate versions of that header, but if possible we should try to avoid that.

[TurboPascal7]

There are some problems with intrinsics though:
1) I'm not sure how well they are supported in vs2005 and 2008. While 2008 is probably okay, 2005 might have some serious issues. I haven't worked with it for years and didn't do any C++ back then so I'm not sure.
2) VC++ doesn't support MMX intrinsics in x64 mode. So one either #ifdefs every MMX routine or drops MMX optimization altogether (I do approve dropping it but I'm not sure how many avisynth users don't even have a P4 these days).

[jpsdr]

If ASM reside most in built-in filter and not in the core, maybe what's allready done with avisynth64 could be used, assuming that differences with actual version are more in the core than filters...

[ultim]

1) I'm not sure how well they are supported in vs2005 and 2008. While 2008 is probably okay, 2005 might have some serious issues. I haven't worked with it for years and didn't do any C++ back then so I'm not sure.
Well, I was supporting VS2005 because there was no need to drop it. If it does not have proper support for the intrinsics we need (thereby stepping in the way of 64bit support), that'd be enough reason for me to drop it. But by looking at this (http://msdn.microsoft.com/en-us/library/kcwz153a%28v=vs.80%29.aspx) page, VC++ 2005 does seem to have intrinsics support for SSE2. And if not, I'm okay to drop it.


2) VC++ doesn't support MMX intrinsics in x64 mode. So one either #ifdefs every MMX routine or drops MMX optimization altogether (I do approve dropping it but I'm not sure how many avisynth users don't even have a P4 these days).
We could #ifdef around those parts for 32-bit users on old machines. In the x64-builds, that wouldn't matter anyway coz the presence of SSE2 is guaranteed.

I think I need to add a dynamic dispatch ability to Avisynth+ that filters and plugins can make use of ;)

[TurboPascal7]

One more important question you are aware of is alignment. Did you decide to keep it the way avisynth works now or provide any reasonable guarantee?

I'm not sure that you need to add anything on the core level to help dispatching. Providing a method to get CPU flags (the way it is now) is enough to make filters implementation as simple as possible. I'm not sure how you could simplify it even more.

[aegisofrime]

I'm not a developer so I can't offer any help, just a normal user, but I just want to say thank you to ultim for initiating this project, as well as other contributors. This is probably the most exciting thing on the Avisynth in my few years of video processing. :thanks:

[qyot27]

We could #ifdef around those parts for 32-bit users on old machines. In the x64-builds, that wouldn't matter anyway coz the presence of SSE2 is guaranteed.
I'm definitely in favor of ifdeffing around it. My main setup is a Pentium III-era Celeron, and this was a point of contention that arose in VapourSynth (http://forum.doom9.org/showthread.php?p=1601345#post1601345). The stated solution: use AviSynth if you've got an ancient CPU (http://www.vapoursynth.com/2012/11/minimum-requirements/).

The alternative would be to just disable asm completely on non-SSE2 CPUs, but if there's benefits to be had from MMX and SSE I wouldn't throw it out completely.

[ultim]

I'm definitely in favor of ifdeffing around it. My main setup is a Pentium III-era Celeron, and this was a point of contention that arose in VapourSynth (http://forum.doom9.org/showthread.php?p=1601345#post1601345). The stated solution: use AviSynth if you've got an ancient CPU (http://www.vapoursynth.com/2012/11/minimum-requirements/).

The alternative would be to just disable asm completely on non-SSE2 CPUs, but if there's benefits to be had from MMX and SSE I wouldn't throw it out completely.

I do agree with Myrsloik that programming for Pentium3 is not worth it nowadays, but I see no harm in keeping existing code there for it. All we need to do is put a define around it so that the compiler ignores it in 64-bit builds, while it can still build in 32-bit versions. This works at least for "stable" filters. Others could have a problem, that when they are updated, the filter maintainer would have to update the MMX routines too, and there probably won't be too much motivation for that.

[ultim]

One more important question you are aware of is alignment. Did you decide to keep it the way avisynth works now or provide any reasonable guarantee?
Some changes that I proposed for inclusion in my very first changeset include a refactor of alignment-related functionality. Let me try to make a very-very cautious statement, that I think a minimum alignment guarantee of 32 bytes should be there, unless a script uses unaligned crop.

Getting rid of unaligned crops would undoubtably strengthen this semi-guarantee, but I also must add that there might be other places I have overlooked. Should you find any violation of this alignment, I'll get rid of it for sure as soon as I get to know of it.

[ARDA]

x64 support is also a goal, but certianly not for the near-future. As many of you have said,
the greatest obstacle here are the pieces of inline asm. I must admit that I'm not very versed
in x86 asm, so here too I will be counting on your help.


x64 VC++ does not support inline asm at all, which means handoptimized pieces of code will either
need to be compiled separately using an external assembler, or they need to be converted to intrinsics.
IMHO, intrinsics are way better. First of all many/most intel-style intrinsics are portable between compilers
(even between GCC, and MSVC), and second, having to compile external .asm files using a dedicated assembler
would complicate the build process a lot. Not to mention CMake ATM doesn't even have support for that when
generating VS project files. So, TurboPascal7, if you con really help porting to intrinsics,
that would be more than great.


Porting to x64 could even be made step-by-step. Most ASM resides in builtin filters, so as long as the core
without filters can be compiled in x64, we can be adding more and more builtin filters back to an already
working x64 version as the porting continues.


VC++ doesn't support MMX intrinsics in x64 mode

Some fast thoughts not ordered by priorities;

To compile assembly code with an external assembler is not so hard, at least according to my experience
Nasm/Yasm syntax is very near inline asm, except some pointers syntax, not too much.
They are also portable between enviroments and platforms if I am not wrong. Well, I work almost always
with Microsoft Visual Studio C++ 2008 and Yasm or Nasm.

The idea of separate many filters out of core to port them to x64 is a good one, and for 32
bit version as well. I always dreamt with a core very light, the lightest possible, and different
projects of filters grouped by their functions, tweak colors, editing, resizing, etc.
It could help to bring more developers with some specific knowledge that don't dare to manage the whole
project, besides that it could make easier to track bugs in the core or in a specifig group of filters.
Many years ago I posted something similar, but it seems there is a limit number of dll that can be loaded.
http://forum.doom9.org/showthread.php?p=698052#post698052

In short if possible I wouldn't include them back to core.

I also think there is a lot work previous port to x64, and I agree is not for near future.
And in that future probably mmx or Isse will not be needed anymore, while 32 bits it doesn't hurt
to mantain mmx and mainly Isse cause the profit of non temporal store instructions.

[qyot27]

I don't think it's worth focusing on computers as old as mine either. It's more that I really couldn't see why a simple --disable-asm option wouldn't have sufficed if running the asm on a non-SSE2 CPU would cause problems. I just couldn't grasp how whether the core could use X certain SIMD set would matter to a plugin author, because how I understand the way this all works, the plugin would simply use the desired core function at whatever level of optimization was there. If it was boosted by SSE2 or whatever, great, if not, then it would fall back to the plain C or C++ version and if the user's on a computer old enough for that to be true, then they already know how slow their computer is, so I still don't see the problem in merely letting them run it in an unoptimized state.

[Groucho2004]

I always dreamt with a core very light, the lightest possible, and different projects of filters grouped by their functions, tweak colors, editing, resizing, etc.
This would be the ideal scenario in my book as well but I don't know how much work would be involved to dig out the core and then add these - let's call them native - filters.

[TheFluff]

Well, I was supporting VS2005 because there was no need to drop it. If it does not have proper support for the intrinsics we need (thereby stepping in the way of 64bit support), that'd be enough reason for me to drop it. But by looking at this (http://msdn.microsoft.com/en-us/library/kcwz153a%28v=vs.80%29.aspx) page, VC++ 2005 does seem to have intrinsics support for SSE2. And if not, I'm okay to drop it.

VC++2005 supports it but the code generation apparently has some minor issues, and MMX is worse off than SSE. Avery Lee has written about intrinsics in VC++ at some length, see for example:
http://virtualdub.org/blog/pivot/entry.php?id=46
http://virtualdub.org/blog/pivot/entry.php?id=301
http://virtualdub.org/blog/pivot/entry.php?id=340

[TurboPascal7]

If it was boosted by SSE2 or whatever, great, if not, then it would fall back to the plain C or C++ version and if the user's on a computer old enough for that to be true, then they already know how slow their computer is, so I still don't see the problem in merely letting them run it in an unoptimized state.
But this requires filters/plugins to have this unoptimized plain C version, which might not always be the case (take a look a sangnom2 for example, which doesn't have a plain C version. Original sangnom didn't either). I dunno about you, but I don't really see any point in adding that only to support some 15-years old CPUs (readability problem aside). This is probably not the problem for internal filters though, since most (all?) of them do have the C path.

Guarantees is a good thing and IMHO requiring a newer-than-10-years-old CPU if this makes plugin authors' lifes easier is reasonable.

This would be the ideal scenario in my book as well but I don't know how much work would be involved to dig out the core and then add these - let's call them native - filters.
The core filters are separated quite well and moving them to an external dll wouldn't be hard.

That said, I can't say I fully support the idea. I believe the core itself must provide basic video operations the way it does now and I don't see how separating them would help anything. Especially when compilation of the core is dead simple and fast now.

Some changes that I proposed for inclusion in my very first changeset include a refactor of alignment-related functionality. Let me try to make a very-very cautious statement, that I think a minimum alignment guarantee of 32 bytes should be there, unless a script uses unaligned crop.

Getting rid of unaligned crops would undoubtably strengthen this semi-guarantee, but I also must add that there might be other places I have overlooked. Should you find any violation of this alignment, I'll get rid of it for sure as soon as I get to know of it.
Here we come to the problem I originally tried to explain to IanB (and failed) - only strong guarantees are good for developers.
Take the core filters for example. Right now, most of them are optimized for MMX only, which does not require any type of alignment the way SSE2 do. While porting to x64 and SSE2 (which is the only option if we use intrinsics), one will need to take alignment into account. And without any strong guarantee, he would essentially need to either
1) Template almost every SSE2-optimized routine ever to use unaligned or aligned loads based on provided data. This becomes even worse if we use external asm because simple templates are much easier than any kind of asm macros (https://github.com/tp7/masktools/blob/6723f85c777e12a0877c5ee5fb0b513320f9c13b/masktools/filters/logic/functions.asm).
2) Always use unaligned loads which makes every single SSE2-optimized filter slower only to provide faster crop and possibly faster source filters which I'm not aware of.

That said, I agree we shouldn't make rush decisions. This is indeed a hard question that might (and probably will) break some older filters. I guess I'll do a build with alignment parameter to NewVideoFrame ignored and see how much breaks later.

----
Also, setting up an IRC channel on freenode/rizon or something to speed up the communication process would be nice, I guess (there's already #avisynth on freenode btw).

[ultim]

Separating *all* the filters out of core is not my plan either, actually. I already made external plugins out of many functions that IMHO do not belong into the core, like DirectShowSource, audio timestrething, support for vdub filters etc, and maybe I'll do the same with a few more, but I think too that the core library should be usable on its own too. When I spoke about a step-by-step porting process with a growing set of x64 filters, I was thinking of #ifdef-ing around filter code and gradually enabling them in builds as porting advances.

I *do* support moving seldomly used filters out of the core, so reviewing what belongs there and what not could go onto the TODO list, but I too don't think that the core sould be totally bare and empty of filters. Moving them out, as TurboPascal7 mentioned, is not hard. It is actually so easy that it could be an introductory exercise for anybody wanting to contribute.

ARDA mentioned a DLL loading limit in Avisynth: in the most recent version of the non-plus Avisynth, it is 50. This limit has already been removed from Avisynth+. It was artificial and not-needed in the first place, and it probably got introduced because people were compiling plugins with a static CRT, in which case Windows imposes a limit of 128 loaded DLLs (at least that's the observable effect, the true reason lies elsewhere, but that's a different story). Anyway, my point is, as long as our plugins are linked against a dynamic CRT, having many DLLs should not pose a problem.

[ultim]

Here we come to the problem I originally tried to explain to IanB (and failed) - only strong guarantees are good for developers
I understand and agree with you fully. The reason I didn't give a strong guarantee is not because I don't want to introduce one, it is because I'm not sure that it is currently fully fulfilled. I also want a proper alignment guarantee to be there, which is why I'm going to correct any left-over pieces of code that violate this, if any is still found. I think avoiding unaligned crops brings pretty far in Avisynth+, and definetely further than plain Avisynth.


Also, setting up an IRC channel on freenode/rizon or something to speed up the communication process would be nice, I guess (there's already #avisynth on freenode btw).
I gues we could. Actually I am regularly on rizon anyway (and when not, you'll find my bouncer), but because there are more people accustomed to the forums than to IRC, I find a forum a better place for discussions. If we used IRC, many wouldn't be able to follow.

[ultim]

On the intrinsics vs. asm topic, I will only say that intrinsics are more favorable to me personally, but I'll certainly accept contributed code written in asm too. I won't reject the integration of asm files just because they are not using intrinsics, as long as the contributor also provides a working integration into the current build system. So I say if the contributor is indifferent about using intrinsics or asm, please use intrinsics, but your contribution will be greatly appreciated otherwise too.

Intrinsics integrate more naturally into C compilers and into the existing build system. But maybe most importantly, pure intrinsics can be uniformly compiled into both 32bit and 64bit code, so they win by large in maintainability too. Many prefer asm because they find it easier to read, that is also important, but it is subjective, so I'll leave the choice up to you.

True, intrinsics do not always generate optimal code, but it will be close, and it will still be much better than traditional C code. And let's be honest, hand-written ASM is not guaranteed to be optimal either. Not to mention, code generation from intrinsics improves with better compilers.

But anyway, if you don't want to use intrinsics, that's fine by me too.

[ultim]

What kind of for-construct would you like to see?
Gavino's "for" looks like
for ( variable = init , limit , step ) {
statements
}
where limit and step must evaluate to integers and are only evaluated once. There is also some built-in trickery whether the loop is up- or down-counting (see original thread (http://forum.doom9.org/showthread.php?t=147846) for reference).

The alternative would be a C-style loop, common not only to C but most other computer languages.
for ( statement1 , condition , statement2 ) {
statements
}
which strictly speaking is just a short form for

statement1
while ( condition ) {
statements
statement2
}


I can see upsides and downsides too to Gavino's version. It might be easier to grasp for non-programmers, and also it gives Avisynth some possibilities to recognize infinite-loops, and helps prevent the user from shooting himself in the foot. C-style for-loops could then be emulated using manually written while-loops. On the other hand, it is less generic, non-standard, and despite all the simplifications and built-in automatic, it still cannot prevent all user errors, like when the loop variable is modified inside the body. Programmers could also argue that it is counter-intuitive, because 'limit' and 'step' are only evaluated once even if their value is dynamic.

So, my question is simple: Do you want Gavino-style for-loops or C-style for-loops in Avisynth+?

[mirkosp]

It's not like they are needed, since they can already be done through recursion if one needs to.
Let's not forget that AviSynth does NOT exist as a normal programming language, but more as a NLE tool; I doubt this nature should change with a fork. As such, common programming structures don't really need to be defined.
If one really needs to employ many loops for a script, it's likely because they need to write something complex, and it would be better to straight out write that in a proper language and build a .dll for use.

[wOxxOm]

What kind of for-construct would you like to see?The simpler the better. I like Gavino's with even less 'noise': for var = init, limit, step { .... } #step could be negative.

It's not like they are needed, since they can already be done through recursionRecursion is not human readable. Making dlls isn't worth the effort in simple cases. Using vapoursynth isn't always an option. And there's nothing wrong to have a small set of simple human readable block statements in avisynth-plus. Also, this topic was discussed like a thousand times.

[Sapo84]

It's not like they are needed, since they can already be done through recursion if one needs to.
Which is harder to write, read and probably less efficient (not that it matters).

Anyway, I'd go with the C style for (and no while, for already covers it).
I find the readability of Gavino's one a bit of a problem (if you don't use named parameters), and while his solution is a lot more in line with Avisynth synthax I still prefer having full statements.

[jmartinr]

+1 for C-style

[Gavino]

I can see upsides and downsides too to Gavino's version. It might be easier to grasp for non-programmers, and also it gives Avisynth some possibilities to recognize infinite-loops, and helps prevent the user from shooting himself in the foot. C-style for-loops could then be emulated using manually written while-loops. On the other hand, it is less generic, non-standard, and despite all the simplifications and built-in automatic, it still cannot prevent all user errors, like when the loop variable is modified inside the body. Programmers could also argue that it is counter-intuitive, because 'limit' and 'step' are only evaluated once even if their value is dynamic.
My goal with for was to provide a simple looping construct without the need to use recursion, making life easier for script writers, especially for non-programmers. The syntax is simple so that errors can be quickly recognised and reported clearly.

I made limit and step fixed as I felt they are the parameters that define the loop.
Anything more complex can be programmed as a while loop.

[TurboPascal7]

I'm not sure for loop is required either, maybe because I haven't seen a single useful example of it used in gscript, that couldn't be rewritten with recursion with maybe a bit more effort. If anyone could provide such an example (preferably used in a real working commonly-used script), I would be grateful.

That said, if you feel like adding features to the language just to simplify a little more than zero scripts - sure, go ahead. It's not like it will make the language worse. I don't have any particular opinion on which version is better, they are both fine.

and probably less efficient (not that it matters).
Efficiency should be pretty much identical because after building the filter graph it's all the same.

[Gavino]

I'm not sure for loop is required either, maybe because I haven't seen a single useful example of it used in gscript, that couldn't be rewritten with recursion with maybe a bit more effort.
Clearly it's not required, as any for loop can be rewritten using recursion.
Its purpose is to avoid that 'bit more effort' for the majority of script writers.

[TurboPascal7]

Gavino, I understand and I would fully agree with you in an ideal situation. I just think it would be better to check if the actual demand is high enough (with examples) rather than simply adding features that "would be cool to have". It's never late to add the feature but it will be really hard to remove it. Just look at C++ right now.

[ultim]

I myself find the new block-if the most useful, but the looping constructs are useful too. Please don't turn this discussion into whether to have looping constructs at all or not. The fact that previously loops have been emulated using recursion proves that loops are needed and sought after in Avisynth. Especially in a language that is designed to be simple to write and simple to interpret by humans, "for" or "while" can mean a lot, because they are way easier to program and read than even the simplest form of recursion.

So there will be ifs, fors and whiles (and admittedly I'll also see if I can hack a "break" statement into the language). If you are more comfortable with recursion, you can still keep using it. Now please, let's get back to the question at hand and let's vote which "for" variant you'd prefer.

Edit: I understand that none of these new constructs are technically needed, as the same effect can be achieved using other methods. However, those methods make scripts harder to write and to read. These new constructs are improvements, and improving the existing functionality of Avisynth is the goal of Avisynth+ after all.

[qyot27]

But this requires filters/plugins to have this unoptimized plain C version, which might not always be the case (take a look a sangnom2 for example, which doesn't have a plain C version. Original sangnom didn't either). I dunno about you, but I don't really see any point in adding that only to support some 15-years old CPUs (readability problem aside). This is probably not the problem for internal filters though, since most (all?) of them do have the C path.
I don't see how it requires plugins to have it. It just means the user is SOL if a plugin doesn't have a fallback, same as they are now if they try to run a plugin with incompatible SIMD on their machine. But I don't see how the core running unoptimized has an effect on what the plugins support.

I'm looking at it more from the perspective of growth toward other machine architectures - having an unoptimized version maximizes portability. Sure, there could be wholesale additions for NEON or AltiVec or SHmedia or whatever else, but that'd still have to be a case-by-case basis.

[kypec]

ultim, you should probably create separate thread with poll about "preferred loop constructs" then as to move it outside this topic. BTW, I'd prefer C-style syntax myself...

[ARDA]

What kind of for-construct would you like to see?
Gavino's "for" looks like


So there will be ifs, fors and whiles (and admittedly I'll also see if I can hack a "break" statement into
the language). If you are more comfortable with recursion, you can still keep using it.

Related with Gavino-style for-loops or C-style for-loops I have no preferences, but obviously the
easier for script writers the better, to make more human readable avisynth language is something to add
to a TODO list, avisynth is far to be intuitive or friendly for normal user. Gavino's work is in that
direction, nor more nor less.(My opinion, never interchanged views with him)
I see you have already decided to conform everybody including both and something else, luckly you
have the energy and skills for everything. Thanks again.



Porting to x64 could even be made step-by-step. Most ASM resides in builtin filters, so as long as the core
without filters can be compiled in x64, we can be adding more and more builtin filters back to an already
working x64 version as the porting continues.


Separating *all* the filters out of core is not my plan either, actually.
but I too don't think that the core sould be totally bare and empty of filters.


I always dreamt with a core very light, the lightest possible, and different
projects of filters grouped by their functions.......

I do not see the word empty or *all*, maybe is my poor english, but I remarked the lightest possible,
that I don't define with precision, so I left opened to a research, study or discussion.


To compile assembly code with an external assembler is not so hard, at least according to my experience
Nasm/Yasm syntax is very near inline asm, except some pointers syntax, not too much.
They are also portable between enviroments and platforms if I am not wrong. Well, I work almost always
with Microsoft Visual Studio C++ 2008 and Yasm or Nasm.


True, intrinsics do not always generate optimal code, but it will be close, and it will still be much better than
traditional C code. And let's be honest, hand-written ASM is not guaranteed to be optimal either. Not to mention,
code generation from intrinsics improves with better compilers.

I do not say asm is better than intrinsics, just remark it is not so hard to move inline asm to Nasm/Yasm.
And is also portable, probably to more platforms.
Obviously you have to work under and enviroment that can use Nasm.
Asm code can be non optimal of course. I was thinking about the task of moving out inline code you already have
in built in filters nowadays, optimal or not, while preserving many mmx and Isse code.
There are some Isse codes that are well optimized in avisynth and it is very difficult to get a SSE2 or higer version
to work better, mainly when you have to work with misaligned ptrs.


I also think there is a lot work previous port to x64, and I agree is not for near future.
And in that future probably mmx or Isse will not be needed anymore, while 32 bits it doesn't hurt
to mantain mmx and mainly Isse cause the profit of non temporal store instructions.

@ultim
All these are just some quick ideas, I see you are working hard with other issues, more immediate needs
and improvements so we can leave all this discussion for future.


Thanks for all the work you are doing, and keep on.

[ultim]

Hu, I can rename the thread :) Btw, Wilbert, I never thanked you for moving us out of the Avisynth a4 thread. So sorry for the trouble and thank you.

[TheFluff]

Separating *all* the filters out of core is not my plan either, actually. I already made external plugins out of many functions that IMHO do not belong into the core, like DirectShowSource, audio timestrething, support for vdub filters etc, and maybe I'll do the same with a few more, but I think too that the core library should be usable on its own too.

There are a bunch of "core" filters that should be removed and some that should be restructured, because the current selection is a horrible mess of old cruft and arbitrary ad-hoc "design". I'll quote myself:

SelectEven/SelectOdd are special cases of SelectEvery and are not going to be implemented (you can of course "implement" them trivially in Python if you want). Similarly, ApplyRange is a special case of Trim (plus splicing) and can be trivially implemented in Python as well. Vapoursynth will not have an 1:1 copy of all Avisynth functions, especially not since the scripting language is now a real programming language so implementing interesting script functions is mostly trivial.

Avisynth has a ton of old crufty functions that are in the "core" for very unclear reasons, anyway. Why are there five different level manipulation functions (coloryuv, levels, limiter, rgbadjust, tweak) with a ton of overlapping functionality? Why are things like PeculiarBlend, FixBrokenChromaUpsampling and FixLuminance core functions? Why does CropBottom exist? Why does the basic "fade" function (equivalent to dissolve+blankclip) need nine different function names and prototypes? God (or maybe BenRG) alone knows.

I mean, check the documentation for FixLuminance (http://avisynth.nl/index.php/FixLuminance). Why the hell has this been in the core for over a decade?

Other redundant functions that should be removed:
ShowFiveVersions - exactly equivalent to StackHorizontal with five arguments. Why?
MonoToStereo - has been explicitly deprecated since 2.5 and replaced by MergeChannels
Pulldown - it's SelectEvery with the cycle forced to 5, use that instead
SwapFields - the documentation explicitly says it's just an alias for SeparateFields().ComplementParity().Weave()

[Myrsloik]

I have one comment to make about the inline assembler/softwire/mess that is in avisynth. Most of it is of a horribly low quality.

Replacing it with proper C++ code probably wouldn't slow it down that much. You also need a proper C++ code path just to test that the assembler you wrote is correct anyway. But that's something no other posters seem to take into consideration.

Anyway, interesting to see some kind of development going on.

[jpsdr]

Being a big user of pulldown, i personnaly prefer to keep it. Some are like macro in C. When something can be used very often, it's nice to have a more compact syntax. I prefer writing PullDown(0,2) than SelectEvery(5,0,2).AssumeFrameBased.
Even if i don't use it, showfiveversion has the same purpose. They are like macro in C, made to simplify syntax of something which may be used very often.
Seriously, i prefer SwapField, name is explicit, syntax is simple, than SeparateFields().ComplementParity().Weave() !
Think of someone who simply want to swap fields... Good luck guessing SeparateFields().ComplementParity().Weave() when you don't have the "swapfield" function !!
Same for pulldown.
Personnaly, i think these shouldn't be removed.

[TurboPascal7]

jpsdr
All that can be solved by shipping a single avsi script with the core, containing all the removed functions.

[ultim]

All that can be solved by shipping a single avsi script with the core, containing all the removed functions.
I'm not sure putting them into .avsi scripts is how much better then being included in the core native filters. Even if in the core, we can cleanly separate them out into their own filters/shortcuts.cpp file (or whatever the name) to not pollute the main filters. But putting them into a separate script incurs a needless startup cost (the'd be autoloaded anyway, but now they'd need to be separately loaded, parsed and evaluated).

[TurboPascal7]

ultim
I'm not saying they should be definitely separated, just stating that the issue can be solved without any real affect on the user.
I think both solutions are fine: the cost of loading a single avsi is not that high, but having them in a separate cpp file would not be bad either, especially since most of them can be implemented in one-two lines.

[TurboPascal7]

While we're at the question of removing stuff: is there any reason TemporalSoften's mode 1 still exists?
Documentation says mode 2 "has a new and better way of blending frame and provides better quality. It is also much faster." and it looks to me almost everyone is using it anyway, so why keeping it at all?

[ultim]

Avisynth Plugin Writing Tips #1: Exceptions

Exceptions thrown from a module should only be caught in the same module. Otherwise you can experience weird and hard-to-debug errors in the plugin. Not adhering to this advice will result in code that can sporadically fail, or work on your computer consistently but fail on other machines.

Unfortunately, avisynth.h contains the AvisynthError class, giving plugin authors the false impression that it is safe to throw and catch these exception objects. It is not. The problem is not in the definition of this class, but in the implicit encouragement to throw C++ exceptions across DLL boundaries. Here are some tips to avoid getting caught in the deepest pits of hell:
- When throwing exceptions on your own, it is best not to use AvisynthError. Not using it will stop you thinking that AvisynthError has some special meaning, or that it can be used to throw to (or to catch from) avisynth.dll.
- Exceptions thrown by you should always be caught inside your plugin. You should not let exceptions propagate outside of your DLL (unless thrown using ThrowError), to Avisynth.
- Errors thrown by Avisynth should not be caught by you. In specific, don't wrap calls to Avisynth in try-catch blocks, because you cannot rely on it working correctly in every situation. If you need to detect errors, validate user parameters in your plugin, or use other API facilities provided by Avisynth, like IScriptEnvironment->FunctionExists().
- If you want to throw an exception to the user and/or to Avisynth, then only use IScriptEnvironment->ThrowError(). You should not call C++'s "throw" yourself for this purpose (see 2. point), and you should not catch the error thrown by ThrowError() yourself (see 3. point).
- If you want to catch an exception, want to do something based on that and finally raise an exception to Avisynth, don't rethrow. Catch your own exception (unless thrown by ThrowError), then call ThrowError separately.

Ignoring the above tips can still result in a fully working binary, but that is only guaranteed under very specific circumstances, more specifically when you've compiled your plugin with the *exact* same compiler version as the avisynth.dll was compiled with, AND when linking to the CRT runtime dynamically. Given that plugin authors can use whatever compiler they want, and that an avisynth binary can be supplied by any community member, it is unwise to rely on such detail.

These tips apply to all Avisynth versions (e.g. to 2.5 and to 2.6, to "classic" Avisynth and to Avisynth+, etc).

[ultim]

qyot27: I looked at the avisynth_c.h you mentioned that has support for the new colorspaces in 2.6. What sucks is that it is neither source- nor binary-compatible with the avisynth_c.h header from upstream avisynth. So I have to the choice of 1) either keeping the original, or 2) merging features from ffms2 but breaking C-compatibility to upstream.

As a sad note, the C-interface has a bunch of baked code. What surprises me is that Avisynth now has three different APIs (2.5, 2.6, and C), and all of them suck. I honestly feel tempted to finally create an API for Avisynth that 1) doesn't rely on the representation of internal data structures, 2) doesn't rely on baked code, and 3) can be used universally from many different compilers and computer languages (right now whichever existing interface you choose, you can pick at most only one of these properties). I'm just afraid that XKCD #927 (http://xkcd.com/927/) will set in.

[Groucho2004]

I'm just afraid that XKCD #927 (http://xkcd.com/927/) will set in.
Or this (http://en.wikipedia.org/wiki/Survival_of_the_fittest). :D:D:D:D

[qyot27]

qyot27: I looked at the avisynth_c.h you mentioned that has support for the new colorspaces in 2.6. What sucks is that it is neither source- nor binary-compatible with the avisynth_c.h header from upstream avisynth. So I have to the choice of 1) either keeping the original, or 2) merging features from ffms2 but breaking C-compatibility to upstream.
As it was a partial merge of 2.6 features into it, I'm not surprised (compatibility with the upstream's 2.5-centric C interface was relegated to a secondary header that defined old versions of avs_get_row_size_p and avs_get_height_p). The intent was probably for the upstream to catch up to these features whenever 2.6 finalized, or something.

Having a new API is likely going to be needed regardless, since the introduction of really-new or novel features would probably break the existing one anyway. Might as well develop the new features against an API that's cleaner and easier to maintain, IMO. It'd also be an opportunity to document it better, if all the complaints I've heard over the years about how poorly documented the existing one is is any indication (noble efforts to rectify this notwithstanding (http://forum.doom9.org/showthread.php?t=162565)).

[Wilbert]

qyot27: I looked at the avisynth_c.h you mentioned that has support for the new colorspaces in 2.6. What sucks is that it is neither source- nor binary-compatible with the avisynth_c.h header from upstream avisynth.
Why don't you create one for upstream avisynth with the baked code removed? I'm sure IanB is interested in it.

[ultim]

Why don't you create one for upstream avisynth with the baked code removed? I'm sure IanB is interested in it.

The largest benefit of that would be so that compatibility to existing C plugins can be kept, like it was done with the 2.6 interface. But that wouldn't solve all the problems. The C interface has multiple structures which were layed out exactly as their C++ counterparts, and they in the end represent core structures, and both Avisynth as well as plugins can depend on their layout. This means it is impossible to modify these types without the possibility of breaking past and future plugins. So even if I removed the baked code, all we'd gain is the ability to modify method definitions. Which is better than nothing I suppose, but I refuse to build a 4th! interface that still has some of the same fundamental problems as the previous three.

That being said, I'm not working on any completely new interface right now. If at all, building a new interface will come later when there is a definite need for it for a feature that is being worked on. But that doesn't mean we cannot think about it until then.

For now, all the new methods that I'm adding to the ScriptEnvironment class are ending up in an "IScriptEnvironment2" abstract class. This has the benefit of never conflicting (layout-wise) with methods that IanB might be inserting into IScriptEnvironment, and it also keeps full compatibilty with existing plugins. Basically, it gives me a nice playground without any risk of breaking plugins until it stabilizes. However, because it is still continuously changing, it is only safe to use IScriptEnvironment2 from inside Avisynth+, or from plugins built and distributed with Avisynth+.

I'll have a real problem though if I ever see the need to modify VideoFrame(Buffer), AVSValue, IClip and so on, thanks to the limitations of the current interfaces. Unlike ScriptEnvironment, these aren't centered around the concept of abstract interfaces like COM-techniques would require.

[ultim]

I have just updated the online repository and the binaries in the first post with a new version, compiled fresh today. There are some goodies here, so let's see:

- First, Gavino's scripting extensions have been integrated. Be sure to say thanks to Gavino for his work again. I've only made minor modifications to his patches, like properly handling empty "if" blocks or missing optional "else" parts, and taking the "last" variable from before the new "if", "while", and "for" statements better into account. I've also added a "break" statement that will allow you to jump out of any loop without reaching the terminating condition. I've counted the votes for the "for"-style carefully, and your votes turned out equally even, so in the end I picked Gavino's original simplified style. You can still write any kind of sophisticated loop using "while".
- I added back proper AvsPmod support, that got temporarily removed earlier 'coz of the plugin system's rewrite. Built-in functions will now work as before, but to get autoloaded functions to show up in AvsPmod will need a slight modification to AvsPmod. This is unfortunately necessary, because autoloading has to be delayed to support adding search folders in scripts. As soon as AvsPmod gets modified you'll have full functionality back again. The needed modifications won't interfere with traditional Avisynth.
- The C interface is now probed before the 2.5 plugin interface, making ffms2 work again even if you're not using qyot27's latest build.
- The "crop" function now defaults to aligned crop. You can still controll alignment using its second parameter, but if you omit it the default is now for the new frame to be aligned. This is important for plugin authors so that they can have a stronger alignment guarantee, in the end leading to faster processing in multiple plugins.
- And as always, there are cleanups and refactors, in an ongoing effort to make the sources higher quality. Not as many as in previous releases, but still. Of course more to come in the future.

64-bit support
There's actually one more feature item missing from the above list. The archive in the first post includes both 32-bit and 64-bit builds. All ASM from the core got replaced with C-code or intrinsics, the inline ASM in filters got sandwiched in ifdefs, and I fixed up any remaining issues that prevented the core from working correctly. But before rushing to download to run your uberscript in 64-bits, please note that no porting of the builtin filters has been done yet, making that build hardly usable. You'll find that many essential filters are missing, like resizers and color space conversions, just to name a few. So trust me, as a user you are most likely better off using the 32-bit build for now. But the 64-bit is there for the adventurous, for motivated testers and developers, and for all those who are wishing to help port the missing functionality to 64-bit. Besides, the new 64-bit build is compatible with existing 64-bit plugins found here (http://code.google.com/p/avisynth64/wiki/PluginLinks), here (http://forum.doom9.org/showthread.php?t=152800) and here (http://members.optusnet.com.au/squid_80/), so you might actually be able to use the 64-bit version for something if you don't rely much on internal filters.

So, there is a working 64-bit build, though farly gutted out. If you know some intrinsics, please consider helping out with the port, even if with only one or two routines. Even without knowledge of intrinsics or ASM, if you can rewrite some algorithms that were only available in ASM before into plain C, that would already be a lot of help. Feel free to start anywhere you like, and rest assured, I'm also continuing my work on Avisynth+.

Edit: Oh, I almost forgot. I added a small number of issues to the GitHub page. They are just ideas and enhancement requests. If you feel like it, you can also pick up something from there if you prefer working on something new, or you can also add your own ideas.

[StainlessS]

I've also added a "break" statement that will allow ...

thumbs up on that, thanx. EDIT: Indeed, lots of good ideas, GScript & Grunt have been ripe for inclusion for eons. :)

[kypec]

Hi ultim,

today I've pulled the trigger and wanted to give your build a try but there seems to be something missing or broken in Windows build avsplus_2013_10_15.zip.
I'm using Windows 7 64-bit version and all my scripts were working correctly with AviSynth 2.60 build May 16, 2012.
As soon as I replaced dynamic libraries in my SysWOW64 directory with files from your package (placed under x86 folder) AviSynth.dll and DevIL.dll my script that uses MCTD plugins stopped working, giving this error message:
AVSMeter 1.7.0 [AVS2.6] by Groucho2004

TemporalSoften::GetFrame is not yet ported to 64-bit.
Here's my script that works fine if MCTD processing is commented out:# Initialize variable with path to my A/V tools
plugindir="C:\PROGRAM FILES (x86)\AVTools\plugins\"
# Load all the plugins that will be used in the script
LoadPlugin(plugindir + "DGDecode.dll")
Import(plugindir + "MCTD_Loader.avsi")
# Open source video clip
MPEG2Source("D:\Source\That70sShow\S1E01.d2v", info=3)
# Crop to get rid of tiny overscan and ugly artifacts at the edges
Crop(2, 2, -2, -2)
# Denoise - must be left out to make it work with AviSynth+
MCTemporalDenoise(settings="low", stabilize=true, enhance=true, edgeclean=true, sharp=false)
# Add black borders to get standard mod16 resolution 720x576
AddBorders(2, 2, 2, 2)


Could you please investigate why internal filter TemporalSoften is throwing this misleading error? AFAIK I'm not using 64-bit of AviSynth, I've never had it installed on my system and unless you mixed DLLs in your package I can't see why standard 32-bit should fail on me...

Thanks in advance! ;)

[vdcrim]

TemporalSoften::GetFrame is not yet ported to 64-bit
It's a typo (https://gist.github.com/vdcrim/bf78785e0a59bea6bd2f).

[kypec]

It's a typo (https://gist.github.com/vdcrim/bf78785e0a59bea6bd2f).

Excellent! :thanks:
Hopefully ultim uploads new fixed build before I even try to get familiar with git on Windows and compile patched sources myself. :D