AviSynth+ thread Vol.2

[kedautinh12]

Thanks

[StainlessS]

Tanks P, I'll give test7 a whirl :)

[wonkey_monkey]

pinterf,

I've been thinking long and hard about Expr for the last few days, because it's so much quicker than the RPN compiler I've been working on for a while. It does lack a number of features that I have found very useful, though - not that I'm asking you to do anything, Expr is excellent enough as it is and you've already been kind enough to implement a few things at my suggestion, but I just thought it might be worth going over my ideas in public in case of it's any interest to anyone.

As I use RPN for a lot of things, a standalone RPN compiler is something of a necessity for me. But before I go off in my own direction - taking plenty of inspiration from Expr, if not actual code; anything I eventually release will be sure to be open source though - I just wondered about your potential interest in such a project, as in whether it would be something that could be usefully developed in parallel with future iterations of Expr (and/or used to create new programmable features of Avisynth).

On the one hand it seems silly to duplicate so much effort, when Expr already shares so much in common with my aims. But on the other hand, what I'm envisioning would also be quite different:


A standalone RPN compiler, which has a concept of input and output "planes" but only as generic data sources, with width, height, (and possibly additional dimensions), type (byte, short, int, float, double) and pitch. Bit depth conversions and colour channel bias would be left for the calling function to inline as boilerplate RPN ops (more or less what Expr already does, just abstracted one more level)
The ability to do per-clip, per-frame, per-line, and per-pixel calculations (storing variables). Per-clip would probably just be constant folding since there'd be no changeable input. Per-line might be easy to implement directly in Expr - it's just a matter of moving the start of the loop and making sure the stack is empty at the start of the loop. Per-frame is trickier. Per-pixel only applies if the output channels are calculated in an interleaved way, which is natural for packed RGB(A) but not so natural for YUV(A) (you have to jump around between the output planes)
Internal multithreading
Choice of float or double precision (my current compiler uses the x87 stack which offers 80-bit precision - not available with SIMD unfortunately)
Trig functions implemented with SIMD (based on ssemath or some other library), floor, ceil, bitwise operators and so on
Automatic type switching between int and float
Option to return black out-of-bounds pixels instead of repeating edges
Something more akin to true conditional execution - difficult with SIMD but not impossible to emulate - rather than just the basic a/b swap ternary operator
Internal multi-dimensional arrays - build your own LUT with per-clip/per-frame calculations!
Run-time calculable pixel offsets
Support for other planes (Y in U, G in Y, etc) to be referenced (if they're the same size). Allow non-matching input clips as long as their non-matching planes aren't referenced
Temporal offsets to access past/future frames
Inlined Avisynth variables
Support for comments (a simple regex can remove several styles)


The last four I plan to implement first as an Expr wrapper plugin, as a separate/proof-of-concept project.

Hmm... you know, now that I write this out it does seem like a bit of a monster. Still, it would be just as helpful to hear if it's of no real interest to you - no offence will be taken! I'd also welcome your thoughts on the subject, or if anyone else has any feature requests or hints.

PS Thanks for everything you do!

[pinterf]

Meanwhile VapourSynth's Expr got sin and cos (https://github.com/vapoursynth/vapoursynth/pull/693) functions.
This is present. And the future: I recommend you having a look at this exciting Expr implementation which may be the future of Expr.
https://github.com/AkarinVS/vapoursynth-plugin
Not specific to Intel, uses LLVM engine, integer option.

[kedautinh12]

Wow sounds good

[Dogway]

Had a similar post in my mind for a few days but since wonkey_monkey shared his thoughts I will share mine as well.

After fiddling with Expr for some months now I observed that min and max (maybe clip too?) operators are very slow, slower than masktools2, so I might think there's room for optimization on that side.

Obviously pixel addressing further acceleration is desired, but also LUT calculations for 8-bit as pinterf is already working on. On this regard I wondered also an option to define the type of calculation of the expression (float, double, int) to speed up the expression or process with more precision.

Another thought was implementation of absolute pixel addressing, although I presume this wouldn't be very optimized.

One observation I found on the Expr() documentation is the following example:
"x[-1,-1] x[-1,0] x[-1,1] y[0,-10] + + + 4 /"

As I could find this is very unoptimized, Expr() prefers continuous calculations on the stack, so:
"x[-1,-1] 0.25 * x[-1,0] 0.25 * + x[-1,1] 0.25 * + y[0,-10] 0.25 * +"

In this example I also change division by 4 with multiplication by 0.25. I read that in expressions the product is faster than the division but not for all divisions, like for example even integers, is this true? For the time being I'm replacing all divisions with the product of the reciprocal.

Finally I'm starting to use optSingleMode=true for lutxyz operations as I saw an increment in performance, but sometimes if the expression is not complex enough performance is lower. Any word on this is appreciated since the documentation is not very clear.

Anyway thanks for the help, I feel like I opened pandora's box lol.

[real.finder]

Meanwhile VapourSynth's Expr got sin and cos (https://github.com/vapoursynth/vapoursynth/pull/693) functions.
This is present. And the future: I recommend you having a look at this exciting Expr implementation which may be the future of Expr.
https://github.com/AkarinVS/vapoursynth-plugin
Not specific to Intel, uses LLVM engine, integer option.

AkarinVS said in https://github.com/AkarinVS/vapoursynth-plugin/releases/tag/v0.60

You can use this to implement arbitrary convolution kernels (i.e. non-regular shapes), and my benchmark indicated that 3x3 convolution implemented this way is as fast as std.Convolution.

Dogway said that vs Convolution is fast, that mean Pixel addressing will kill vs Convolution? don't know if we can made Expr version for http://www.vapoursynth.com/doc/functions/boxblur.html and have same speed, or boxblur will need backport

[wonkey_monkey]

One observation I found on the Expr() documentation is the following example:
"x[-1,-1] x[-1,0] x[-1,1] y[0,-10] + + + 4 /"

As I could find this is very unoptimized, Expr() prefers continuous calculations on the stack, so:
"x[-1,-1] 0.25 * x[-1,0] 0.25 * + x[-1,1] 0.25 * + y[0,-10] 0.25 * +"

I found the first to be slightly faster than the second, and faster still when I replaced "4 /" with "0.25 *". I haven't dug too deep into Expr code yet, but I don't think having a few items on the stack will cause a slowdown. It may not even evaluate those pixel references until they're required by an operation anyway.

In this example I also change division by 4 with multiplication by 0.25. I read that in expressions the product is faster than the division but not for all divisions, like for example even integers, is this true? For the time being I'm replacing all divisions with the product of the reciprocal.

I would expect multiplication by a reciprocal to be faster in pretty much all cases. Floating point reciprocals are accurate for powers of 2, but may not be perfectly accurate otherwise. 1/3, for example, can't be precisely represented by an IEEE-754 floating point number. But it's unlikely you'll bump into any rounding errors anyway, certainly not with a simple script like this one.

[kedautinh12]

I remember Dogway have ex_boxblur in extools
https://github.com/Dogway/Avisynth-Scripts/blob/2ef63fbb4fa97c8657a3a61461c03d0467133e3d/ExTools.avsi#L392

[real.finder]

I remember Dogway have ex_boxblur in extools
https://github.com/Dogway/Avisynth-Scripts/blob/2ef63fbb4fa97c8657a3a61461c03d0467133e3d/ExTools.avsi#L392

yes, seems I forget it

import vapoursynth as vs
core = vs.get_core(threads=1)
import mvsfunc as mvf
clip=core.lsmas.LWLibavSource(source=r'test.avi')
clip=mvf.GetPlane(clip, 0)
clip=core.std.BoxBlur(clip,hradius =4,vradius =4)
clip.set_output()

about 207 fps and with get_core() (mt) its about 640 fps


LWLibavVideoSource("test.avi")
#RequestLinear(clim=100)
ConvertToY
ex_boxblur(4)
#Prefetch()


about 328 fps and about 854 fps with RequestLinear(clim=100) and Prefetch()

[wonkey_monkey]

I get 308fps from ex_boxblur(4), but 340fps if I modify ex_boxblur so the Expr strings use one trailing multiply instead of multiplying each pixel separately.

[Dogway]

I found the first to be slightly faster than the second.
I think you are right, I might have mixed my benchmarks with the per pixel reciprocal product, will test further. As a side note I found at least for these synthetic benchs that convolutions perform best with Prefetch(physical cores) than with threads.


ex_boxblur(1) is actually very fast, 87% (maybe more with above note) of removegrain(19) but with the benefit of being variable. The problem is the same for all pixel addressing convolutions which are limited to SSSE3, I don't know about VS acceleration on boxblur or Convolution() (benchmarks welcome) but if an internal boxblur in AVS+ can be implemented I think it could improve performance. In any case I don't think this to be a priority, I would rather improve Expr() or implement functions not doable in Expr() like mt_hysteresis().

I also thought on a sandbox function that operates on double float for algebra operations not involving clips, so I can do all the derivations with double float and ultimately feed that to Expr() which -for the moment- operates on single float.


EDIT: some benchs over 16-bit and Prefetch(4). Moved to ExTools thread (https://forum.doom9.org/showthread.php?p=1946603#post1946603)

[wonkey_monkey]

Perhaps we should move this to your thread!

[zorr]

- Clip content support for propGetAsArray and propGetAll

(no propSetArray changes in this version)

Thanks, I tested propGetAsArray and now it works with clips.

[wonkey_monkey]

A long, long time ago I pointed out that BicubicResize, with its default parameters, blurs when (ideally) it shoud not. The principle of least surprise dictates that a resizer should not do anything to the original pixels if it's called as a "null" resizer (e.g. resizing to the exact width and height).

For compatibility reasons this was not changed, however a shortcut which bypassed resizing entirely was, if I recall correctly, removed, so that a "null" resize would still perform the resize. This was to keep it consistent with non-null resizes.

This seems to have been reverted at some point, as this script shows if you step through:

version
shifted = bicubicresize(width, height, src_left=0.00001, src_top = 0.00001) # a very minor pixel shift to avoid the "null-resize" skip code
unshifted = bicubicresize(width, height, src_left = 0, src_top = 0) # this should be nearly identical to the above, but it isn't
interleave(unshifted, shifted)
pointresize(width*4, height*4) # you may need to zoom in to see the difference


The two clips should be practically identical, being only shifted by a fraction of a pixel, but the blur/non-blur versions are distinguishable.

Personally I still think the default b and c parameters should be changed to 0 and 0.5 respectively to remove the blur entirely (backwards compatability be damned! ;) ), but could the "skip" be removed again to remove this discontinuity?

[pinterf]

After fiddling with Expr for some months now I observed that min and max (maybe clip too?) operators are very slow, slower than masktools2, so I might think there's room for optimization on that side.

In Expr _all_ operations are done on solely 32 bit floats. Unlike masktools' lut expressions which operates on 64 bit doubles. But masktools has no internal JIT compiler like Expr, so if the use case is not a LUT (which evaluates the expression N times for filling the lut table once then use this lookup for all frames) then is slow like hell. It's because if the lut table is too large in masktools (imagine a 16 bit xy lookup table which requires 8 GB RAM) then it switches to 'realtime' working mode - in masktools this is not accelerated.

- All inputs are immediately converted to float, 8-16 bit integers. Float pixel type is not converted during load.
- Integer constansts are already stored as floats (no extra time during the evaluation)
- Constants and operators on constants are evaluated during the preprocessing phase so 2 + 2 is not evaluated for each frame because it is stored as 4.
- Expression then is evaluated using 32 bit float instructions
- Output is then stored; either as-is (float pixel format) or after rounding + clamp to valid range + convert back to 8-16 bit integer format

Helper options which do input auto-scaling to 8 (or whatever) bit internal format then back, have an extra overhead during pixel-read and pixel store.

All these auto-scale things (both in masktools and Avisynth Expr) are 'convenience' functions which were requested during the beginning of HBD transition era. They may not provide a fully speed optimized script but helped in translating a lot of 'old 8 bit compatible' scripts for 10+ bits environment.

Note that other auto-scaler keywords (such as 255 scalef) mean no extra burden on execution time because of the preprocessor constant folding (for a 10 bit clip 255 scalef is translated to 255 * (1023.0f/255.0f) that is "255 4.011764705882353 *" in RPN, and finally converted into single constant of 1023 - all this is optimized out in the parsing/preparation phase.

VapourSynth expressions and scripts are solving the high bit depth problem by manually providing scaled constants into the scripts.


On this regard I wondered also an option to define the type of calculation of the expression (float, double, int) to speed up the expression or process with more precision.

Optimizing the expression into assembler code is a handcrafted task in Expr. Now it is using 32 bit float instruction set. How the actual operators are translated into SSE2/AVX2 mnemonics - assembler code - is more or less hardcoded. Using 64 bit doubles (which I have tried once but abandoned due to its extra complexity) requires rewriting everything once more.
And we are still not talking about mixing the types together.

This is why the above mentioned LLVM approach is exciting, their group of developers probably spent years on the background optimization technology.


One observation I found on the Expr() documentation is the following example:
"x[-1,-1] x[-1,0] x[-1,1] y[0,-10] + + + 4 /"

As I could find this is very unoptimized, Expr() prefers continuous calculations on the stack, so:
"x[-1,-1] 0.25 * x[-1,0] 0.25 * + x[-1,1] 0.25 * + y[0,-10] 0.25 * +"

In this example I also change division by 4 with multiplication by 0.25. I read that in expressions the product is faster than the division but not for all divisions, like for example even integers, is this true? For the time being I'm replacing all divisions with the product of the reciprocal.

Float multiplication is quicker on all processor architectures.
_mm_mul_ps (https://software.intel.com/sites/landingpage/IntrinsicsGuide/#expand=5080,3538,4676,348,5538,5595,5080,3573,3556,5316,3077,3928&text=_mm_mul_ps) and _mm_div_ps (https://software.intel.com/sites/landingpage/IntrinsicsGuide/#expand=5080,3538,4676,348,5538,5595,5080,3573,3556,5316,3077,3928,2156&text=_mm_div_ps) (and their 256 bit equivalents) are used.



Finally I'm starting to use optSingleMode=true for lutxyz operations as I saw an increment in performance, but sometimes if the expression is not complex enough performance is lower. Any word on this is appreciated since the documentation is not very clear.

Anyway thanks for the help, I feel like I opened pandora's box lol.
Expr (running a machine code generated by its JIT compiler) is using multimedia registers (128 bit XMM registers for SSE2 or 256 bits YMM registers for AVX2) for the internal operations.
A 128 bit register can hold 4 pixels (4x32 bit float)
A 256 bit register can hold 8 pixels (8x32 bit float)

Originally Expr is handling two sets of such registers in a parallel way, so it loads 2x4 consecutive pixels into two registers (or 2x8 pixels for AVX2) then performs the same operations on both 'lanes'.

Then the results in the two registers are converted back and stored into 2x4 or 2x8 bytes (8 bit pixel type) or 2x8/2x16 bytes (16 bit pixel types)

optSingleMode option is loading/using/storing only a single register intead of the above mentioned dual two-lane method.

I made the optSingleMode option because I saw in the JIT generated code that using two parallel register sets are not always optimal, complex scripts will turn into heavy register-to-memory swapping.

If the expression is complex that is the actual RPN stack is deep, intermediate calculations cannot be fit into available registers. On 32 bit we have only eight registers: XMM0..XMM7. The default dual-lane mode is using them in pairs, so effectively we can use only 4 of them. When the depth of expression/internal stack is over 3 then we are out of available registers. The currently unused registers (holding the result of an earlier calculation) will be swapped into memory in order to allow loading data for the new operations. Then swapped back from memory.
Memory access has cost.

By the fact that optSingleMode needs practically half of the registers than the default working method, it is effectively allowing twice as much playground in the expression evaluation depth before the swap-unused-registers-to-memory event kicks in.

In 64 bit the situation in not that bad, we have extra registers there, it allows more complex expressions w/o penalty compared to 32 bit x86 code.

[feisty2]

pinterf,

I've been thinking long and hard about Expr for the last few days, because it's so much quicker than the RPN compiler I've been working on for a while. It does lack a number of features that I have found very useful, though - not that I'm asking you to do anything, Expr is excellent enough as it is and you've already been kind enough to implement a few things at my suggestion, but I just thought it might be worth going over my ideas in public in case of it's any interest to anyone.

As I use RPN for a lot of things, a standalone RPN compiler is something of a necessity for me. But before I go off in my own direction - taking plenty of inspiration from Expr, if not actual code; anything I eventually release will be sure to be open source though - I just wondered about your potential interest in such a project, as in whether it would be something that could be usefully developed in parallel with future iterations of Expr (and/or used to create new programmable features of Avisynth).

On the one hand it seems silly to duplicate so much effort, when Expr already shares so much in common with my aims. But on the other hand, what I'm envisioning would also be quite different:


A standalone RPN compiler, which has a concept of input and output "planes" but only as generic data sources, with width, height, (and possibly additional dimensions), type (byte, short, int, float, double) and pitch. Bit depth conversions and colour channel bias would be left for the calling function to inline as boilerplate RPN ops (more or less what Expr already does, just abstracted one more level)
The ability to do per-clip, per-frame, per-line, and per-pixel calculations (storing variables). Per-clip would probably just be constant folding since there'd be no changeable input. Per-line might be easy to implement directly in Expr - it's just a matter of moving the start of the loop and making sure the stack is empty at the start of the loop. Per-frame is trickier. Per-pixel only applies if the output channels are calculated in an interleaved way, which is natural for packed RGB(A) but not so natural for YUV(A) (you have to jump around between the output planes)
Internal multithreading
Choice of float or double precision (my current compiler uses the x87 stack which offers 80-bit precision - not available with SIMD unfortunately)
Trig functions implemented with SIMD (based on ssemath or some other library), floor, ceil, bitwise operators and so on
Automatic type switching between int and float
Option to return black out-of-bounds pixels instead of repeating edges
Something more akin to true conditional execution - difficult with SIMD but not impossible to emulate - rather than just the basic a/b swap ternary operator
Internal multi-dimensional arrays - build your own LUT with per-clip/per-frame calculations!
Run-time calculable pixel offsets
Support for other planes (Y in U, G in Y, etc) to be referenced (if they're the same size). Allow non-matching input clips as long as their non-matching planes aren't referenced
Temporal offsets to access past/future frames
Inlined Avisynth variables
Support for comments (a simple regex can remove several styles)


The last four I plan to implement first as an Expr wrapper plugin, as a separate/proof-of-concept project.

Hmm... you know, now that I write this out it does seem like a bit of a monster. Still, it would be just as helpful to hear if it's of no real interest to you - no offence will be taken! I'd also welcome your thoughts on the subject, or if anyone else has any feature requests or hints.

PS Thanks for everything you do!

in that case you should just write a normal C++ plugin, which will be way easier to read, to write and to maintain than ultra complex RPN expressions.

[pinterf]

A long, long time ago I pointed out that BicubicResize, with its default parameters, blurs when (ideally) it shoud not. The principle of least surprise dictates that a resizer should not do anything to the original pixels if it's called as a "null" resizer (e.g. resizing to the exact width and height).

For compatibility reasons this was not changed, however a shortcut which bypassed resizing entirely was, if I recall correctly, removed, so that a "null" resize would still perform the resize. This was to keep it consistent with non-null resizes.

This seems to have been reverted at some point, as this script shows if you step through:

...but could the "skip" be removed again to remove this discontinuity?

The omission is here:
https://github.com/AviSynth/AviSynthPlus/blob/master/avs_core/filters/resample.cpp#L709
and
https://github.com/AviSynth/AviSynthPlus/blob/master/avs_core/filters/resample.cpp#L740

This is the specific commit (https://github.com/AviSynth/AviSynthPlus/commit/47758e185af0205c38f846a8428b9807f8da8a13).

The behaviour didn't change since then.

It keeps returning the original clip in only one case: when everything is unchanged.

[pinterf]

A video file with Utf-8 characters in the file name is opened with AvsPmod. AvsPmod has no problem with that. It is now written there
LWLibavVideoSource ("Это тестовый Video.mkv")
As I wrote, with 'Eval' I get an error message, I save this script and open it with AviSource there are no problems
Could you please send me the exact files mentioned here?
The file you are opening with avspmod.
The saved file you are opening with AviSource.
(But no need for the Это тестовый Video.mkv, I can create the file with such a name for myself), thanks

[wonkey_monkey]

in that case you should just write a normal C++ plugin, which will be way easier to read, to write and to maintain than ultra complex RPN expressions.

I think you've misunderstood. My bullet points are a list of features I plan on implementing, or have already implemented, in a C++ RPN-compiling plugin. I don't plan on implementing them with RPN. That'd be crazy.

It keeps returning the original clip in only one case: when everything is unchanged.

Well... I'm not sure it should do that either. If you use animate BicubicResize to scroll something, or do a zoom in, there'll be an inconsistency if one frame is uncropped and unresized:

version
animate(0,16, "bicubicresize", width,height,4,4,-16.0,-16.0, width,height,4,4,16.0,16.0)

See frames 7,8,9.

[feisty2]

no, what you're trying to do is insane. your proposed Expr filter is no longer an expression evaluating filter, but rather an exotic DSL.
you can theoretically implement extremely complex filters like mvtools using this so called Expr, so basically you're creating yet another DSL inside a DSL (the avisynth script itself is already a DSL), and that's very harmful, especially considering that the RPN expression is a rather exotic language.

you best bet is to directly extend the avisynth scripting language with frame access, pixel access, ..., whatever feature you planned for your Expr DSL. at least avisynth is a readable DSL.


I think you've misunderstood.

you're the one who misunderstood. I was saying that, if someone wants to write something that requires all of your proposed features (arbitrary spatiotemporal pixel access, etc.) he/she should write a C++ plugin for that, rather than resort to an unreadable RPN-based DSL.

[wonkey_monkey]

no, what you're trying to do is insane.

I've already done it; I'm just looking to improve it.

your proposed Expr filter is no longer an expression evaluating filter, but rather an exotic DSL.

It is what it is. Most of my bullet points are just extensions to what Expr already provides (or Expr provides a subset of my filter's features). Isn't Expr already slightly more than just an expression evaluating filter?

and that's very harmful

What's harmful about it?

you best bet is to directly extend the avisynth scripting language

Doesn't that also apply to Expr itself?

My best bet is definitely this. I've got use cases for it beyond Avisynth, for a start.

(arbitrary spatiotemporal pixel access, etc.) he/she should write a C++ plugin for that, rather than resort to an unreadable RPN-based DSL.

I really don't understand what you're trying to say. I'm writing (written, really, it's just a bit rough around the edges) a C++ plugin that compiles user-provided RPN strings, one feature of which is spatiotemporal pixel access (only arbitrary spatially, not temporally). This is just an extension to Expr's spatial pixel access (an idea which I think may have originated with me in the first place). I'm really not sure what's so offensive about that.

[StainlessS]

I'm really not sure what's so offensive about that.
Aint nuttin' offensive bout it, you know what Feisty's like, just ignore him :)
I'm eager to see what weird stuff you come up with [even if it is a bit niche, and even if I dont understand it].

I'm guessin' that nobody is gonna be forced to use it.

[feisty2]

You change the nature of something when you add enough new things to it. Expr was originally an expression evaluating filter and by its nature, not capable of doing many complex things, that’s why it suited just fine with a somewhat exotic language (RPN).

Now that you’ve changed the nature of Expr, you want DSL capabilities, then you should go for an actual DSL instead of having DSL stuff layered on top of something that no longer serves the actual purpose

[wonkey_monkey]

I'm guessin' that nobody is gonna be forced to use it.

Shhh, you're giving away my grand plan!

[gispos]

Could you please send me the exact files mentioned here?
The file you are opening with avspmod.
The saved file you are opening with AviSource.
(But no need for the Это тестовый Video.mkv, I can create the file with such a name for myself), thanks
You don't need a video from me for this.
1.) Name a video Japanese
2.) Open it with AvsPmod (error)
3.) Save this script
4.) Open the saved script in AvsPmod with AviSource

https://i.postimg.cc/jDr4wSZr/Error.jpg (https://postimg.cc/jDr4wSZr)
https://i.postimg.cc/ZCNL5dPc/Avi-Source.jpg (https://postimg.cc/ZCNL5dPc)

[wonkey_monkey]

This expr behaves unexpectedly (at least to me, I might be missing something):

expr("sxr 0 < 128 192 M^ 255 ?")

"192 M^" should amount to a no-op as far as the stack is concerned, but instead of returning a white clip (which is what happens if you take out "192 M^"), a black clip is returned (it's returning the value of "sxr 0 <", as if the ternary operator and subsequent stack elements have been completely removed).

I think the problem might lie with constant folding and findBranches.

[Dogway]

pinterf, out of curiosity, how is the lookup table filling? If I have 256 values in 8-bit the expression evaluates 256 times for Y plane, so 256 possible values. How is that taking so much RAM space?
Same for 16-bit, for single LUT, it's only 65536 values, I have seen 1D LUTs of length 130000 taking less than 4Mb. Besides you might probably get away with half length and use some kind of interpolation (ie. 3D LUTs can use tetrahedral).
YUV planes are decorrelated so you can use 3x1D LUTs instead of 3D LUTs which are heavier.
Sorry if this is dumb as I have no programming background.

[pinterf]

pinterf, out of curiosity, how is the lookup table filling? If I have 256 values in 8-bit the expression evaluates 256 times for Y plane, so 256 possible values. How is that taking so much RAM space?
Same for 16-bit, for single LUT, it's only 65536 values, I have seen 1D LUTs of length 130000 taking less than 4Mb. Besides you might probably get away with half length and use some kind of interpolation (ie. 3D LUTs can use tetrahedral).
YUV planes are decorrelated so you can use 3x1D LUTs instead of 3D LUTs which are heavier.
Sorry if this is dumb as I have no programming background.
Masktools LUT calculation is allocating 256 (8 bit) or 65536*2 (10-16 bits) bytes for 1D Luts.
This is per instance and per plane.

Latest public version of masktools is able to reuse the similar expression lookups for lut_xyz in order to spare with memory and calculation time.

My workbench version extends this behaviour to mt_lutxy and mt_lut as well. As a side effect, it will no longer allocate a larger buffer size than needed (e.g. 65536 elements for a 10 bit video which would require only 1024 elements).
Maybe I'm gonna build a test version from the actual code.

[pinterf]

Avisynth+ 3.7.1 test build 8 (20210705) (https://drive.google.com/uc?export=download&id=1NK_XRbelWytZYOl7KLDb4JgLGvaYEFjV)
20210705 WIP
------------
- Expr: add 'round', 'floor', 'ceil', 'trunc' operators (nearest integer, round down, round up, round to zero)
Acceleration requires at least SSE4.1 capable processor or else the whole expression is running in C mode.
- Fix: Expr: wrong constant folding optimization when ternary operator and Store-Only (like M^) operator is used together.
For the expression "sxr 0.5 < 128 192 M^ 255 ?" the expected result must be the same as for "sxr 0.5 < 128 255 ?"
Since 192 M^ is no-op regarding the expression parts.
- Add rest of Clip-type frame property setter methods: propSetArray

[pinterf]

This expr behaves unexpectedly (at least to me, I might be missing something):

expr("sxr 0 < 128 192 M^ 255 ?")

"192 M^" should amount to a no-op as far as the stack is concerned, but instead of returning a white clip (which is what happens if you take out "192 M^"), a black clip is returned (it's returning the value of "sxr 0 <", as if the ternary operator and subsequent stack elements have been completely removed).

I think the problem might lie with constant folding and findBranches.
Thanks, good catch, fixed, see test8

[kedautinh12]

Thanks

[wonkey_monkey]

Excellent, thanks pinterf!

[FranceBB]

Well done, thanks! :)

[Dogway]

Thanks for the update!

Is a sort operator possible? I found myself the last days fighting with implementing some median algos (removegrain modes 2-4). I know I can't compete in speed but for completion in ExTools this is probably the last function I'm adding.

Tired to do permutations to no avail (https://pastebin.com/2U9XYJZz) I searched some maths and found Sorting Networks, this is also proving hard to achieve with current Expr in-string tools.

Pseudo-code for 8 inputs (swap here means variable swap, 0 as nop sub)
A C < A C swap 0 ?
B D < B D swap 0 ?
E G < E G swap 0 ?
F H < F H swap 0 ?
A E < A E swap 0 ?
B F < B F swap 0 ?
C G < C G swap 0 ?
D H < D H swap 0 ?
A B < A B swap 0 ?
C D < C D swap 0 ?
E F < E F swap 0 ?
G H < G H swap 0 ?
C E < C E swap 0 ?
D F < D F swap 0 ?
B E < B E swap 0 ?
D G < D G swap 0 ?
B C < B C swap 0 ?
D E < D E swap 0 ?
F G < F G swap 0 ?

I thought on assigning new variables but Expr falls short with only A to Z. Maybe possible add AA, AB...?

I also found myself occasionally wanting to do the next:
"x y * sqrt A^ A[-1,0] B^ A[0,0] C^ ...."

Not asking, just thinking out loud, feel free to implement what's easy/useful.

[pinterf]

Thanks for the update!

Is a sort operator possible? I found myself the last days fighting with implementing some median algos (removegrain modes 2-4). I know I can't compete in speed but for completion in ExTools this is probably the last function I'm adding.

Tired to do permutations to no avail (https://pastebin.com/2U9XYJZz) I searched some maths and found Sorting Networks, this is also proving hard to achieve with current Expr in-string tools.

Pseudo-code for 8 inputs (swap here means variable swap, 0 as nop sub)
A C < A C swap 0 ?
B D < B D swap 0 ?
E G < E G swap 0 ?
F H < F H swap 0 ?
A E < A E swap 0 ?
B F < B F swap 0 ?
C G < C G swap 0 ?
D H < D H swap 0 ?
A B < A B swap 0 ?
C D < C D swap 0 ?
E F < E F swap 0 ?
G H < G H swap 0 ?
C E < C E swap 0 ?
D F < D F swap 0 ?
B E < B E swap 0 ?
D G < D G swap 0 ?
B C < B C swap 0 ?
D E < D E swap 0 ?
F G < F G swap 0 ?

I thought on assigning new variables but Expr falls short with only A to Z. Maybe possible add AA, AB...?

I also found myself occasionally wanting to do the next:
"x y * sqrt A^ A[-1,0] B^ A[0,0] C^ ...."

Not asking, just thinking out loud, feel free to implement what's easy/useful.

Getting median is a bit easier for limited number of entries e.g. 3 or 5, and needs no explicite sorting, like is done here.
For example here:
Median of 3
https://github.com/pinterf/MedianBlur2/blob/master/MedianBlur2/medianblur_sse2.cpp#L81
Median of 5
https://github.com/pinterf/MedianBlur2/blob/master/MedianBlur2/medianblur_sse2.cpp#L155

For these magnitudes (3, 5) they are not even complex ones and can be written specifically.

For larger radius we probably need a generic implementation with sorting but then Expr would starting to become an enormously complex filter (embedded MedianBlur :) ) rather than an expression evaluator.

Generic sorting algorithms are available for avx2 and avx512, (search for "avx2 avx512 simd sorting"); I can say that there are really nice solutions, but they are beyond my mental limits :).

[pinterf]

I thought on assigning new variables but Expr falls short with only A to Z. Maybe possible add AA, AB...?

At the moment I have exactly 26 memory slots for them (A..Z), assigning letters to slots means one-to-one relation at the moment.

[Myrsloik]

Thanks for the update!

Is a sort operator possible? ...

Why are you not using real compilers at this point? WHY?

I mean even modern javascript engines will probably run better with that number of variables and have cleaner syntax.

Obviously the solution is to implement a secondary stack where you can push and pop overflow values. As a solution to the number of addressable input clips use stackvertical to get around it.

WE DON'T NEED NO STINKING COMPILERS!

[wonkey_monkey]

Thanks for the update!

Is a sort operator possible?

Pseudo-code for 8 inputs (swap here means variable swap, 0 as nop sub)
A C < A C swap 0 ?


The thing to remember about the ternary operator ("?") is that everything prior to that is already evaluated, so you've already swapped the variables and put 0 on the stack. In fact, what you're evaluating there is the value of C, because the stack currently contains:

(A C <) - 0 or 1
C
A
0


You could do the following:

A C < A C ? A C < C A ?

This will put either A C or C A on the stack depending on the comparison.

This isn't great because of the two comparisons. You could always store the first result into a variable:

A C < R@ A C ? R C A ?

Or you could be a bit cleverer and do:

A C < A C ? dup A C + -

Then again it's probably most efficient just to do

A C min A C max

Or you might be able to do some swaps and dups to avoid loading the variables twice (I'm not sure how efficiently Expr recalls them).

Thank you for listening to my TED Talk.

[Dogway]

wonkey_monkey: nice TED hehe. I was going with the new var assignment starting from Z, and then thought what a sudoku I would end playing so asked for AA kind of vars. I will go with the most efficient method, ternaries are costly so min max probably. I'm not sure it might work but will test. Hopefully I can build a nice median and repair library as I did with ex_edge().

Myrsloik: It's just too much for me... and my 2013 SSD. I'm art oriented so my knowledge and disk space goes to DCC (3D, compo, video) and audio software mostly (no games). I fear that getting into nitty gritty programming can derail me too much, or simply make either my brain or SSD explode. I only do basic scripting in AVS, AHK and Python, some regex in there too and very basic GLSL. Will look forward how Expr evolves.

[pinterf]

New version
Avisynth+ 3.7.1 test build 9 (20210706) (https://drive.google.com/uc?export=download&id=1lbiMMPsSFTKoKhUKdNl0NdcKiv0bgLKY)
- Expr: allow arbitrary variable names (instead of A..Z), up to 256 variables can be used. Do not use existing keywords.
Variable names must start with '_' or alpha, continued with '_' or alphanumeric characters.

[kedautinh12]

Thanks

[FranceBB]

I just replaced the files on my installation and there's already a new version of AVS+.
You never take a break, Ferenc, thanks!! :D

[feisty2]

Myrsloik: It's just too much for me... and my 2013 SSD. I'm art oriented so my knowledge and disk space goes to DCC (3D, compo, video) and audio software mostly (no games). I fear that getting into nitty gritty programming can derail me too much, or simply make either my brain or SSD explode. I only do basic scripting in AVS, AHK and Python, some regex in there too and very basic GLSL. Will look forward how Expr evolves.

it is a common misconception to assume that writing a proper C++ plugin requires more work than dirty hacks like feature creep Expr.

to describe a 3x3 gauss blur filter, you have in Expr: "x[-1, -1] x[-1, 0] 2 * + x[-1, 1] + x[0, -1] 2 * + x 4 * + x[0, 1] 2 * + x[1, -1] + x[1, 0] 2 * + x[1, 1] + 16 /"

meanwhile to describe the same thing in C++: https://github.com/IFeelBloated/vapoursynth-plusplus/blob/master/Examples/GaussBlur.hxx#L22

you tell me which one is easier to write and to understand.

a temporal median filter (https://github.com/IFeelBloated/vapoursynth-plusplus/blob/master/Examples/TemporalMedian.hxx#L25) that works for any temporal radius is also no more than a few lines in C++. but in this case, you can't even describe it (by ab)using Expr.

it's year 2021 and things have changed, drastically. writing a C++ plugin is scripting (given that it's a simple filter like 3x3 conv or temporal median), and C++ is a scripting language, if you will it to.

[StainlessS]

As Feisty says, some things are actually easier in C/CPP, and Doggy would make for a good coder. [So would Real.Finder].
The only real difficult bit about learning C is figuring out where to put all them damn semi-colons, after that its no more
difficult than AVS script.

[feisty2]

davidh*****, I wasn't replying to you. I was replying to Dogway continuing on Myrsloik's point. I suggest that you do not cut in on a conversion not centered around you, and stop acting like you own the place or something. It's a public forum, you do not have the right to stop me from talking just because you don't agree with my opinions.

[feisty2]

now back to your points

Oh, feisty, please give it a rest with the moaning. No-one's ever going to force you to use Expr. Clearly some of us find it very useful, even those like me who can write fully-fledged C++ plugins when we need to. I just don't understand why you're so offended that we have a different tool to use when the occasion rises.

I am a frequent user of Expr and it would be ridiculous for you to think that I am "offended" by Expr. However, I use Expr for the right purpose, namely expression evaluation.
what I am concerned about is that, if this feature creep Expr gets widely adopted, people would start abusing Expr for things that should be written in a proper programming language (Dogway is already asking for the sort functionality). the consequence of this is that it creates a shit mountain of unreadable and unmaintainable code, because: a) RPN itself is a rather "exotic" language, hard to decipher by human. b) Expr, even with all that feature creep, is still nowhere as expressive as a real programming language, so it is very likely that sometimes there's no direct way to describe what you want using Expr. and people would then "invent" weird tricks to do what they want indirectly, and those "tricks" can be extremely confusing to the reader of the code.

[feisty2]

Expr makes customised filtering incredibly quick and easy. I have scripts where I have to do dozens of Expr-friendly operations. It's ridiculous to write dozens of C++ filters, or even one multi-choice filter, when a single line of Expr per use will do.
...
And then when you're testing, you need to close VirtualDub or AVSpmod every time you recompile.

a C++ filter takes arguments, you simply make the WIP part, something like a not yet determined convolution kernel, a parameter of the filter and you're free to fill in all kinds of different arguments in your script.


And it's disingenous to imply that it universally doesn't require more work to write a C++ plugin (and Expr is not a dirty hack).

At the very best, with a .cpp template already saved somewhere, you've still got to create your project, add your source code, link avsynth.lib, make sure avisynth.h is included, fiddle about with optimisation switches, make sure you've got the right architecture targetted, that you're not accidentally doing a Debug build, add a build event to copy the DLL to your plugins folder... the list goes on.

that's your overcomplicated workflow. for me it's just one command line: g++ -shared -std=c++2b -lstdc++ -static -O3 -flto -march=native -finline-limit=1000000000000000000000000000 -funroll-all-loops -funsafe-loop-optimizations -o Filter.dll EntryPoint.cxx vapoursynth.lib

[feisty2]

Since you ask, the Expr one, definitely on the first count - it's far less code and far easier to debug

is this supposed to be a joke? RPN is easy to debug? RPN expressions are incredibly "nonlocal", in ways that there's no parentheses to identify the local sections of a complex formula. you'd have to imagine a stack in your head, and emulate the evaluation process with your imaginary stack to determine the operands of an operator. good luck with modifying a local term of a multi-line long expression.


x[-1, -1] x[-1, 0] 2 * + x[-1, 1] + x[0, -1] 2 * + x 4 * + x[0, 1] 2 * + x[1, -1] + x[1, 0] 2 * + x[1, 1] + 16 /

is hardly "far less code" than

auto GaussKernel = [](auto Center) {
auto WeightedSum = Center[-1][-1] + Center[-1][0] * 2 + Center[-1][1] +
Center[0][-1] * 2 + Center[0][0] * 4 + Center[0][1] * 2 +
Center[1][-1] + Center[1][0] * 2 + Center[1][1];
return WeightedSum / 16;
};

[feisty2]

First you complain about feature creep, now you're complaining that it doesn't have enough features. Make your mind up! Expr is never going to do everything and that's not why it exists.


it is clear that you want Expr to do everything, don't lie.
pinterf,

I've been thinking long and hard about Expr for the last few days, because it's so much quicker than the RPN compiler I've been working on for a while. It does lack a number of features that I have found very useful, though - not that I'm asking you to do anything, Expr is excellent enough as it is and you've already been kind enough to implement a few things at my suggestion, but I just thought it might be worth going over my ideas in public in case of it's any interest to anyone.

As I use RPN for a lot of things, a standalone RPN compiler is something of a necessity for me. But before I go off in my own direction - taking plenty of inspiration from Expr, if not actual code; anything I eventually release will be sure to be open source though - I just wondered about your potential interest in such a project, as in whether it would be something that could be usefully developed in parallel with future iterations of Expr (and/or used to create new programmable features of Avisynth).

On the one hand it seems silly to duplicate so much effort, when Expr already shares so much in common with my aims. But on the other hand, what I'm envisioning would also be quite different:


A standalone RPN compiler, which has a concept of input and output "planes" but only as generic data sources, with width, height, (and possibly additional dimensions), type (byte, short, int, float, double) and pitch. Bit depth conversions and colour channel bias would be left for the calling function to inline as boilerplate RPN ops (more or less what Expr already does, just abstracted one more level)
The ability to do per-clip, per-frame, per-line, and per-pixel calculations (storing variables). Per-clip would probably just be constant folding since there'd be no changeable input. Per-line might be easy to implement directly in Expr - it's just a matter of moving the start of the loop and making sure the stack is empty at the start of the loop. Per-frame is trickier. Per-pixel only applies if the output channels are calculated in an interleaved way, which is natural for packed RGB(A) but not so natural for YUV(A) (you have to jump around between the output planes)
Internal multithreading
Choice of float or double precision (my current compiler uses the x87 stack which offers 80-bit precision - not available with SIMD unfortunately)
Trig functions implemented with SIMD (based on ssemath or some other library), floor, ceil, bitwise operators and so on
Automatic type switching between int and float
Option to return black out-of-bounds pixels instead of repeating edges
Something more akin to true conditional execution - difficult with SIMD but not impossible to emulate - rather than just the basic a/b swap ternary operator
Internal multi-dimensional arrays - build your own LUT with per-clip/per-frame calculations!
Run-time calculable pixel offsets
Support for other planes (Y in U, G in Y, etc) to be referenced (if they're the same size). Allow non-matching input clips as long as their non-matching planes aren't referenced
Temporal offsets to access past/future frames
Inlined Avisynth variables
Support for comments (a simple regex can remove several styles)


The last four I plan to implement first as an Expr wrapper plugin, as a separate/proof-of-concept project.

Hmm... you know, now that I write this out it does seem like a bit of a monster. Still, it would be just as helpful to hear if it's of no real interest to you - no offence will be taken! I'd also welcome your thoughts on the subject, or if anyone else has any feature requests or hints.

PS Thanks for everything you do!

I don't see anything wrong with my argument which suggests that you attempt to make Expr capable of everything (thus the feature creep) and your attempt failed.