vsFilterScript: writing C++ plugins like python scripts (WIP)

[feisty2]

https://github.com/IFeelBloated/vsFilterScript

this is yet another C++ wrapper for VSAPI. However, it is much higher level than vsxx and provides a "scripting" kinda experience to help you sketch your filter in the fastest possible way.

take a look at the 3x3 gauss blur example (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/GaussBlur.hxx), less than 40 lines of code and you got your filter up and running. A temporal median example (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/TemporalMedian.hxx) is also provided to show you how to write temporal or spatiotemporal filters.

there're 2 more examples, Crop (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Crop.hxx) and Rec601ToRGB (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Rec601ToRGB.hxx) showing you filters with advanced vaporsynth features.
Crop (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Crop.hxx) shows you how to write filters that modify the measures of the input (e.g. frame size) and filters that adapt to inputs with arbitrary bitdepths.
Rec601ToRGB (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Rec601ToRGB.hxx) converts a YUV444 clip to RGB using the Rec601 matrix, it shows you how to write filters that modify the format of the input (e.g. YUV->RGB) and how to manipulate frame properties.

C++20 support required (you probably need GCC10 from the trunk). The scripting style syntax is only possible with the latest C++ standard.

I haven't finished porting all C APIs to this wrapper, but the filter skeleton generator is here: https://github.com/IFeelBloated/vsFilterScript/blob/master/Include/Interface.hxx, it requires certain properties, some constants, some member functions as shown in the example filter. The skeleton generator works in a duck typing manner, it generates a set of skeleton functions as long as the filter struct has all the required properties.

you should write each filter in a header filer and include the headers in "EntryPoint.cxx (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/EntryPoint.cxx)" and register each filter with "VaporInterface::RegisterFilter".

The "Clip" object could be accessed as a 4D array ([time (frame)][channel][height][width]) with "GetFrames" or as a 3D array ([channel][height][width]) with "GetFrame". The "time" dimension is relative to the current frame (t=0 for the current frame, t=-1 for the previous frame and t=1 for the next), the other 3 dimensions are absolute. Out-of-bound access is allowed and triggers automatic padding, the behavior of out-of-bound access is defined by concrete padding policies (repeat, reflect, zero...) and the default padding policy is "repeat" for both spatial and temporal dimensions. More details about this part are defined in Plane.hxx (https://github.com/IFeelBloated/vsFilterScript/blob/master/Include/Plane.hxx), Frame.hxx (https://github.com/IFeelBloated/vsFilterScript/blob/master/Include/Frame.hxx) and Clip.hxx (https://github.com/IFeelBloated/vsFilterScript/blob/master/Include/Clip.hxx)

latest update:
new functionality: full integration of C++ exceptions. with exceptions, you no longer have to manually handle any of the following errors:
a) failing to invoke an external plugin (plugin does not exist)
b) failing to invoke an external filter
c) failing to invoke a python function
d) failing to invoke SelfInvoker
... and possibly many more. SelfInvoker is now allowed to throw exceptions so the earlier restriction requiring SelfInvoker to always be successfully evaluated has been removed. Any of these errors will transparently pass through your filters and propagate to a root caller like Create() (https://github.com/IFeelBloated/vsFilterScript/blob/master/include/Interface.vxx#L48) which automatically handles any error.
To you, it would be like the error does not exist so you NEVER have to worry about errors. It's now one step closer to python scripts.

Initialize() has been replaced by normal constructors because with exceptions, it is no longer required to return a value to introspect if the filter has been successfully constructed.

[Myrsloik]

I'm curious, what does the actual generated code look like for this? What's the performance penalty for all the abstraction?

[feisty2]

I'm curious, what does the actual generated code look like for this?
oh I see, you meant generated (machine) code...

What's the performance penalty for all the abstraction?
The main runtime overhead is automatic padding (out-of-bound access detection) which gcc -O3 seems to handle pretty well. everything else is determined at compile time and thus zero cost abstraction.

[feisty2]

did some speed tests,

clp = core.test.GaussBlur(clp)

runs @ 1822.59 fps at 640 x 480 GRAYS (compiled with GCC -O3)


clp = core.std.Convolution(clp, [1,2,1,2,4,2,1,2,1])

runs @ 2425.51 fps at 640 x 480 GRAYS

the comparison is not completely fair tho, test.GaussBlur is a 100% C++ filter (GCC doesn't seem to autovectorize any loop) and std.Convolution has manual avx2 optimization.

[feisty2]

new example: temporal median (https://github.com/IFeelBloated/vsFilterScript/blob/master/TemporalMedian.hxx)
44 lines of perfectly readable, script-like code, and probably even easier to write
vs
376 lines of cryptic C code (https://github.com/dubhater/vapoursynth-temporalmedian/blob/master/src/temporalmedian.cpp)

[zorr]

I have to say this looks very promising. I have a couple of python filters that are annoyingly slow but it seems too much work to port them to actual C/C++ plugins.

What's the performance delta with temporal median?

[Are_]

I noticed when using vsedit benchmark utility CPU cores are at 50% for this and at 20% for convolution.
I rerun the test with vspipe and got this:

14488fps all cores at ~85% load (Convolution)
4098fps all cores at 100% load (test)

[feisty2]

I noticed when using vsedit benchmark utility CPU cores are at 50% for this and at 20% for convolution.
I rerun the test with vspipe and got this:

14488fps all cores at ~85% load (Convolution)
4098fps all cores at 100% load (test)

emm... 14488fps is probably the effect of SIMD optimization, I'll later write a gauss blur filter with the C API and let's see...

[feisty2]

@Are_
could you compile this GaussBlur filter written with low level APIs (https://github.com/IFeelBloated/test_c_filters/blob/master/GaussBlur.cxx) and run a speed test again?

[Are_]

11748fps all cores at 100% load

:(

[feisty2]

interesting... guess I'll have to profile it a little bit and find the main cause that's slowing it down

[josemaria.alkala]

I am amazed how fast this is. Where is the video you are testing?

My Nim version is dead slow (most likely my bad, not Nim's) when compared. I am getting something like 40fps :(

[StainlessS]

I am amazed how fast this is
Looks like GRAYS is a form of blankclip [so timing mostly speed of filter rather than video decoder etc].
From here:- https://forum.doom9.org/showthread.php?p=1905459#post1905459

clip = core.std.BlankClip(format=vs.GRAYS, length=100000, fpsnum=24000, fpsden=1001, keep=True)

[josemaria.alkala]

I just got a bit better: 80fps.

[feisty2]

I am amazed how fast this is. Where is the video you are testing?

My Nim version is dead slow (most likely my bad, not Nim's) when compared. I am getting something like 40fps :(

I'm actually more concerned if the runtime overhead for a given format is constant, so it would be negligible compared to the actual algorithms for complex filters.

[josemaria.alkala]

I am sorry I am not a pro-developer (it is my hobby) so I am not sure if I understand you correctly. Nim compiles to C language. It is garbage collected, but you can disable the garbage collector (just passing "--gc:none"). So my understanding is that there is no runtime (so no overhead in that regard).

It is my first time dealing with memory, so probably I am doing something really bad. I have asked for advice here (https://forum.nim-lang.org/t/6134). You might want to take a look.

When I compile the following code:

import ../vapoursynth
import options

BlankClip( format=pfGrayS.int.some,
width=640.some,
height=480.some,
length=100000.some,
fpsnum=24000.some,
fpsden=1001.some, keep=1.some).Convolution(@[1.0,2.0,1.0,2.0,4.0,2.0,1.0,2.0,1.0]).Savey4m("/dev/null")

by means of:

$ nim c -f --threads:on --gc:none -d:release -d:danger modifyframe
$ time ./modifyframe
real 0m58,879s
user 0m54,969s
sys 0m7,433s

which is 1698fps.

This uses the Convolution filter plus a custom made filter (Savey4m) that for sure is adding some overhead, despite is sending the data to "/dev/null".

How do you read the memory once you have the plane's pointer? Could you send me a link to that particular piece of code (I don't understand much C/C++, I hope to understand enough).

[amichaelt]

How do you read the memory once you have the plane's pointer?

Just looks to be a combination of pointer arithmetic and indexing using the [] operator.

[josemaria.alkala]

I am using the same approach (but for sure I am adding a lot of overhead somewhere).

By the way, just for reference I am using a laptop with a: i7-4770HQ (4cores and 8Gb).

[amichaelt]

I am using the same approach (but for sure I am adding a lot of overhead somewhere).

By the way, just for reference I am using a laptop with a: i7-4770HQ (4cores and 8Gb).

What does the generated C code look like, though? Are you sure there's no extra bounds checking or other code being inserted by the C code generator?

[feisty2]

How do you read the memory once you have the plane's pointer? Could you send me a link to that particular piece of code (I don't understand much C/C++, I hope to understand enough).
https://github.com/IFeelBloated/vsFilterScript/blob/master/Plane.hxx#L47

also in your other post

The results are 80fps (for me) against about 2400fps for a C++ wrapper and 11748fps for a pure C version.

it's 4098fps for the C++ wrapper, not 2400fps (compared to 11748fps for the C version). you have to understand you can only compare these numbers on the same machine... so you can't compare your numbers with numbers reported by Are_ and me, because they were evaluated on different machines.

then, it seems that your nim version operates on int8 clips, the C and C++ plugins were coded for fp32 clips, there's also a significant performance gap here, you can't compare like that.

[josemaria.alkala]

Could you try the convolution (https://github.com/mantielero/VapourSynth.nim/blob/master/src/filters/convolution) and the filter (https://github.com/mantielero/VapourSynth.nim/blob/master/src/filters/modifyframe)? Those are just linux binaries that return:

$ ./modifyframe
Time : 164.1971027851105
Num. frames: 100000
FPS : 609.0241441767271


I would like to see how does it perform on your system.

[MeteorRain]

I was thinking how fast can you run temporal median considering it'll be a lot slower if the radius is large.

If done in SIMD, probably I can use a sorting network but I can imagine it still runs at O(n^2).

[josemaria.alkala]

I manage to compare in my computer the pure C++ filter and a Nim based version.

C++ Version
I took it from here (https://github.com/IFeelBloated/test_c_filters). I compiled with:

g++ -Wall -O3 -shared -fPIC -I. -o libfilter.so GaussBlur.cxx

Create a VapourSynth python filter like test_filter.vpy:

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path='./libfilter.so')
core.std.SetMaxCPU('none')
clip = core.std.BlankClip(format=vs.GRAYS, length=100000, fpsnum=24000, fpsden=1001, keep=True)
clip = core.testc.GaussBlur(clip)
clip.set_output()

and finally:

$ vspipe test_filter.vpy /dev/null
Output 100000 frames in 29.53 seconds (3386.27 fps)


Nim Version
I use custom_filter.nim (https://github.com/mantielero/VapourSynth.nim/blob/master/test/custom_filter.nim) which uses DrawFrame.nim (https://github.com/mantielero/VapourSynth.nim/blob/master/test/DrawFrame.nim).

I compile it like:

$ nim c -f --threads:on -d:release -d:danger custom_filter

And test it by doing:

$ ./custom_filter
Time : 9.394126653671265
Num. frames: 100000
FPS : 10644.94909283766

Using int32.

If I use float32:

$ ./custom_filter
Time : 16.52139902114868
Num. frames: 100000
FPS : 6052.756178335272

I wasn't using multithreading before while vspipe does.

[josemaria.alkala]

Another test that I have done is using the Convolution filter from Nim: convolution.nim (https://github.com/mantielero/VapourSynth.nim/blob/master/test/convolution.nim) and I get:

$ ./convolution
Time : 5.210163354873657
Num. frames: 100000
FPS : 19193.25617813089


Using vspipe and python script:

$ vspipe convolution.vpy /dev/null
Output 100000 frames in 26.87 seconds (3721.76 fps)


I think that vspipe is actually writting frames to /dev/null while I just request the frame and then dismiss them without further processing.

[Myrsloik]

Another test that I have done is using the Convolution filter from Nim: convolution.nim (https://github.com/mantielero/VapourSynth.nim/blob/master/test/convolution.nim) and I get:

$ ./convolution
Time : 5.210163354873657
Num. frames: 100000
FPS : 19193.25617813089


Using vspipe and python script:

$ vspipe convolution.vpy /dev/null
Output 100000 frames in 26.87 seconds (3721.76 fps)


I think that vspipe is actually writting frames to /dev/null while I just request the frame and then dismiss them without further processing.

Correct. Use "vspipe script.vpy ." to simply discard the output

[josemaria.alkala]

It is not showing faster in my computer:

$ vspipe convolution.vpy .
Output 100000 frames in 26.94 seconds (3712.54 fps)
$ vspipe test_filter.vpy .
Output 100000 frames in 29.58 seconds (3380.22 fps)

[MeteorRain]

v = core.std.BlankClip(format=vs.GRAY8, length=10000, fpsnum=24000, fpsden=1001, keep=True)
v = core.neo_tmedian.TemporalMedian(v, radius=3)
v.set_output()

Output 10000 frames in 6.10 seconds (1638.35 fps)

radius=10

Output 10000 frames in 18.55 seconds (539.06 fps)

That uses the same algorithms as F2's, std::nth_element.

EDIT:

* On radius=2

C code:
Output 10000 frames in 5.83 seconds (1714.24 fps)

AVX2 code:
Output 10000 frames in 0.41 seconds (24321.61 fps)

* On radius=3

C code:
Output 10000 frames in 6.10 seconds (1638.35 fps)

AVX2 code:
Output 10000 frames in 0.52 seconds (19154.49 fps)

* On radius=4

C code:
Output 10000 frames in 9.45 seconds (1057.94 fps)

AVX2 code:
Output 10000 frames in 0.64 seconds (15681.07 fps)

[feisty2]

update:
new functionality: Buffer (https://github.com/IFeelBloated/vsFilterScript/blob/master/Buffer.hxx), a writable matrix similar to a Plane, you may access the buffer as a plane to enable automatic padding when're done writing it, useful to filters that generate intermediate representations of the input clip.

revamped: the argument extracting process now has a prettier syntax, and looks more pythonic.

to get the argument for a parameter:

param = Arguments["param"];


to get the argument for an optional parameter:

if (Arguments["param"].Exists())
param = Arguments["param"];


to get the argument for a parameter array:

for (auto x : Range{ param_arr.size() })
if (Arguments["param"][x].Exists())
param_arr[x] = Arguments["param"][x];

or

for (auto x : Range{ Arguments["param"].Size() })
param_arr[x] = Arguments["param"][x];

[feisty2]

update:
revamped: even prettier syntax for parameter arrays

you may also fetch each element for a parameter array with range-for syntax

for (auto x : Arguments["param"])
param_arr.push_back(x);


revamped: you no longer have to specify the "Name + ..." caption in your error messages, this is now automatically appended via reflection.

// this is now deprecated
Console.RaiseError(Name + ": some error message"s);

//do this instead
Console.RaiseError("some error message");

[feisty2]

I'm thinking of adding support for frame properties, but very few filters seem to utilize these and I can't find an example filter to test if my implementations are correct.

[MeteorRain]

Interlaced content? There should be some IVTC/deinterlace filters that check tff/bff.

[feisty2]

update:
bug fixes: apparently in rare cases stride / sizeof(PixelType) != width, fixed.

new functionality: ability to manipulate frame properties, you could read the properties attached to the source frame similarly to how you fetch arguments for parameters, except that you may have to manually cast the property back to its original type.

to get the "_Matrix" property:

auto matrix = static_cast<int>(srcFrame["_Matrix"]);


to get a property array:

auto prop_arr = std::vector<PropType>{};
for (auto x : srcFrame["_Prop"])
prop_arr.push_back(x);


assigning properties to the output frame is also easy!

to assign a property:

dstFrame["_Prop"] = prop_value;


to assign a property array:

for (auto x : prop_arr)
dstFrame["_Prop"] += x;


to erase a property:

dstFrame["_Prop"].Erase();


there're 3 assigning operators, operator= corresponds to paReplace, operator+= corresponds to paAppend and operator|= corresponds to paTouch.

the semantics of "someFrame[x]" is dependent on the type of x
if x is an integer, someFrame[x] is the x-th channel of the frame.
if x is a string (of type char*, const char*, std::string, std::string_view), someFrame[x] is the property x attached to the frame.

[feisty2]

update:
revamped: dropped support for multiple outputs, I've never seen a single filter that uses this feature and my implementation was incorrect anyways.

revamped: eliminated all indirect addressing

// no more the following
InputClip.Info->Width
InputFrame.Format->PlaneCount
InputClip.Info->Format->BitsPerSample

// now you have direct access to all these items
InputClip.Width
InputFrame.PlaneCount
InputClip.BitsPerSample

note that for clips with mutable format, format members are filled with garbage data at clip level

new example: Crop (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Crop.hxx), this example shows you how to write filters that modify the measures of the input (e.g. frame size) and filters that adapt to inputs with arbitrary bitdepths.

new example: Rec601ToRGB (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Rec601ToRGB.hxx), this example converts a YUV444 clip to RGB using the Rec601 matrix, it shows you how to write filters that modify the format of the input (e.g. YUV->RGB) and how to manipulate frame properties.

I think this thing is almost feature complete now.

[feisty2]

update:
new functionality: ability to call external filters before the actual filtering (DrawFrame()) begins, you may define a Preprocess() function to do things that happen before DrawFrame() is invoked. you don't need to define an empty Preprocess() if there's no preprocessing required, the wrapper will automatically detect if your filter has the Preprocess() member via hasattr reflection, if Preprocess() exists, it will be invoked. Likewise, you don't need to define DrawFrame() and its relevant functions or attributes if the filter itself does not do any filtering (something like nnedi3_rpow2), in such case, you should pass the output clip to Console with Console.Receive(), see Transpose (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Transpose.hxx) for a concrete example.

new functionality: calling external filters.

the syntax to invoke external filters is as follows:

Core["namespace"]["filter name"]("param1", argument1, "param2", argument2, ...);

the call returns a video clip, the same as what you expect in vpy scripts. you may pass any iterable container as the argument to array parameters, the wrapper will examine if the argument is a scalar or a container via reflection (thru hasattr(argument, begin)) and take care of everything.

it is now no longer possible to apply some trivial patches and compile vsFilterScript with GCC9.3, the hasattr reflection is implemented by core language features of C++20 (concepts and constraints).

[feisty2]

update:
bug fixes: fixed a bug where std::string and std::string_view were assumed to be containers when forwarded as arguments.

revamped: if you allocate a temporary plane with Buffer, each row is now guaranteed to have a 32-byte alignment. (https://github.com/IFeelBloated/vsFilterScript/blob/master/Include/Buffer.hxx#L14)

revamped: you can now alter the padding policy of a plane at runtime

frame[c].PaddingPolicy = new_policy;


new functionality: PeekFrameFormat(), useful to deal with clips with mutable format. you may use this function to determine the bitdepth of a frame before calling GetFrame<PixelType>() which relies on the bitdepth information.

[MysteryX]

This looks very interesting. It should make it easier to port Avisynth filters to VapourSynth; which I still haven't looked into!

[feisty2]

This looks very interesting. It should make it easier to port Avisynth filters to VapourSynth; which I still haven't looked into!

this thing relies heavily on c++20 features and currently, the only known compiler that has implemented all required features is the yet unreleased GCC10. you have to first compile master branch GCC then you get to play with vsFilterScript

[MysteryX]

this thing replies heavily on c++20 features and currently, the only known compiler that has implemented all required features is the yet unreleased GCC10. you have to first compile master branch GCC then you get to play with vsFilterScript
Surely Visual Studio 2025 will support it

[StainlessS]

this thing replies heavily
Too many 'p' s

yet unreleased GCC10.
I dont as yet use Linux very much, but a couple of days back I think Cinnamon Mint [EDIT: 19.3 Tricia] installed GCC10 as update.

Just saying.

EDIT: some stuff extracted from /var/log/dpkg.log

2020-04-11 22:14:35 startup archives unpack
2020-04-11 22:14:44 install gcc-10-base:amd64 <none> 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:44 status half-installed gcc-10-base:amd64 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:44 status unpacked gcc-10-base:amd64 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:44 status unpacked gcc-10-base:amd64 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:44 install gcc-10-base:i386 <none> 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:44 status half-installed gcc-10-base:i386 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:44 status unpacked gcc-10-base:i386 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:44 status unpacked gcc-10-base:i386 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:45 startup packages configure
2020-04-11 22:14:45 configure gcc-10-base:amd64 10-20200405-0ubuntu1~18.04 <none>
2020-04-11 22:14:45 status unpacked gcc-10-base:amd64 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:45 status half-configured gcc-10-base:amd64 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:45 status installed gcc-10-base:amd64 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:45 configure gcc-10-base:i386 10-20200405-0ubuntu1~18.04 <none>
2020-04-11 22:14:45 status unpacked gcc-10-base:i386 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:45 status half-configured gcc-10-base:i386 10-20200405-0ubuntu1~18.04
2020-04-11 22:14:45 status installed gcc-10-base:i386 10-20200405-0ubuntu1~18.04



EDIT: I forced a 'Refresh' for the updates to appear in Update Manager.

[MeteorRain]

SssS: Yes that's the 20200405 snapshot of GCC 10 that's still in development. I don't know why a distribution would put a software still in development on users' computer.

Clang 10 however, has just released. But that doesn't support all the features I believe.

[StainlessS]

As in above EDIT:, I forced a refresh for it to appear in Update Manager.
Thanks for the info.
EDIT: Maybe I have it set to install unstable packages or something [I remember I was looking for something that was unavailable as standard].

[feisty2]

SssS: Yes that's the 20200405 snapshot of GCC 10 that's still in development. I don't know why a distribution would put a software still in development on users' computer.

Clang 10 however, has just released. But that doesn't support all the features I believe.

I might be able to make a GCC9.3 compatible version if anyone needs it. I would have to rewrite a bunch of concepts and constraints stuff with SFINAE tho.

[feisty2]

@Are_
can you run a speed test for the latest gauss blur (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/GaussBlur.hxx) again?
I have eliminated some frequent memory reallocations which could potentially be the performance bottleneck in earlier versions

[Are_]

(vsFilterScript)
Output 100000 frames in 28.35 seconds (3527.91 fps)
(c_filter)
Output 100000 frames in 9.06 seconds (11036.62 fps)
(Convolution)
Output 100000 frames in 9.05 seconds (11045.38 fps)

This is not the same computer but it's quite similar in specs.

[feisty2]

so it wasn't reallocation either...
I guess it can only be the cost of small objects then (Plane::Proxy for each pixel)

[qyot27]

SssS: Yes that's the 20200405 snapshot of GCC 10 that's still in development. I don't know why a distribution would put a software still in development on users' computer.
Judging by some of the responses here:
https://forums.linuxmint.com/viewtopic.php?t=317303

I'm pretty sure it was due to the ubuntu-toolchain-r PPA, not the distro.

[StainlessS]

I'm pretty sure it was due to the ubuntu-toolchain-r PPA, not the distro.

Yep mine was same problem, thanx qyot.

[feisty2]

I have pinpointed the performance bottleneck, it is the indirection (Plane::Proxy and Plane::Offset)
and it shouldn't be a problem unless the filter is ultra fast, like 3x3 gauss blur. it is a constant cost for a given format and should be negligible for slow filters like NLMeans.

[feisty2]

update:
revamped: Preprocess() is now replaced by RegisterInvokingSequence() (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Transpose.hxx#L12), you may invoke a sequence of filters (including the very filter that is being defined, you may invoke this "self" filter via SelfInvoker) any way you like. basically, you get to write something like a vpy script in RegisterInvokingSequence and if this function is absent in your filter, it is assumed that the output is SelfInvoker()

say you defined a 3x3 blurring filter in DrawFrame and you wanna repeat it 10 times:

auto RegisterInvokingSequence(auto Core, auto&& SelfInvoker, auto Console) {
for (auto _ : Range{ 10 })
InputClip = SelfInvoker("clip", InputClip);
Console.Receive(InputClip);
}


you wanna upsize it by a factor of 2 after blurring.

auto RegisterInvokingSequence(auto Core, auto&& SelfInvoker, auto Console) {
for (auto _ : Range{ 10 })
InputClip = SelfInvoker("clip", InputClip);
InputClip = Core["fmtc"]["resample"]("clip", InputClip, "width", InputClip.Width * 2, "height", InputClip.Height * 2);
Console.Receive(InputClip);
}


and you wanna transpose the clip before all this happens

auto RegisterInvokingSequence(auto Core, auto&& SelfInvoker, auto Console) {
InputClip = Core["std"]["Transpose"]("clip", InputClip);
for (auto _ : Range{ 10 })
InputClip = SelfInvoker("clip", InputClip);
InputClip = Core["fmtc"]["resample"]("clip", InputClip, "width", InputClip.Width * 2, "height", InputClip.Height * 2);
Console.Receive(InputClip);
}

[feisty2]

update:
revamped: you can now call SelfInvoker() with an argument list, the calling syntax is unified with external filters. However unlike external filters, SelfInvoker does not have any error handling capacity. It is the filter developer's responsibility to ensure that only valid arguments are passed to SelfInvoker since it represents the "self" filter and it's private to the developer. The behavior is undefined (you will most likely encounter a segfault) if you call SelfInvoker with invalid arguments. all above examples have been updated to reflect the new calling syntax.

[feisty2]

update:
Transpose is now replaced by the more interesting nnedi3_rpow2 (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/nnedi3_rpow2.hxx) example. center shift correction and other features that require resampling are however not implemented for the sake of simplicity.

[feisty2]

update:
removed hasattr, it seems to fail on some corner cases.

[feisty2]

new functionality: calling python functions, the syntax is the same as invoking filters (MATLAB function call syntax)
new functionality: Frame and Function can now be map items, all VSPropTypes are now supported.
new functionality: PeekFrameFormat() for map items, if you need to convert a map item to a Frame and the PixelType of the Frame cannot yet be determined, use this function to peek the format before type conversion.

a new ModifyFrame (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/ModifyFrame.hxx)example is added to demonstrate the use of Function, a second order example for ModifyFrame (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/ModifyFrameExample.vpy) is also provided.

[feisty2]

update:
new functionality: support for filters that return multiple clips, this also affects SelfInvoker. SelfInvoker now returns an std::vector<Clip> if the filter returns multiple clips, otherwise, it still returns a Clip. Console.Receive() now accepts a Clip or a sequence of clips stored in any iterable container.
issue: support for calling external filters that return multiple clips is not yet implemented, this breaks the static type system of C++ since it is only known at runtime if an external filter returns a Clip or an std::vector<Clip>, I haven't figured out an elegant approach to deal with this issue.

new example: Palette (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Palette.hxx), this is a simple source filter showing you how to write filters that return multiple clips. it accepts an array of N numbers, and returns N blank clips, each filled with a shade of gray that corresponds to a number in the array.

x, y, z = core.test.Palette([0, 0.5, 1])
# x is filled with black, y is filled with gray, and z is filled with white

[feisty2]

can someone do a quick test and see if this thing compiles with the latest version of clang?
msvc support will soon be possible with the implementation of abbreviated templates in msvc 16.8

[feisty2]

update:
new functionality: support for calling external filters with multiple outputs.
support for filters that return multiple clips is now complete, to receive every clip returned by external filters with multiple outputs, you have to call ConfigureForMultipleOutputs(), otherwise it is assumed that the filter only has one output and only the first returned clip will be fetched.


auto ffms2 = Core["ffms2"]["Source"].ConfigureForMultipleOutputs();
auto composite = ffms2("source", "something_with_alpha_channel.mov");
auto& content_clip = composite[0];
auto& alpha_channel = composite[1];


is equivalent to, in Python


content_clip, alpha_channel = core.ffms2.Source("something_with_alpha_channel.mov")

[feisty2]

update:
revamped: various syntactic sugar for stuff dealing with multiple outputs, this affects both SelfInvoker() and the calling syntax of external filters.

for SelfInvoker():
a) if RegisterVideoInfo() returns a VSVideoInfo scalar, it is assumed that the filter has only one output, and SelfInvoker() returns a Clip.
b) if RegisterVideoInfo() returns a VSVideoInfo sequence stored in a container of dynamic length (something like std::vector), it is assumed that the filter has multiple outputs, and the number of outputs could not be determined at compile time, in this case SelfInvoler() returns an std::vector<Clip>.
c) if RegisterVideoInfo() returns a VSVideoInfo sequence stored in a container of constexpr length N (something like std::array), it is assumed that the filter has N outputs and SelfInvoker() returns an std::array<Clip, N>, in this case you can unpack the returned clip array using a structural binding:

auto [clip_1, clip_2, ..., clip_n] = SelfInvoker(...);

d) if RegisterVideoInfo() is not defined, it is obvious that SelfInvoker() will never be called and thus it doesn't matter what it may return.

for external filters:
a) if the filter is obtained by Core["namespace"]["filter"], it is assumed that the filter has only one output, and filter() returns a Clip.
b) if the filter is obtained by Core["namespace"]["filter"].ConfigureForMultipleOutputs(), it is assumed that the filter has multiple outputs, and the number of outputs could not be determined at compile time, in this case filter() returns an std::vector<Clip>.
c) if the filter is obtained by Core["namespace"]["filter"].ConfigureForMultipleOutputs(N), where N is a compile time constant, it is assumed that the filter has N outputs and filter() returns an std::array<Clip, N>, in this case you can unpack the returned clip array using a structural binding:

auto [clip_1, clip_2, ..., clip_n] = filter(...);

it is safe to specify an N that is not equal to the actual number of returned clips M. if N > M, the surplus items left in the returned array would simply be empty Clips. if N < M, only the first N clips will be fetched.



the ffms2 example above could now be rewritten in a more pythonic manner:

auto ffms2 = Core["ffms2"]["Source"].ConfigureForMultipleOutputs(2);
auto [content_clip, alpha_channel] = ffms2("source", "something_with_alpha_channel.mov");

[feisty2]

@Are_
could you compile the latest version and run a speed test for core.test.GaussBlurFast? I added a new direct access mode for maximum performance.

[Are_]

This looks really good:
Convolution:
Output 100000 frames in 9.29 seconds (10769.29 fps)

GaussBlur:
Output 100000 frames in 27.54 seconds (3631.56 fps)

GaussBlurFast:
Output 100000 frames in 9.16 seconds (10917.06 fps)

[feisty2]

update:
revamped: you may now also directly pass arbitrary number of clips to Console.Receive() as its arguments, no need to manually wrap them in a container.
removed: Buffer, it's not really that relevent and I might add it back later with various new utility stuff.
new functionality: direct frame access. when you're done sketching the prototype of your filter and ready to optimize it for speed, you should enable direct access mode by specifying the DirectAccess template parameter whenever your filter reads a frame. this exposes raw pointer access to every row of every plane of a frame. out-of-bound access detection, automatic padding and the ability to create views will all be disabled and you have to manually take care of the corner cases. with direct access mode on, vsFilterScript provides only zero cost abstraction and the execution speed of your filter will be no different from filters written directly with the low level C API.

A GaussBlurFast (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/GaussBlurFast.hxx) example is provided to demonstrate how to write filters with direct frame access, you can compare it to the earlier GaussBlur (https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/GaussBlur.hxx) example which utilizes indirect access features like automatic padding and views for cleaner and more elegant code at the cost of slower execution speed.

[feisty2]

new functionality: full integration of C++ exceptions. with exceptions, you no longer have to manually handle any of the following errors:
a) failing to invoke an external plugin (plugin does not exist)
b) failing to invoke an external filter
c) failing to invoke a python function
d) failing to invoke SelfInvoker
... and possibly many more. SelfInvoker is now allowed to throw exceptions so the earlier restriction requiring SelfInvoker to always be successfully evaluated has been removed. Any of these errors will transparently pass through your filters and propagate to a root caller like Create() (https://github.com/IFeelBloated/vsFilterScript/blob/master/include/Interface.vxx#L48) which automatically handles any error.
To you, it would be like the error does not exist so you NEVER have to worry about errors. It's now one step closer to python scripts.

Initialize() has been replaced by normal constructors because with exceptions, it is no longer required to return a value to introspect if the filter has been successfully constructed.

[feisty2]

I have nearly finished rewriting this whole thing for API v4, and I'd like to do a survey
does any one of you, including anyone that uses the vaporsynth API in external applications, use the following functions?

createCore/freeCore (implemented)
addMessageHandler/removeMessageHandler (implemented)
getFrameAsync (implemented)
createFunc (implemented)
queryCompletedFrame (deprecated)
releaseFrameEarly (deprecated)

[Myrsloik]

I have nearly finished rewriting this whole thing for API v4, and I'd like to do a survey
does any one of you, including anyone that uses the vaporsynth API in external applications, uses the following functions?

createCore/freeCore
addMessageHandler/removeMessageHandler
getFrameAsync
createFunc
queryCompletedFrame
releaseFrameEarly

createFunc/createCore/freeCore - will be used in applications that manually build graphs
addMessageHandler/removeMessageHandler - vspipe/vsedit/just about all consumers of vsscript will be using it
getFrameAsync - vspipe/vsedit/just about all applications retrieving frames
queryCompletedFrame - deprecated and 0 known uses in the wild
releaseFrameEarly - 0 known uses

[feisty2]

I have implemented the wrapper for createFunc() in the form of Core.Mount() (https://github.com/IFeelBloated/vsFilterScript/blob/master/include/Core.vxx#L114), but I'm not sure if the implementation is correct because this function is rarely used and certain cython functions relating to it seem "fishy". you're welcome to examine the code of Core.Mount or play with it if you're familiar with createFunc().

Core.Mount() could be used to mount any callable entity (function, function pointer, lambda expression, or any type that overloads operator()) compatible with any of the following signatures:

auto()
auto(auto Arguments)
auto(auto Arguments, auto Core)
// the return value may be of any type supported by VSMap, or an iterable container of any type supported by VSMap, or void

into the VS core, and the mounted function could be called in any context. for instance, you may mount a C++ function and assign the mount point to a frame property, and call the mounted function in a vpy script. the following example works as expected.


// C++ code, inside getFrame()
auto f = [](auto args) {
auto msg = static_cast<std::string>(args["msg"]);
vsapi->logMessage(VSMessageType::mtWarning, msg.data());
throw RuntimeError{ "hello world" };
};
ProcessedFrame["test"] = Core.Mount(f);


#Python script
clip = core.test.Test(clip)
clip.get_frame(0).props['test'](msg = 'aaaaa') # prints 'aaaaa', then prints 'hello world' in the error message.


you cannot currently mount non-void functions even though this functionality has been implemented, because:
a) the cython function Func.__call__ does not seem to handle the return value of the mounted function properly, the return value seems to be discarded in the process.
b) the type deduction required by Core.Mount() is so complex that it actually crashes GCC10 if you specify a non-void function to it (internal compiler error: in cp_get_fndecl_from_callee, at cp/cvt.c:1000). I'll see if this problem has been fixed in GCC11 and file a bug report if not.

[feisty2]

I updated my toolchain and the compiler bug mentioned above has been fixed in GCC trunk. I did some quick tests and Core.Mount() properly mounts any supported function-like entity, with a void or non-void return value. Any mounted function can be properly invoked in C++ programs, however you still cannot call non-void mounted functions in python and that's something only @Myrsloik can fix. The corresponding cython code has been broken since 2014 and still hasn't been fixed.

[Myrsloik]

I updated my toolchain and the compiler bug mentioned above has been fixed in GCC trunk. I did some quick tests and Core.Mount() properly mounts any supported function-like entity, with a void or non-void return value. Any mounted function can be properly invoked in C++ programs, however you still cannot call non-void mounted functions in python and that's something only @Myrsloik can fix. The corresponding cython code has been broken since 2014 and still hasn't been fixed.

YES! Your software is at my mercy! MUAHAHAHA

[feisty2]

well, I don't really have a problem with this thing being partially broken since it's rarely used anyways...
I simply wanna do some tests to verify if my implementation is correct. I'll just move on to the next function in line if there's no plan to fix this any time soon

[feisty2]

I have implemented getFrameAsync() as Node::ExpectFrame() (https://github.com/IFeelBloated/vsFilterScript/blob/master/include/Node.vxx#L90). This is yet another function that I'm not 100% sure if I have implemented correctly since I don't have a standalone VS application like vsedit to test it. Based on my quick (and evil) test which invokes this thing inside a filter's constructor, it seems to work properly. After reading the internal code of getFrame()/getFrameAsync(), I realized it's actually a lot simpler than I originally thought. I don't need to mess with threads and locks, getFrameAsync() already renders the frame in a new thread and I simply need to return a future and fulfill the promise inside the callback.

a call to Node::ExpectFrame() instantly starts rendering the requested frame in a new thread and returns a future that lets you retrieve the frame some time later when needed.

auto f = Clip.ExpectFrame(n); // instantly start rendering frame n in a new thread
... // do something else while waiting for the rendering completes
auto frame = f.get(); // now retrieve frame n, if the rendering hasn't finished yet, block the current thread until the frame is available and retrieved.

[feisty2]

I have finally implemented support for custom message loggers by year 2021
the mutex thing really got me good... (https://github.com/IFeelBloated/vsFilterScript/blob/master/include/Logger.vxx#L45)

you may send a message to all registered loggers by calling Core.DebugPrint/Print/Alert/CriticalAlert/Abort(Message), or send a message to a particular logger x by calling x.DebugPrint/Print/Alert/CriticalAlert/Abort(Message)

Core.TetherLogger() tethers a logger function to VS and returns a corresponding logger object. the logger function may be any callable entity compatible with any of the following signatures:

auto(auto Message)
auto(auto MessageType, auto Message)

"MessageType" may be of type auto/MessageTypes/VSMessageType/int, auto assumes MessageTypes
"Message" may be of type auto/const char*/std::string_view, std::string or anything constructible from const char*, auto assumes std::string_view

the logger object is reference counted and the corresponding logger function will be automatically removed from VS when the reference count drops to 0. alternatively, you may call logger x's member function x.Untether() to remove its corresponding logger function early, or x.Detach() to release the ownership of its corresponding logger function. The detached logger function will be permanently installed into VS and is removed only when the VS process terminates (API v3) or when the core is freed (API v4). you may directly add a permanent logger by calling Core.InstallLogger() which is equivalent to TetherLogger().Detach()

[feisty2]

soon this thing will have 100% coverage of the C API, once I fill in the last missing piece of the puzzle (createCore/freeCore)
yay!

[feisty2]

100% coverage (excluding 2 deprecated functions) of the C API achieved! finally!

[feisty2]

@Are_
could you run a speed test for GaussBlur and GaussBlurFast again? I applied more optimizations, mostly tags for more aggressive inlining and less indirection (so probably less cache miss)

[Are_]

Ran them three times and seemed consistent.
Old GaussBlur:
Output 100000 frames in 27.22 seconds (3674.02 fps)
Old GaussBlurFast:
Output 100000 frames in 5.92 seconds (16891.87 fps)
New GaussBlur:
Output 100000 frames in 31.10 seconds (3215.69 fps)
New GaussBlurFast:
Output 100000 frames in 5.90 seconds (16959.92 fps)

[feisty2]

it just occurred to me that I actually changed the default padding function to "reflect" in the current version, it's a bit slower but better preserves the structure of the image, and thus GaussBlur also got a bit slower. however GaussBlurFast was not affected by such change and it is indeed now a bit faster, that's enough to prove that the optimizations do work as expected :-)

[vcmohan]

The filter skeleton generator link does not exist. Get 404 error.

[lansing]

Just curious, will this works with making filter loader plugins like Vdubfilter? I tried with C API but got lost really quick.

[feisty2]

The filter skeleton generator link does not exist. Get 404 error.

it is recommended to use the stable branch for now, you can find the skeleton generator here: https://github.com/IFeelBloated/vsFilterScript/blob/stable/include/Interface.vxx

[feisty2]

Just curious, will this works with making filter loader plugins like Vdubfilter? I tried with C API but got lost really quick.

something like std.LoadPlugin? presumably yes, I can write an example if necessary

[lansing]

something like std.LoadPlugin? presumably yes, I can write an example if necessary

Like avs.LoadPlugin

[feisty2]

this thing is now feature complete and I renamed it vapoursynth++

things left to do:
code cleanup
documentation
replace header files with C++ modules

[feisty2]

I will make a few final updates to the current API v3 branch and I will then freeze the v3 branch and move on to v4

[feisty2]

MSVC support should be ready soon, considering MSVC is finally feature complete for C++20