AVSTP — A library for multithreaded plug-in development

[cretindesalpes]

AVSTP is a programming library for Avisynth plug-in developers. It helps supporting native multi-threading in plug-ins. It works by sharing a thread pool between multiple plug-ins, so the number of threads stays low whatever the number of instantiated plug-ins. This helps saving resources, especially when working in an Avisynth MT environment.

There is a low-level API using a C interface to access the shared functions of the library. There is also a set of C++ classes on the top of that, for convenient use.


Download

>>>> avstp-1.0.4.zip <<<< (https://ldesoras.fr/src/avs/avstp-1.0.4.zip)

Documentation, library and source code is included in the archive.


Short user manual

From the Avisynth user point of view, an AVSTP-enabled plug-in requires the avstp.dll file to be installed. Put it in the usual AviSynth 2.5\plugins\ directory, or load it manually with LoadPlugin("path\avstp.dll") in your script. The dll is shared between all plug-ins using AVSTP, so keep only one avstp.dll file in your plug-in set. If you're updating from a previous version, make sure that Avisynth will get access only to the latest one.

If a plug-in requiring AVSTP cannot find the dll file, it could crash, emit an error, or fall back gracefully on a single-threaded mode, depending on its design and implementation. There is no mandatory or pre-defined behaviour for such a case.

The number of threads is automatically set to the number of available logical processors. The thread count can also be controlled via an Avisynth function, so multi-threading can be disabled globally if not desired.

[mastrboy]

Good news, just hope more developers start using it so we can get a unified queue for multithreaded plugins.

Personally i hope someone would implement it into TTempsmooth :)

[VIIII]

I'd be incredibly pleased if this was used in more filters.

Even with just the ones in the dither package, I've already seen noticeable fps increases.

It's nice to be able to utilise my cpu without having to rely on the finicky beast that SetMTMode() and MT() are.

[CruNcher]

We need benchmarks evaluations (Intel, AMD) and a final solution in the end am i right that we have 5 different implementations already now for Multithreading ?

Sets = MT
LANTIS = ThreadRequest
SAPikachu = MP_Pipeline
Leimings2006 = SoraThread

First unified Plugin Approach:

cretindesalpes = Avstp

anyone i missed ?

[cretindesalpes]

avstp 1.0.1 (see first post):
Removed any floating point code from the implementation, so avstp doesn't get confused when the client code doesn't flush the FP register state after MMX operations.
It was occasionally causing deadlocks.

[tormento]

It was occasionally causing deadlocks.
Perhaps I have found the cause of locked encodings... :D

[Revgen]

Download

>>>> avstp-1.0.1.zip <<<< (http://ldesoras.free.fr/src/avstp-1.0.1.zip)



Link isn't working for me. Dither link isn't working either.

EDIT

Link works now. Thank You.

[Pat357]

Thanks for your nice piece of work !
I got it working using MVTools v2.6.0.4 and completed just a almost 2h during film with it : got zero problems !! Nice :)
Just one small question, how exactly can we manually set the number of threads being used ?

From the documentation :
avstp_set_threads (var c , number of threads)

What should I put for "var c" ? A name of a variable, a value, ...?

To get for example 4 threads being used, I've tried the following (without success) :

Added a line containing one of the following lines to the end of my script (before the latest "last"is returned)
avstp_set_threads (var c , 4)
avstp_set_threads (klm , 4)
avstp_set_threads (c , 4)
avstp_set_threads (4 , 4)
avstp_set_threads (4)

Because none of these work, it seems I have to put something there.... but what ??

Thanks in advance !

[cretindesalpes]

avstp_set_threads (4)
This one should work. If it doesn't, what is the error message? On which Avisynth version? The "var c" is mostly a workaround to make the function work even if last isn't defined.

[Gavino]

The "var c" is mostly a workaround to make the function work even if last isn't defined.
I don't see the point of that.
If c only exists so that last can be preserved, then it is unnecessary, since a function that returns an int leaves last unchanged. (last is only set when a function returns a clip.)

[tormento]

Is it possible to get the previous version? I think this one makes appear some strange glitches.

[Pat357]

It has been quite a while since the last post.... has development been stopped on AVSTP ? :scared:

[cretindesalpes]

Not really, but what would you expect to be added? It’s a simple library with a simple goal, and I think it doesn’t really need more. Maybe at the wrapping code level, to handle complex dependencies or different ways to split a frame.

[TurboPascal7]

I tried avstp lately and while I definitely like the idea, current implementation has some issues.

First - library interface is a bit dated, imho. While moving from C++11 thread pool with lambda support to avstp, I had to replace code like
for (int i = 0; i < threadPool.numberOfThreads(); ++i) {
threadPool.enqueue([=]{
prepareBuffers_op(pSrc + (offset_+i*heightPerThread)*srcPitch, buffers_, width, heightPerThread+2, srcPitch, bufferPitch_, heightPerThread / 2 * i * bufferPitch_);
});
}
with
struct RunData {
const BYTE* pSrc;
BYTE** buffers;
int width;
int height;
int srcPitch;
int bufferPitch;
int bufferOffset;
decltype(prepareBuffers_op) op;
};

std::vector<RunData> datas;

for (int i = 0; i < pool.numberOfThreads(); ++i) {
RunData d;
d.pSrc = pSrc + (offset_+i*heightPerThread)*srcPitch;
d.buffers = buffers_;
d.width = width;
d.height = heightPerThread + 2;
d.srcPitch = srcPitch;
d.bufferPitch = bufferPitch_;
d.bufferOffset = heightPerThread / 2 * i * bufferPitch_;
d.op = prepareBuffers_op;
datas.push_back(d);
}

for (int i = 0; i < pool.numberOfThreads(); ++i) {
pool.enqueue(dispatcher_, [](avstp_TaskDispatcher *dispatcher, void *userData){
auto data = reinterpret_cast<RunData*>(userData);
data->op(data->pSrc, data->buffers, data->width, data->height, data->srcPitch, data->bufferPitch, data->bufferOffset);
}, &datas[i]);
}

While lambdas support isn't something you can't live without, it would be definitely nice to have.

Second - avstp is slow. I get about 10-15% fps drop compared to a simple implementation like https://github.com/progschj/ThreadPool. I'm not sure what is the reason for this slowdown but it's here and it's really annoying. Maybe I'm doing something wrong in the code above, but I don't think it's the case.

And a personal question - why are you reimplementing a lot of things that have been available in different compilers/boost for ages? I do appreciate extremely easy compilation process of your plugins, but most of avstp code looks like a rewrite of boost to me.

-------
Okay, nevermind the performance part, but your API is just way too confusing.
When called avstp_set_threads(4), thread pool will actually have 3 threads. This is good because we also have the main thread, but I didn't find any mention of this in the documentation. At the same time avstp_get_nbr_threads (also, please rename to "number_of_threads" or "thread_count", nbr is just bad) returns 4, leading to the wrong assumption that there are actually 4 threads in the pool and hence incorrect workload distribution. You should either make this part correct at API level (return actual number of threads in the pool) or note it in the documentation.

[cretindesalpes]

While lambdas support isn't something you can't live without, it would be definitely nice to have.
I agree. I haven’t any C++11-compatible compiler installed at the moment so I cannot develop it, but it would simplify the writing, indeed.

Second - avstp is slow. I get about 10-15% fps drop compared to a simple implementation
I don’t know why. The thread synchronization is done by calling WaitForSingleObjectEx() and ReleaseSemaphore(). Do faster synchronization primitives exist in the modern Windows APIs? I don’t know what std::condition_variable uses on your compiler. But this shouldn’t make any noticeable difference with coarse-grained tasks.

And a personal question - why are you reimplementing a lot of things that have been available in different compilers/boost for ages? I do appreciate extremely easy compilation process of your plugins, but most of avstp code looks like a rewrite of boost to me.
Some of theses primitives are actually quite old and I use them for a long time. I’m comfortable with them and I don’t see any reason to switch to a complicated external library.

Okay, nevermind the performance part, but your API is just way too confusing.
When called avstp_set_threads(4), thread pool will actually have 3 threads. This is good because we also have the main thread, but I didn't find any mention of this in the documentation. At the same time avstp_get_nbr_threads (also, please rename to "number_of_threads" or "thread_count", nbr is just bad) returns 4, leading to the wrong assumption that there are actually 4 threads in the pool and hence incorrect workload distribution. You should either make this part correct at API level (return actual number of threads in the pool) or note it in the documentation.
When avstp_set_threads() is called with 4, there are exactly 4 working threads, because the main thread is also a working thread. Check the individual thread load with Process Explorer to make sure. I’ll mention it in the documentation. Maybe this misunderstanding is the cause of the slowdown you noticed previously?

But why “nbr” is bad?

[TurboPascal7]

When avstp_set_threads() is called with 4, there are exactly 4 working threads, because the main thread is also a working thread. Check the individual thread load with Process Explorer to make sure. I’ll mention it in the documentation. Maybe this misunderstanding is the cause of the slowdown you noticed previously?

Yes, this was the cause of the slowdown, that's why I said "nevermind the performance part". Basically I allocated one more task to avstp threads running on the assumption that number of threads returned by avstp == number of threads in the pool.

I still think it's an API problem. Avstp is a thread pool - nothing more. Whatever a user is doing with his main thread should not concern it. When a user calls get_nbr_threads in avstp, it expects to get number of threads in avstp, not number of threads + 1 because it assumes that you're doing some work in the main thread. I think the fact that I spent some time using this library in a wrong way speaks for itself.

Or maybe it's just me and having more opinions would help.

But why “nbr” is bad?
It's just a bad naming practice. Calling it "number" won't make your program slower, but will make reading it a bit easier. Of course, in my opinion.

[cretindesalpes]

I still think it's an API problem. Avstp is a thread pool - nothing more. Whatever a user is doing with his main thread should not concern it. When a user calls get_nbr_threads in avstp, it expects to get number of threads in avstp, not number of threads + 1 because it assumes that you're doing some work in the main thread.

No, it doesn’t work exactly like a regular thread pool as you assumed. What happens in the main thread when avstp_wait_completion() is called is not up to the user; the main thread becomes the last worker thread, whereas it would just wait for the other threads if implemented as a naive thread pool.

But these are implementation details. For a client application, the important thing to consider is the number of simultaneous working threads. I would find much more confusing that get_nbr_threads() would return something different from the actual parallel processing capability.

[kotuwa]

>>>> avstp-1.0.1.zip <<<< (http://ldesoras.free.fr/src/avs/avstp-1.0.1.zip)
Will this work for x64 dither.dll ?
Or is there another 64 bit version of avstp somewhere else?
or no support for x64 ?

[cretindesalpes]

avstp 1.0.3 (http://forum.doom9.org/showthread.php?t=164407):
Recompiled with MSVC 2013 with various internal updates. API not changed at the moment.
64-bit support.

[Groucho2004]

Quick test with your mvtools2 2.6.0.5 and avstp 1.0.3 vs. 1.0.1 (i5 2500K @ 4GHz, 4 cores):

Script:
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1).trim(0, 199)
super = MSuper()
multi_vec = MAnalyse (super, multi = true, delta = 9)
MDegrainN(super, multi_vec, 9, thSAD = 400, thSAD2 = 150)


Result with 1.0.1:
Frames processed: 200 (0 - 199)
FPS (min | max | average): 3.196 | 8.265 | 4.213
Memory usage (phys | virt): 539 | 538 MB
Thread count: 4
CPU usage (average): 86%

Result with 1.0.3:
Frames processed: 200 (0 - 199)
FPS (min | max | average): 3.203 | 8.198 | 4.143
Memory usage (phys | virt): 538 | 537 MB
Thread count: 8
CPU usage (average): 87%


Avstp 1.0.3 creates 4 additional threads compared to 1.0.1. Using "avstp_set_threads(16)" for example would create 20 threads as opposed to 16 with 1.0.1. I suppose this is by design?

Also, the speed/efficiency is slightly worse with the new version.

[cretindesalpes]

I’ve got more or less the same performances and thread counts in both versions (i7 4770K, 4 cores 8 threads).

Old 1.0.1:
Frames processed: 200 (0 - 199)
FPS (min | max | average): 3.366 | 9.113 | 4.687
Memory usage (phys | virt): 555 | 553 MB
Thread count: 14
CPU usage (average): 81%

New 1.0.3:
Frames processed: 200 (0 - 199)
FPS (min | max | average): 3.407 | 8.902 | 4.642
Memory usage (phys | virt): 554 | 552 MB
Thread count: 14
CPU usage (average): 82%


Forcing avstp_set_threads(10):

Old 1.0.1:
Frames processed: 200 (0 - 199)
FPS (min | max | average): 3.286 | 7.798 | 3.997
Memory usage (phys | virt): 555 | 553 MB
Thread count: 16
CPU usage (average): 65%

New 1.0.3:
Frames processed: 200 (0 - 199)
FPS (min | max | average): 3.270 | 7.877 | 3.948
Memory usage (phys | virt): 554 | 552 MB
Thread count: 16
CPU usage (average): 65%

Note that in all cases, a few threads are created (https://stackoverflow.com/questions/2484921/strange-threads-in-application-in-win7-wow64) after the processing of 50–100 frames, not directly related to the avstp internal thread pool. I guess they are created by the VS2013 runtime library because of the use of some synchronisation primitives.

[Groucho2004]

Note that in all cases, a few threads are created (https://stackoverflow.com/questions/2484921/strange-threads-in-application-in-win7-wow64) after the processing of 50–100 frames, not directly related to the avstp internal thread pool. I guess they are created by the VS2013 runtime library because of the use of some synchronisation primitives.
Yes, checking with Process Explorer shows the same number of avstp threads for both versions. For 1.0.3, some ntdll threads are created.
But why does AVSMeter show the same number of threads for both versions in your setup?

[edcrfv94]

Kaspersky slow avstp.dll it is Trojan.Win32.Clicker.mj
Look like it is False positive?

[SSH4]

Kaspersky slow avstp.dll it is Trojan.Win32.Clicker.mj
Look like it is False positive?

Trow away this slow monster (Kaspersky).
Use VirusTotal uploader for checking files.
https://www.virustotal.com/en/file/48778a61cd4fdbb557b800a699e1e1d4b45b85ab1f302b74d8972c8ecf1608fc/analysis/1455158866/

[bilditup1]

Put it in the usual AviSynth 2.5\plugins\ directory, or load it manually with LoadPlugin("path\avstp.dll") in your script.

When I try manual loading, I get this:

LoadPlugin: unable to load "C:\blu\MeGUI\tools\avisynth_plugin\cretindesalpes\avstp.dll", error=0xc1

Putting avstp.dll in the "C:\Program Files (x86)\AviSynth 2.5\plugins\" and having it autoload appears to avoid this error. But, I am not actually sure if it's loaded or not though.

ED: I was attempting to use avstp with AviSynth MT 2.60, which is apparently stupid and pointless (https://www.doom9.org/showthread.php?p=1730980#post1730980). My mistake.

ED2: I was mistakenly attempting to use the X64 version with 32-bit AviSynth (both ST and MT) and it appears that's what caused the error. Loading the one with the proper architecture (duh) avoids this nonsense.

[cretindesalpes]

For some reason, I just updated the internal code and compiled AVSTP with modern tools. From the end user point of view, there shouldn’t be any significant difference with the previous version, excepted that Win XP is not supported anymore (I guess this isn’t a big deal nowadays).

See the first post for download.

[real.finder]

For some reason, I just updated the internal code and compiled AVSTP with modern tools. From the end user point of view, there shouldn’t be any significant difference with the previous version, excepted that Win XP is not supported anymore (I guess this isn’t a big deal nowadays).

See the first post for download.

can this take benefit of internal pool of Avs+? https://forum.doom9.org/showthread.php?p=1777021#post1777021 and https://forum.doom9.org/showthread.php?p=1778346#post1778346

[cretindesalpes]

real.finder:
I wasn’t aware of it. Using the Avs+ threadpool is probably more logical, but supporting two APIs in each filter is a lot of work.

[real.finder]

but supporting two APIs in each filter is a lot of work.

what about make AVSTP use it (act like a warper for avs+ one) if that possible? if it possible then maybe making special version for avs+ or maybe making AVSTP see if it avs+ or not to either use it own or avs+ one

[FranceBB]

For some reason, I just updated the internal code and compiled AVSTP with modern tools. From the end user point of view, there shouldn’t be any significant difference with the previous version, excepted that Win XP is not supported anymore (I guess this isn’t a big deal nowadays).

See the first post for download.

It kinda is for some people... :(
That's too bad, in fact the new version of the package produces an access violation on XP.
Is it just a compilation issue or did you change something complex internally not to make it XP compatible?
Would you be so kind to re-compile with the v141_xp?

[StainlessS]

From the end user point of view, there shouldn’t be any significant difference with the previous version, excepted that Win XP is not supported anymore
Well, as Cretindesalpes rarely visits any more, and only significant difference being that it dont work with XP,
just continue using the old version, thats what I would be doing.

EDIT: Cretindesalpes probably only updated his project files for new compiler [and dumped the older compiler],
just so it would compile properly, probably just done for 'uniformity' of his projects.

[FranceBB]

I see...
Fair enough, I'll stick with the older one as I've been doing so far.
Time to go to bed now, though, it's 4AM over here...

[kedautinh12]

mvtools bug with avstp.dll here:
https://forum.doom9.org/showthread.php?p=1942490#post1942490

[real.finder]

mvtools bug with avstp.dll here:
https://forum.doom9.org/showthread.php?p=1942490#post1942490

I think this is wrong link, you mean this https://forum.doom9.org/showthread.php?p=1942510#post1942510 ?

anyway, if it not avstp bug (mostly (https://forum.doom9.org/showthread.php?p=1730914&highlight=avstp#post1730914)) this should be reported to pinterf since he do update mvools recently, the last login of pinterf was from 2 weeks ago I think, lets hope he is fine

[tormento]

Is it possible to have an updated list of the plugins (w/versions) that work with AVSTP.DLL?

Thanks.

[pinterf]

Hi, I saw you have fmtconv on git, will it happen to avstp as well?
I found some issues with actual 1.04 (both in linux/gcc compatibility and a threading issue), just asking if we can discuss it on git "issues" or "discussions" section or here (or in PM)

[FranceBB]

I found some issues with actual 1.04

and let's also remember the "frozen as ice" thingy (https://github.com/pinterf/mvtools/issues/46) caused by avstp...

[cretindesalpes]

Hi, I saw you have fmtconv on git, will it happen to avstp as well?
I have no plan for this at the moment, but if there is a request I could add a repository. However I’m not sure about the proper way to manage the common libraries I share between my different projects. I’m far from being a git expert :D
I found some issues with actual 1.04 (both in linux/gcc compatibility and a threading issue), just asking if we can discuss it on git "issues" or "discussions" section or here (or in PM)
Of course, here or on Github if you like it better. I wasn’t aware of the freeze issue, I would try to help if I could get some “minimal steps to reproduce”.

[kedautinh12]

and let's also remember the "frozen as ice" thingy (https://github.com/pinterf/mvtools/issues/46) caused by avstp...

It's can fix now, check AVSTP v.1.0.4.1
https://github.com/pinterf/AVSTP/releases

[FranceBB]

Outstanding work, Ferenc!
Issue closed! ;)

[kedautinh12]

Outstanding work, Ferenc!
Issue closed! ;)

I wonder are you check work correctly???

[FranceBB]

I wonder are you check work correctly???

So far so good, but I only run a few tests.
I'll be back to work on February 'cause I had to take an exam as part of my master degree in Computer Science Engineering (Mathematical modelling) and I really didn't want to leave something like this in production while I wasn't there.
The stress test will be in February then, when I'll let it run with a series of Remaster workflows. ;)