KNLMeansCL: OpenCL NLMeans de-noising algorithm [2018-01-29]

[Magik Mark]

For CPU use, Is there a way to use multiple threads? I have a 16C/28T Xeon. Will this speed it up? Faster than the GPU?

[Khanattila]

Can anybody explain how the parameter "rclip" works?

Through my test, it seems that the weighting calculation is done not only on reference clip, but also on the source clip, if "rclip" is specified as a clip which is different to the source clip.

Besides, if the "rclip" is specified the same as the source clip, the result is also different to not specifity any clip in "rclip". Why there exists difference?

I tried to find more details in the source code, but failed, because of my poor ability in programming.
This is something that should not happen. And I should check.

[Khanattila]

For CPU use, Is there a way to use multiple threads? I have a 16C/28T Xeon. Will this speed it up? Faster than the GPU?

The plugin uses all available resources.
However, a graphics card is faster I think.

[MysteryX]

I've seen several people mentioning that they would rather use xNLMeans to do their computation on a server where the GPU isn't available.

I just had an idea and did a quick search...

It *IS* possible to have GPU Cloud Computing! (http://www.nvidia.com/object/gpu-cloud-computing-services.html)

[asarian]

The plugin uses all available resources.
However, a graphics card is faster I think.

Earlier, Groucho2004 and I loosely established that OpenCL via CPU-only (not with onboard graphics, of course), is roughly 80x slower than using a modern GPU. Aka, completely unusable for all daily purposes and intent. :)

Speaking of which, stll planning to buy this for my ESXi server:

https://www.asus.com/Graphics-Cards/GTX750TIPH2GD5/overview/

Peculiarly enough, ASUS itself doesn't list its OpenCL support. A German site said OpenCL 1.1 (that's no good); others say 1.2 (which is good, of course). It's a GTX 750, though, so, can I safely assume it will work with KNLMeansCL?

[Groucho2004]

Peculiarly enough, ASUS itself doesn't list its OpenCL support. A German site said OpenCL 1.1 (that's no good); others say 1.2 (which is good, of course). It's a GTX 750, though, so, can I safely assume it will work with KNLMeansCL?The GTX-750 supports OpenCL 1.2. The minimum driver version for OCL1.2 is as far as I recall 347.xx (350.xx for sure). If you're not interested in games and stuff, the plain GTX-750 (non-Ti) is just as fast.

[asarian]

Thanks, Groucho2004. :)

[asarian]

The GTX-750 supports OpenCL 1.2. The minimum driver version for OCL1.2 is as far as I recall 347.xx (350.xx for sure). If you're not interested in games and stuff, the plain GTX-750 (non-Ti) is just as fast.


Yeah, btw, this is pure for KNLMeansCL OpenCL rendering on my ESXi VM. Gonna set the card pass-thru, and Bob's your uncle. :)

[MysteryX]

I have a feature request if a new version gets developed.

You recommended not to use MT with KNLMeans. One problem with single-thread GPU usage is that the CPU and GPU can't work fully at the same time. AviSynth waits for KNLMeans to finish its GPU work before continuing and asking other filters to do their CPU work, and only request another frame from the GPU once the CPU work is done.

A way to work around this is to implement it as MT_NICE_FILTER. Still a single instance gets created but you can get several calls at once. Then, you can prevent parallel execution by using a unique_lock. This creates a very simple form of work queue. 4 or 8 threads can be working fully and you'll serve them when the OpenCL engine is free.

From my implementation of AviSynthShader, I also found that I got best performance (full GPU usage) by creating 2 engines and alternating between them; but that's something to test.

[tormento]

You recommended not to use MT with KNLMeans.

On AviSynth 2.6 and pinterf mod of AVS+, I can get it to work with 6/8 threads, rock solid ;)

(yeah... image corruption is another topic)

[MysteryX]

On AviSynth 2.6 and pinterf mod of AVS+, I can get it to work with 6/8 threads, rock solid ;)

(yeah... image corruption is another topic)
It "works" but he still recommended not to do it. I'm not exactly sure of the details why.

[asarian]

I have a feature request if a new version gets developed.

You recommended not to use MT with KNLMeans. One problem with single-thread GPU usage is that the CPU and GPU can't work fully at the same time.

In VapourSynth, I always find my CPU fully saturated (because x264 fills up the 'idle' spots, primarily). So, even if I could run multiple GPU-threads, and have my GPU run at full load too (which it isn't now), then yes, vspipe would likely take near 100% CPU too, but leave less for x264, is my point. So, I'm not even sure you'd win much on the deal, overall.

[Khanattila]

It "works" but he still recommended not to do it. I'm not exactly sure of the details why.

I do not recommend it because it creates CPU overhead.
In addition, each thread creates a new OpenCL environment.

[MysteryX]

OK then it works but it's the same situation as I had.

Here are some tests


colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
KNLMeansCL(d = 2, a = 2)


FPS (min | max | average): 1.286 | 1.391 | 1.308
Memory usage (phys | virt): 134 | 183 MiB
Thread count: 9
CPU usage (average): 11%
OK


SetFilterMtMode("KNLMeansCl", 2, true)
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
KNLMeansCL(d = 2, a = 2)
Prefetch(2)


FPS (min | max | average): 0.477 | 10.41 | 1.356
Memory usage (phys | virt): 317 | 399 MiB
Thread count: 11
CPU usage (average): 13%
3.7% performance boost


colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
KNLMeansCL(d = 2, a = 2)
nnedi3_rpow2(4, nns=4)


FPS (min | max | average): 0.855 | 1.030 | 0.923
Memory usage (phys | virt): 400 | 557 MiB
Thread count: 105
CPU usage (average): 24%
CPU and GPU are waiting for each other.


SetFilterMtMode("KNLMeansCl", 3, true)
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
KNLMeansCL(d = 2, a = 2)
nnedi3_rpow2(4, nns=4)
Prefetch(2)


FPS (min | max | average): 0.426 | 1.324 | 1.207
Memory usage (phys | virt): 705 | 862 MiB
Thread count: 107
CPU usage (average): 34%

30.8% better


SetFilterMtMode("KNLMeansCl", 2, true)
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
KNLMeansCL(d = 2, a = 2)
nnedi3_rpow2(4, nns=4)
Prefetch(2)


FPS (min | max | average): 0.625 | 1.339 | 1.204
Memory usage (phys | virt): 861 | 1051 MiB
Thread count: 107
CPU usage (average): 36%
Same, but higher memory usage.

You're having the same situation that I faced with AviSynthShader. The main issue being the CPU and GPU waiting for each other instead of both working at the same time. The second issue is 3.7% underutilisation of the GPU due to memory transfers and overheads.

Here we get 30% performance improvements when using NNEDI3, and with more CPU-intensive scripts, the difference will be even greater.

You might want to take a look at how I implemented alternating between 2 engines running in MT_NICE_FILTER. (https://github.com/mysteryx93/AviSynthShader/blob/master/Src/ExecuteShader.cpp#L36) That way, 8 threads can be working with the CPU and the GPU work waits until either of the 2 engines is ready to process the next frame. This gave a serious performance boost.

Only thing to keep in mind is that the calls come from various threads and the engine must be multithreading-safe, yet called by only one thread at a time. The GPU work itself must be queued and protected by a unique_lock, but the memory transfers back and forth may not need to be locked. I tried to minimize the area that was locked. While I'm transferring back from the GPU to the CPU and into AviSynth, the next frame is already being pushed into the engine.

[Groucho2004]

The second issue is 3.7% underutilisation of the GPU due to memory transfers and overheads.
I don't know where your obsession with memory bottlenecks comes from. Here (http://www.bealto.com/gpu-benchmarks_mem.html) is a good article about device/device and host/device memory performance. At a worst case scenario of 5GB/second, how many frames/sec. do you think can be copied from the video card to the main memory?

[Groucho2004]

MysteryX:
Your tests and conclusions are utterly flawed. A few pointers:
1. The Avisynth filter chain cannot be faster than its slowest filter.
2. Multi-threading KNLMeansCL doesn't give better performance, no matter what
3. Don't multi-thread internally multi-threaded filters like nnedi3, the performance will be worse

Re. (2):
Maybe you realize the truth of that statement when you run 2 separate instances of a simple script containing KNLMeansCL with AVSMeter simultaneously - The speed of each script will be exactly half.

Re. (3):
Compare the speed of these two scripts:
colorbars(width = 1920, height = 540, pixel_type = "yv12").killaudio().assumefps(25, 1)
NNEDI3(dh = true, nsize = 0, nns = 4, qual = 2)

SetFilterMTMode("DEFAULT_MT_MODE", 2)
colorbars(width = 1920, height = 540, pixel_type = "yv12").killaudio().assumefps(25, 1)
NNEDI3(dh = true, nsize = 0, nns = 4, qual = 2, threads = 1)
Prefetch(4) #set to the number of logical cores of your CPU

[MysteryX]

The issue is very simple. In single-threaded mode, each filter receive a call to GetFrame. Every other filter has to wait until it returns from that GetFrame function.

When calling CPU-intensive functions like NNEDI3, that's fine. If it is a regular filter, the only limitations is that it can only use 1 core (out of 8 cores means 12.5% of CPU). NNEDI3 handling multi-threading makes it work well.

The problem is when calling functions that don't use the CPU but instead wait for the GPU, such as AviSynthShader, KNLMeans or any other GPU-bound plugin.

The plugin then receives a call from GetFrame, and instead of using the CPU, it *waits* for the GPU to finish, and all other plugins that could use the CPU also have to wait.

Then we get the data back from the GPU, the processing moves on to the next plugin and it does CPU work. While the CPU is working, the GPU is iddle; KNLMeans won't receive the next request until the other filters are done and AviSynth is ready to process the next frame.

Thus, you'll get something along the lines of 50% GPU usage and 50% CPU usage.

Unless I'm missing something? I've done plenty of testing and coding around that.



2. Multi-threading KNLMeansCL doesn't give better performance, no matter what

I got 3.7% improvement by running 2 instances. Not a big improvement, but it's an improvement. I suspect the difference will be bigger if running with lower settings and processing more frames per second.


3. Don't multi-thread internally multi-threaded filters like nnedi3, the performance will be worse

In my test results, threads overhead is irrelevant because CPU usage never went above 36%

Here's another test

colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
KNLMeansCL(d = 0, a = 1)


FPS (min | max | average): 9.324 | 29.33 | 22.42
Memory usage (phys | virt): 102 | 148 MiB
Thread count: 12
CPU usage (average): 10%


SetFilterMtMode("KNLMeansCl", 2, true)
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
KNLMeansCL(d = 0, a = 1)
Prefetch(2)


FPS (min | max | average): 10.12 | 55677 | 29.24
Memory usage (phys | virt): 203 | 265 MiB
Thread count: 14
CPU usage (average): 14%
Just as expected, this time I got 30.4% higher performance with 2 threads! (actually it's more than I expected)

4 threads should yield no further benefits

SetFilterMtMode("KNLMeansCl", 2, true)
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
KNLMeansCL(d = 0, a = 1)
Prefetch(4)


FPS (min | max | average): 5.187 | 155896 | 30.05
Memory usage (phys | virt): 303 | 382 MiB
Thread count: 16
CPU usage (average): 13%
Yet it still gives another 2.8% increase. To me that's an indication that the implementation isn't optimal. This should not give any more benefit than 2 threads.

On a side-note, my Radeon 7670M gets slain here on that last script. 47% slower than my Intel HD 4000 !??

FPS (min | max | average): 2.680 | 167032 | 18.88
Memory usage (phys | virt): 128 | 197 MiB
Thread count: 16
CPU usage (average): 29%

[asarian]

While the CPU is working, the GPU is iddle; KNLMeans won't receive the next request until the other filters are done and AviSynth is ready to process the next frame.

Thus, you'll get something along the lines of 50% GPU usage and 50% CPU usage.

Unless I'm missing something? I've done plenty of testing and coding around that.


I'd say the part you're missing, is that you keep considering only 2 concurrent processes: CPU on AVS/VPS, vs. KNLMeansCL on GPU. Whilst, in fact, there's a third process: x264 (unless you're doing just a synthetic test, and are discarding the output).

So, to re-iterate, what idle time you 'lose' on your CPU isn't really lost at all, but being usurped by your x264 process (greatly speeding up your general encode, of course).

Only if throughput would be so dismal that your x264 process would effectively become starved, would you ever experience hinder from the lack of full CPU utilization from AVS/VPS. That has, with HD material, never been the case for me.

[Groucho2004]

Just as expected, this time I got 30.4% higher performance with 2 threads!
I can't reproduce these results, maybe it depends on the graphics chip, driver, etc. It just shows how complex the whole thing is.

I'll stick to the authors recommendation to not multi-thread this filter, not only because there's no benefit in my case.

[MysteryX]

Only if throughput would be so dismal that your x264 process would effectively become starved, would you ever experience hinder from the lack of full CPU utilization from AVS/VPS. That has, with HD material, never been the case for me.
Encoding with x265 Medium preset gives smaller file, better results and better performance than x264 with veryslow preset. The AVS script is generally much heavier than that.

In my case, the CPU very rarely goes above 80%. When running SMDegrain(prefilter=4), I've been stuck with a ~35% CPU usage, but now I see part of it is because of running KNLMeans with MT mode 3 that is a performance killer. Just switching it to MT mode 2 will probably help.

That being said, I'm running it on an Intel HD 4000 (because it doesn't like my Radeon 7670M) while my CPU is just as powerful as brand new laptops you buy now 4 years later. The CPUs now are kind of catching up, but a year or two ago the CPU went down into much lower performance than the one I got back then. If you have a slower CPU with 4 cores, then you might saturate it more easily than my i7 with 8 cores.

My CPU is a Intel Core i7-3630QM @ 2.40GHz. It ranks 7,618 here. (https://www.cpubenchmark.net/laptop.html) 4 years later, it's still hard to find laptops with better CPUs, but now I just saw Intel Core i7-6700HQ @ 2.60GHz is becoming popular. Slightly better: 8,044. Still lots of laptops come with Intel Core i5-6200U; nearly twice slower.

Perhaps that's why I'm putting more attention on MT than others.

[MysteryX]

Here's another test with SMDegrain. prefilter=4 is KNLMeans


colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
SMDegrain(prefilter=4)

FPS (min | max | average): 0.887 | 2.063 | 1.782
Memory usage (phys | virt): 477 | 532 MiB
Thread count: 9
CPU usage (average): 11%



SetFilterMtMode("KNLMeansCl", 3, true)
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
SMDegrain(prefilter=4)
Prefetch(4)

Frames processed: 105 (0 - 104)
FPS (min | max | average): 0.467 | 4.370 | 2.954
Memory usage (phys | virt): 748 | 804 MiB
Thread count: 16
CPU usage (average): 31%


SetFilterMtMode("KNLMeansCl", 2, true)
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
SMDegrain(prefilter=4)
Prefetch(4)

FPS (min | max | average): 0.456 | 4.358 | 3.052
Memory usage (phys | virt): 1191 | 1319 MiB
Thread count: 16
CPU usage (average): 31%


We're dealing with 2 issues.
1. CPU and GPU waiting for each other.
2. GPU not being fully utilized due to overheads

Here we see that problem #1 is already handled quite well with MT mode 3.

As for problem #2, it is only making a 3.3% difference here.

One thing that stands out is that MT is a must when working with the GPU. If MT is a must, then better as well develop such plugins with support for MT mode 1 and handle the multi-threading optimally.

I can't reproduce these results, maybe it depends on the graphics chip, driver, etc. It just shows how complex the whole thing is.

I'll stick to the authors recommendation to not multi-thread this filter, not only because there's no benefit in my case.
If you run these scripts, I'm sure you'll get the same difference between test #1 and #2. It's technically impossible that you don't.


Test #1 is close to CPU limit of 1 core which can distort the result. Here's another way to look at it where the CPU still has legal room. Here's the difference between 2 and 4 threads with MT mode 3. With 4 threads, the CPU vs GPU waiting time is reduced. With 2 threads, the CPU is well below 25%.

SetFilterMtMode("KNLMeansCl", 3, true)
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
SMDegrain(prefilter=4)
Prefetch(2)

FPS (min | max | average): 0.631 | 3.491 | 2.812
Memory usage (phys | virt): 617 | 672 MiB
Thread count: 14
CPU usage (average): 19%


SetFilterMtMode("KNLMeansCl", 3, true)
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(25, 1)
SMDegrain(prefilter=4)
Prefetch(4)

FPS (min | max | average): 0.477 | 5.034 | 3.171
Memory usage (phys | virt): 769 | 825 MiB
Thread count: 16
CPU usage (average): 28%
That's a 12.8% difference between 2 and 4 threads.

[Groucho2004]

If you run these scripts, I'm sure you'll get the same difference between test #1 and #2. It's technically impossible that you don't.
I suggest you think about that last statement while taking into consideration the complexity of the processes involved and the differences between our setups (HW & SW).

[MysteryX]

I suggest you think about that last statement while taking into consideration the complexity of the processes involved and the differences between our setups (HW & SW).
Just run the scripts. It takes 2 minutes.

Your setup might change the outcome of problem #2, but problem #1 will remain just the same.

[asarian]

Encoding with x265 Medium preset gives smaller file, better results and better performance than x264 with veryslow preset. The AVS script is generally much heavier than that.

In my case, the CPU very rarely goes above 80%. When running SMDegrain(prefilter=4), I've been stuck with a ~35% CPU usage, but now I see part of it is because of running KNLMeans with MT mode 3 that is a performance killer. Just switching it to MT mode 2 will probably help.

(...)

Perhaps that's why I'm putting more attention on MT than others.


I see. Thanks for the clarification. I am not using x265 yet, so I'll take your word for it. :)

[Groucho2004]

Just run the scripts. It takes 2 minutes.
I already ran a whole bunch of tests with KNLMeansCL.
Remember, we're discussing multi-threading of a single fiilter (KNLMeansCL). You're testing SMDegrain which introduces a gigantic shitload of complexity and will certainly mask any meaningful results related to KNLMeansCL. I'll stand by my previous statement - your tests are flawed and your conclusions logical fallacies.

[MysteryX]

If you want to stand by beliefs instead of by results, then there's no point in discussing any of this. Beliefs=fiction. Results=reality. I'm only interested in reality.

The principle here is very simple. There's no "complex stuff". With a single thread, calls to GetFrame area linear. The next filter won't receive the call until the previous filter returns. This causes the CPU and the whole chain to *wait* while the GPU is working even through the CPU is still available and both could be working at the same time. You will not see this if you're not doing any CPU work besides KNLMeans.

Please don't post that "what I'm saying is crap" until you've done your basic homeworks and made this simple test yourself. Otherwise you're wasting everybody's time. THANKS!

[Groucho2004]

Beliefs=fiction. Results=reality. I'm only interested in reality.I fully agree.

[MysteryX]

If someone else wants to take 2 minutes to run the 3 test scripts to confirm what I shared, you can post your results.

[Khanattila]

I have a feature request if a new version gets developed.

You recommended not to use MT with KNLMeans. One problem with single-thread GPU usage is that the CPU and GPU can't work fully at the same time. AviSynth waits for KNLMeans to finish its GPU work before continuing and asking other filters to do their CPU work, and only request another frame from the GPU once the CPU work is done.

A way to work around this is to implement it as MT_NICE_FILTER. Still a single instance gets created but you can get several calls at once. Then, you can prevent parallel execution by using a unique_lock. This creates a very simple form of work queue. 4 or 8 threads can be working fully and you'll serve them when the OpenCL engine is free.

From my implementation of AviSynthShader, I also found that I got best performance (full GPU usage) by creating 2 engines and alternating between them; but that's something to test.

Or let AviSynth die.

[Khanattila]

If you want to stand by beliefs instead of by results, then there's no point in discussing any of this. Beliefs=fiction. Results=reality. I'm only interested in reality.

The principle here is very simple. There's no "complex stuff". With a single thread, calls to GetFrame area linear. The next filter won't receive the call until the previous filter returns. This causes the CPU and the whole chain to *wait* while the GPU is working even through the CPU is still available and both could be working at the same time. You will not see this if you're not doing any CPU work besides KNLMeans.

Please don't post that "what I'm saying is crap" until you've done your basic homeworks and made this simple test yourself. Otherwise you're wasting everybody's time. THANKS!

The only thing I notice is the filter can not saturate the GPU. And that's why you get the best results by invoking it multiple times.

[MysteryX]

The only thing I notice is the filter can not saturate the GPU. And that's why you get the best results by invoking it multiple times.
That's problem #2. Problem #1 has more impact.

The VapourSynth version probably behaves the same way.

[Khanattila]

That's problem #2. Problem #1 has more impact.

The VapourSynth version probably behaves the same way.

VapourSynth version is fmParallelRequests:
For filters that are serial in nature but can request in advance one or more frames they need. A filter’s “getframe” function will be called from multiple threads at a time with activation reason arInitial, but only one thread will call it with activation reason arAllFramesReady at a time.

And I should make a mess to make it work even for AviSynth?

[Khanattila]

The talk of freeing the CPU for other plugin is simply nonsense in any case.
KNLMeansCL can not release a frame that has not yet finished processing.
While the plugin works the other plugins wait. And that's how it should be.

[MysteryX]

So you did optimize it for VapourSynth's multi-threading.

If the OpenCL engine is well encapsulated into its own class, it's not difficult to handle the muti-threading in AviSynth.

1. Simply adding a unique_lock within GetFrame is generally enough to convert it from a MT_SERIALIZED to a MT_NICE_FILTER

2. Ensure the OpenCL engine can manage calls from various threads (serialized)

3. Reduce the area covered by the unique_lock; only the core processing needs to be locked

4. Once 1 engine runs optimally, you can create a 2nd engine and alternate between both for full GPU usage (view my code as reference)

At the end of the day, these 4 simple points made a big difference with my implementation of AviSynthShader ... except that I had to refactor large parts of the application for proper encapsulation.

Then we can run 8 threads and only 2 instances of KNLMeans get created. If efforts have already been made to support VapourSynth's MT structure, the benefit might not be as big there but still both versions would benefit.

The talk of freeing the CPU for other plugin is simply nonsense in any case.
KNLMeansCL can not release a frame that has not yet finished processing.
While the plugin works the other plugins wait. And that's how it should be.
You cannot release frame 1 while it is still processing, but frames 2, 3, 4, 5, 6, 7 and 8 can be processed during that time.

[Khanattila]

So you did optimize it for VapourSynth's multi-threading.

If the OpenCL engine is well encapsulated into its own class, it's not difficult to handle the muti-threading in AviSynth.

1. Simply adding a unique_lock within GetFrame is generally enough to convert it from a MT_SERIALIZED to a MT_NICE_FILTER

2. Ensure the OpenCL engine can manage calls from various threads (serialized)

3. Reduce the area covered by the unique_lock; only the core processing needs to be locked

4. Once 1 engine runs optimally, you can create a 2nd engine and alternate between both for full GPU usage (view my code as reference)

At the end of the day, these 4 simple points made a big difference with my implementation of AviSynthShader ... except that I had to refactor large parts of the application for proper encapsulation.

Then we can run 8 threads and only 2 instances of KNLMeans get created. If efforts have already been made to support VapourSynth's MT structure, the benefit might not be as big there but still both versions would benefit.



You are wrong. I have not optimized for VapourSynth, the versions are identical. Simply VapourSynth is of this century.

[MysteryX]

I have not optimized for VapourSynth, the versions are identical. Simply VapourSynth is of this century.
Then both versions would benefit from this, if you decide to release another version.

Since I've already coded those optimizations, it would make it a lot easier to duplicate.

[Khanattila]

You cannot release frame 1 while it is still processing, but frames 2, 3, 4, 5, 6, 7 and 8 can be processed during that time.

This is what is currently happening in the various versions.

#AviSynth
[Thread0] -> GetFrame(0) -> KNLMeansCL(0) -> GetFrame(1) -> KNLMeansCL(1) -> GetFrame(2) -> KNLMeansCL(2)

#AviSynth MT
[Thread0] -> GetFrame(0) -> KNLMeansCL:Work(0) -> KNLMeansCL:Work(0) -> KNLMeansCL:Work(0) -> KNLMeansCL:Work(0) -> GetFrame(4)
[Thread1] -> GetFrame(1) -> KNLMeansCL:Work(1) -> KNLMeansCL:Work(1) -> KNLMeansCL:Work(1) -> KNLMeansCL:Work(1) -> GetFrame(5)
[Thread2] -> GetFrame(2) -> KNLMeansCL:Work(2) -> KNLMeansCL:Work(2) -> KNLMeansCL:Work(2) -> KNLMeansCL:Work(2) -> GetFrame(6)
[Thread3] -> GetFrame(3) -> KNLMeansCL:Work(3) -> KNLMeansCL:Work(3) -> KNLMeansCL:Work(3) -> KNLMeansCL:Work(3) -> GetFrame(7)

#VapourSynth
[Thread0] -> GetFrame(0) -> KNLMeansCL(0) -> GetFrame(4) -> KNLMeansCL(4) -> Wait(5) -> Wait(5)
[Thread1] -> GetFrame(1) -> Wait(1) -> KNLMeansCL(1) -> GetFrame(5) -> KNLMeansCL(5) -> Wait(5)
[Thread2] -> GetFrame(2) -> Wait(2) -> Wait(2) -> KNLMeansCL(2) -> GetFrame(6) -> KNLMeansCL(6)
[Thread3] -> GetFrame(3) -> Wait(3) -> Wait(3) -> Wait(3) -> KNLMeansCL(3) -> GetFrame(7)

[Khanattila]

Then both versions would benefit from this, if you decide to release another version.

Since I've already coded those optimizations, it would make it a lot easier to duplicate.

No, because KNLMeansCL is already divided into many small kernel. What I can do is to write the individual kernels better, and some other small correction.
And it is among the best ways to saturate the GPU.

[MysteryX]

VapourSynth really confuses me... either it is not developer-friendly, or I just don't understand its logic.

Aren't all of the small kernels encapsulated into 1 core engine? If not, then that would need to be done to implement what I said. It can mean a lot of refactoring, but also leads to much better structured and cleaner code.

[Khanattila]

VapourSynth really confuses me... either it is not developer-friendly, or I just don't understand its logic.
Rewritten a little, maybe now it is more clear.

Aren't all of the small kernels encapsulated into 1 core engine? If not, then that would need to be done to implement what I said. It can mean a lot of refactoring, but also leads to much better structured and cleaner code.
But you understand that if a single instance of KNLMeansCL saturates the GPU, create more of it give you only more overhead?

[MysteryX]

Rewritten a little, maybe now it is more clear.

But you understand that if a single instance of KNLMeansCL saturates the GPU, create more of it give you only more overhead?
According to my tests, 1 instance doesn't saturate. 2 instances give between 2.5% and 30% increased performance depending on settings.

OK now I see what you're doing with VapourSynth. While waiting for the GPU, you call the processing of another frame. However that call won't return until that frame is done and may delay the liberation of the GPU. Unless you're just informing VapourSynth to process it and it does so in a different thread without locking your code.

I prefer my code chain better.

[Groucho2004]

But you understand that if a single instance of KNLMeansCL saturates the GPU, create more of it give you only more overhead?
If you look at some of his posts/scripts, this is clearly something that does not register. I pointed out several times in various topics that every additional thread creates overhead in the Windows thread scheduler but he keeps multi-threading the crap out of everything, even internally multi-threaded filters like nnedi3/eedi3 without realizing that efficiency goes down the drain.

[MysteryX]

Groucho2004, with 8 cores, MT is a must. Especially when working with the GPU on top of that. But that's off-topic. Let's keep the discussion technical to this situation with KNLMeans.

[asarian]

In my case, the CPU very rarely goes above 80%. When running SMDegrain(prefilter=4), I've been stuck with a ~35% CPU usage, but now I see part of it is because of running KNLMeans with MT mode 3 that is a performance killer. Just switching it to MT mode 2 will probably help.

Btw, I forgot to mention, I use VapourSynth nowadays (almost exclusively). Maybe that makes a difference too?

AviSynth MT() calls, while I was fortunate to have it work more-or-less stable on my machine, remains a very ugle hack, IMHO. I avoid it as the plague. :) It's also very crude. Better, as Khanattila said, to take a 1 process, and divide it up yourself, as the author of the code, into several mini-threads on the CPU, than MT()-ing (can you use that as a verb?) the cr*p out of the entire process.

While I have no proof, in my experience, the native threading model of VapourSynth is far superior than that of the MT() hack in AviSynth. I made the transition to VapourSynth a while back already, and haven't looked back.

I see you have an octa-core. Yeah, that may be harder to fully saturate (on either).

P.S. This was not to slam Sett, whose AS's MTMode stuff I have used gratefully over the years. But VapourSynth simply started from scratch, and got the threading right from the get-go this time.

[MysteryX]

Threading is actually working pretty well in Avisynth+. With v2.6, we couldn't configure per filter so it's pretty much all running on MT mode 2 which means all filters get recreated for each thread (high memory usage). With AviSynth+, many filters support MT mode 1 which means a single instance is being created and it handles simultaneous requests. It's actually working very well (except bugs in a few plugins).

[feisty2]

Wanna heat some bread with laptop?
Fine, 2 scripts, one runs knlmeanscl on gpu and another on cpu
Now you execute these 2 scripts simultaneously, and there you go

[MysteryX]

Wanna heat some bread with laptop?
Fine, 2 scripts, one runs knlmeanscl on gpu and another on cpu
Now you execute these 2 scripts simultaneously, and there you go
Bad idea. The CPU is busy doing more important work.

I did try running 2 instances on the 2 devices and for some reason I saw no performance improvement. mmmmm... perhaps that needs to be tested again.

[feisty2]

I know, microwave does the heating work much better
It's just you're so obsessed with 100% cpu usage so...

[MysteryX]

I know, microwave does the heating work much better
It's just you're so obsessed with 100% cpu usage so...
I almost forgot you couldn't care less about performance :)

[feisty2]

I almost forgot you couldn't care less about performance :)

well, I simply failed to understand what kind of "multi-threading" do you want, like Khanattila said, you don't really wanna multi-threaded knlmeanscl since one is heavy enough to bottleneck your cheesy laptop gpu, and you want more threads to what? make a gpu time bomb?
and cpu will have no other option but waiting, post-knlmeanscl filters do their calculations based on the knlmeanscl processed clip and they simply can't time travel and get to work before the required data reaches them