KNLMeansCL: OpenCL NLMeans de-noising algorithm [2018-01-29]

[KingLir]

Another job for you
https://github.com/Khanattila/KNLMeansCL/archive/osx.zip

Builds a bit differently:
http://pastebin.com/fcCQ9bG5

Runs with a bit different output (but still with flickering and artifacts):
https://mega.nz/#!d5hTmTIL!cAV9dh_IW8rX5ec0qs5bjnWQQbYDFrZ8ATUrLY5OF1Y

[Khanattila]

Builds a bit differently:
http://pastebin.com/fcCQ9bG5

Runs with a bit different output (but still with flickering and artifacts):
https://mega.nz/#!d5hTmTIL!cAV9dh_IW8rX5ec0qs5bjnWQQbYDFrZ8ATUrLY5OF1Y

https://github.com/Khanattila/KNLMeansCL/archive/osx.zip
I removed some optimizations, try it.

[KingLir]

https://github.com/Khanattila/KNLMeansCL/archive/osx.zip
I removed some optimizations, try it.

Seems better but now with different artifacts. Maybe you can compare to what you get on Windows with my input ?

Here is the new output:
https://mega.nz/#!Nthh1L4T!H4K7ziB5rFmp9k2r_vDHA78xXYn4-nbnyVddjry4d5M

And here my script:

clip = core.ffms2.Source(source=filename)
clip = core.bifrost.Bifrost(clip, interlaced=True) # Remove rainbows
clip = core.tcomb.TComb(clip=clip, mode=0) # Remove dotcrawl
clip = core.vivtc.VFM(clip=clip, order=1) # IVTC part 1
clip = core.vivtc.VDecimate(clip=clip) # IVTC part 2

clip = core.knlm.KNLMeansCL(clip=clip)

clip.set_output()

and again the input:
https://mega.nz/#!ggxF0bxA!IksjCkXKATd1VOS3ps4NrgV91p1k6YdQIRYcyvIyowY

[Khanattila]

Seems better but now with different artifacts. Maybe you can compare to what you get on Windows with my input ?

Here is the new output:
https://mega.nz/#!Nthh1L4T!H4K7ziB5rFmp9k2r_vDHA78xXYn4-nbnyVddjry4d5M

And here my script:

clip = core.ffms2.Source(source=filename)
clip = core.bifrost.Bifrost(clip, interlaced=True) # Remove rainbows
clip = core.tcomb.TComb(clip=clip, mode=0) # Remove dotcrawl
clip = core.vivtc.VFM(clip=clip, order=1) # IVTC part 1
clip = core.vivtc.VDecimate(clip=clip) # IVTC part 2

clip = core.knlm.KNLMeansCL(clip=clip)

clip.set_output()

and again the input:
https://mega.nz/#!ggxF0bxA!IksjCkXKATd1VOS3ps4NrgV91p1k6YdQIRYcyvIyowY
https://github.com/Khanattila/KNLMeansCL/archive/osx.zip
There is always a precision problem

[KingLir]

There is always a precision problem

New version with artifacts:
https://mega.nz/#!N5Q3DIjD!Umhy-DFb0DcNGR5CX15sBs8UQeGyWt7YYHPuZ-uvLhs

[KingLir]

Also, when I tried to run Plum I am getting errors that may help you track the issue/s. Running with the latest build you posted here - I haven't tried the previous builds.

https://forum.doom9.org/showthread.php?p=1799964#post1799964

[Khanattila]

Also, when I tried to run Plum I am getting errors that may help you track the issue/s. Running with the latest build you posted here - I haven't tried the previous builds.

https://forum.doom9.org/showthread.php?p=1799964#post1799964

Maybe it's a problem of the AMD driver, because you have strange problems.

I do not know if I can work around it.
https://github.com/Khanattila/KNLMeansCL/archive/osx.zip

[KingLir]

Maybe it's a problem of the AMD driver, because you have strange problems.

I do not know if I can work around it.
https://github.com/Khanattila/KNLMeansCL/archive/osx.zip

Thanks for trying!
It's almost certainly not the AMD driver since, on macOS, there is only the official Apple-supplied driver and millions use it with many different OpenCL software.
It's probably just some different variation of one of your code lines that is needed when running on macOS' OpenCL.

With latest version, failed on run:
http://pastebin.com/tLzEmvqm

[KingLir]

No new ideas for me to try ?

[MysteryX]

Why does KNLMeans runs slower on VapourSynth than on Avisynth?
https://github.com/Khanattila/KNLMeansCL/wiki/Benchmark

[TheFluff]

Why does KNLMeans runs slower on VapourSynth than on Avisynth?
https://github.com/Khanattila/KNLMeansCL/wiki/Benchmark

They run at almost exactly the same speed (less than 0.05 fps difference in my tests) if Avs+ is run single threaded. If you run Avs+ with prefetch(4) at the end of the script you get a completely trivial speed increase of like 1 fps over VS (in which KNLMeansCL runs in mode fmParallelRequests, so there's only one instance of it) at the expense of eating almost four times as much memory. A guess pulled out of my ass says that the extra fps in Avs+ comes from the extra threads getting to use the GPU during the time when a filter instance that just finished processing a frame is busy copying buffers, or something like that.

See also https://forum.doom9.org/showthread.php?t=174437

[Khanattila]

For benchmarking I'm using old fashioned AviSynth v2.6.0 32-bit.
Otherwise I would have writen AviSynth+

[MysteryX]

For benchmarking I'm using old fashioned AviSynth v2.6.0 32-bit.
Otherwise I would have writen AviSynth+
So what could cause such difference?

Running AVS+ with 2 threads would most likely further result in slightly higher performance.

[Groucho2004]

Running AVS+ with 2 threads would most likely further result in slightly higher performance.
Not my experience. Using Prefetch() with 2 or 4 threads on my 4 core CPU actually lowers performance slightly and obviously increases GPU (and CPU) memory usage.

[MysteryX]

...

With this simple code, it produces random results and mixes frames together. I can't believe I went to long without realizing it.

It fails when running as MT_SERIALIZED. The only MT mode that works is MT_MULTI_INSTANCE -- which isn't ideal for this type of filter.


AviSource("Preview.avi")
KNLMeansCL(D=1, A=1, h=1.5, device_type="GPU")
Prefetch(4)


For example, I opened the same script 3 times at the same frame and got this
https://s22.postimg.org/d8nochfgd/img1.png (https://postimg.org/image/d8nochfgd/) https://s22.postimg.org/5h6ydxbb1/img2.png (https://postimg.org/image/5h6ydxbb1/) https://s22.postimg.org/nl9yyk8zh/img3.png (https://postimg.org/image/nl9yyk8zh/)

[TheFluff]

#2 there looks kinda like h264 decoder error recovery. Are you sure this is all KNLMeans?

[MysteryX]

Changing KNLMeans to MT_MULTI_INSTANCE fixes the issue -- and in this code I'm loading an AVI file, no H264.

[Khanattila]

Changing KNLMeans to MT_MULTI_INSTANCE fixes the issue -- and in this code I'm loading an AVI file, no H264.

The new branch v1.* has changed some things.

[MysteryX]

I'll be waiting, in the rain, for your return... with AVS+ native high-bit-depth support and fix to this MT bug

[LigH]

Are there any more requirements to use KNLMeansCL release v1.0.2 (https://github.com/Khanattila/KNLMeansCL/releases) under AviSynth+ MT r2455?

I put KNLMeansCL-v1.0.2.zip:KNLMeansCL.dll into ...\AviSynth 2.5\plugins; AVSMeter reports Windows system error 127 (ERROR_PROC_NOT_FOUND).

I put KNLMeansCL-v1.0.2.zip:x64/KNLMeansCL.dll into ...\AviSynth+\plugins64; AVSMeter64 reports Windows system error 127 (ERROR_PROC_NOT_FOUND).

OpenCL.dll exists in Windows\system32 and in Windows\SysWoW64 (Windows 7 SP1, GeForce 9600, Nvidia driver 342.01; Windows Device Manager lists Nvidia OpenCL drivers among display drivers).

[Groucho2004]

I put KNLMeansCL-v1.0.2.zip:x64/KNLMeansCL.dll into ...\AviSynth+\plugins64; AVSMeter64 reports Windows system error 127 (ERROR_PROC_NOT_FOUND).

OpenCL.dll exists in Windows\system32 and in Windows\SysWoW64 (Windows 7 SP1, GeForce 9600, Nvidia driver 342.01; Windows Device Manager lists Nvidia OpenCL drivers among display drivers).Look at the requirements (https://github.com/Khanattila/KNLMeansCL/wiki/Requirements). GT9600 does not support OpenCL 1.2, neither does driver version 342.01. You have to downgrade to KNLMeansCL 0.6.11.

[LigH]

OK, that explains some ... I wish the ZIP file included at least a small README with a summary or link to these requirements. I promise to read README's, at least in case of issues, if there are any.

So I guess I should avoid QTGMC with options which would try to access KNLMeansCL on this machine. Fortunately, it's optional for this use.

[Khanattila]

...

With this simple code, it produces random results and mixes frames together. I can't believe I went to long without realizing it.

It fails when running as MT_SERIALIZED. The only MT mode that works is MT_MULTI_INSTANCE -- which isn't ideal for this type of filter.


AviSource("Preview.avi")
KNLMeansCL(D=1, A=1, h=1.5, device_type="GPU")
Prefetch(4)


For example, I opened the same script 3 times at the same frame and got this
https://s22.postimg.org/d8nochfgd/img1.png (https://postimg.org/image/d8nochfgd/) https://s22.postimg.org/5h6ydxbb1/img2.png (https://postimg.org/image/5h6ydxbb1/) https://s22.postimg.org/nl9yyk8zh/img3.png (https://postimg.org/image/nl9yyk8zh/)

#2 there looks kinda like h264 decoder error recovery. Are you sure this is all KNLMeans?

Changing KNLMeans to MT_MULTI_INSTANCE fixes the issue -- and in this code I'm loading an AVI file, no H264.
I'm completing the latest checks, but MT_NICE_FILTER seems to work properly

PS. Forget it, the design of the new version is not really compatible.

[Khanattila]

https://github.com/Khanattila/KNLMeansCL/releases/tag/v1.1.0

KNLMeansCL v1.1.0
* Added AviSynth MT support (MT_MULTI_INSTANCE).
* Added AviSynth+'s high bit depth support.
* Replaced 'lsb_inout' with 'stacked'.
* Info now display 'Bits per sample'.

[Dreamland]

Thx a lot new version of KNLMeansCL
excuse me for my bad english
I have an issue
https://t9.pixhost.org/thumbs/385/39811504_appunti01.png (https://pixhost.org/show/385/39811504_appunti01.png)

my smdegrain is v. 3.1.2.92s mod
my QTGMC is v. 3.357 ( i think last version)
older version of KNLMeansCL work very well

Help me Khanattila
thx

[MysteryX]

so... MT_NICE_FILTER works or not?

AFAIK MT_MULTI_INSTANCE was already working; but the only mode giving consistent results.

[dipje]

@Dreamland: SMDegrain needs to be changed to the newer KNLMeans versions it seems.

If you want to try fixing it yourself, search and replace in your SMDegrain script, replace all occurances of 'lsb_inout' with 'stacked'

[Dreamland]

ok..I can do..hope it works
thanks a lot!

[Khanattila]

so... MT_NICE_FILTER works or not?

AFAIK MT_MULTI_INSTANCE was already working; but the only mode giving consistent results.

MT_NICE_FILTER works only with Prefetch(1)... so basically, it doesn't work.

[dipje]

@Khanattila:

A bit off topic, but on your Github I see an old project KPassFilterCL. Do I see correctly that it's Avisynth only? I tried using it in Vapoursynth through the core.avs.LoadPlugin stuff, and it appears to be working. I use fmtconv routines to convert 16bit Vapoursynth to avisynth's "stacked" format, then use the routines from KPassFilterCL with their 'lsb_inout' set to True. This works OK (I guess) for 'mode = magnitude' but seems to only produce black output when 'mode = phase' ? If I switch to 8bit I do suddenly get output.

Can you explain a bit of it? I'm trying to separate an image into multiple images based on frequeny content (or wavelets maybe?), to apply different filtering to different frequency-ranges and then merge it all back into a single image. KPassFilterCL seems to be made for that but I'm not sure if that's going to work or if I'm using it wrong.

With the 'low pass' routine I'm trying to get, well, only low-frequency content, which I expect to be a very blurry picture (I'm talking about a grayscale luminance only clip BTW), but in brightness / contrast / levels to be pretty much the same as the starting image.
But I only seem to get very dark output, when I change the 'cutoff' parameter the image starts to look more like the original in brightness / contrast, but also less blurry, so more high-freq content is getting in :(.

Is it normal that the cutoff parameter seems to require very high values (as in over 250000.0) to do something?

And if I would want to split an image into three parts, very blurry parts (almost no details, low-frequency), course details (mid-frequency) and small (micro) details (high frequency) can I do that with this plug? If so, could you give an example?

[BakaProxy]

@Khanatila, I think I found a(n edge case) bug.
http://screenshotcomparison.com/comparison/209076
(If it's not obvious, check the black smudge at the top.)
This happens with bigger a values (9 on this example on a 480p clip) and If I recall correctly knlmeanscl internally pads the video clip, so I think that this is just the black padding leaking into the image. (if so, maybe you could mirror instead?)
Though this is just my hypothesis, feel free and I'd love you to prove me wrong.

As for the data on this specific scenario they're in the screenshot using info=true, if more is necessary I'm ready to provide such.

[Logan9778]

Hey guys, I'm new to all this GitHub stuff. How would you compile the code for KNLMeansCL to get the .dll file in Windows 10 ( I use AviSynth )?

[Groucho2004]

Hey guys, I'm new to all this GitHub stuff. How would you compile the code for KNLMeansCL to get the .dll file in Windows 10 ( I use AviSynth )?Why do you feel the need to compile it if there are perfectly adequate releases (https://github.com/Khanattila/KNLMeansCL/releases)?

[Khanattila]

@Khanattila:

A bit off topic, but on your Github I see an old project KPassFilterCL. Do I see correctly that it's Avisynth only? I tried using it in Vapoursynth through the core.avs.LoadPlugin stuff, and it appears to be working. I use fmtconv routines to convert 16bit Vapoursynth to avisynth's "stacked" format, then use the routines from KPassFilterCL with their 'lsb_inout' set to True. This works OK (I guess) for 'mode = magnitude' but seems to only produce black output when 'mode = phase' ? If I switch to 8bit I do suddenly get output.

Can you explain a bit of it? I'm trying to separate an image into multiple images based on frequeny content (or wavelets maybe?), to apply different filtering to different frequency-ranges and then merge it all back into a single image. KPassFilterCL seems to be made for that but I'm not sure if that's going to work or if I'm using it wrong.

With the 'low pass' routine I'm trying to get, well, only low-frequency content, which I expect to be a very blurry picture (I'm talking about a grayscale luminance only clip BTW), but in brightness / contrast / levels to be pretty much the same as the starting image.
But I only seem to get very dark output, when I change the 'cutoff' parameter the image starts to look more like the original in brightness / contrast, but also less blurry, so more high-freq content is getting in :(.

Is it normal that the cutoff parameter seems to require very high values (as in over 250000.0) to do something?

And if I would want to split an image into three parts, very blurry parts (almost no details, low-frequency), course details (mid-frequency) and small (micro) details (high frequency) can I do that with this plug? If so, could you give an example?

Sorry if I did not answer you first but that filter was born almost for fun and I never optimized or improved its effectiveness.
If you really want to use it for something I have to check it for a moment, because I really do not remember what all the functions do.

[Khanattila]

@Khanatila, I think I found a(n edge case) bug.
http://screenshotcomparison.com/comparison/209076
(If it's not obvious, check the black smudge at the top.)
This happens with bigger a values (9 on this example on a 480p clip) and If I recall correctly knlmeanscl internally pads the video clip, so I think that this is just the black padding leaking into the image. (if so, maybe you could mirror instead?)
Though this is just my hypothesis, feel free and I'd love you to prove me wrong.

As for the data on this specific scenario they're in the screenshot using info=true, if more is necessary I'm ready to provide such.
Ok, this is bug maybe. I have to check. Or maybe I did it, since the version you use is old.

[Logan9778]

Why do you feel the need to compile it if there are a perfectly adequate releases (https://github.com/Khanattila/KNLMeansCL/releases)?

Ahh, thanks, I didn't see the zip had the .dll inside it.

[MysteryX]

With this simple code, it produces random results and mixes frames together. I can't believe I went to long without realizing it.

It fails when running as MT_SERIALIZED. The only MT mode that works is MT_MULTI_INSTANCE -- which isn't ideal for this type of filter.


For the records, this is probably due to this bug in Avisynth+ with MT_SERIALIZE
https://forum.doom9.org/showthread.php?p=1807903#post1807903

[MysteryX]

This doesn't work in 16-bit


ConvertToYV24()
ConvertBits(16)
KNLMeansCL(D=2, A=2, h=1.5, channels="YUV")

KNLMeansCL: 'channels = YUV' require a YV24 pixel format!

[Khanattila]

This doesn't work in 16-bit


ConvertToYV24()
ConvertBits(16)
KNLMeansCL(D=2, A=2, h=1.5, channels="YUV")

KNLMeansCL: 'channels = YUV' require a YV24 pixel format!

I am very busy so I do not have time now, but I notice your reports :goodpost:

[goorawin]

I have been using KNLMeansCL 64bit for sometime without problems on windows10 pro with a GeForce GTX 650 graphics card.
Now after some major windows updates, it no longer works in 64 bit. It will not load the plugin and comes up with the error “there is no function named KNLMeansCL”. I have tried a number of versions of KNLMeansCL (including v1.1.0) but all have the same problem. All other 64bit plugin work as they should. Even xnlmeans works which I assume is not much different to KNLMeansCL except for GPU usage.
Interestingly the32 bit version of KNLMeansCL works just fine, so what's the difference?.
The other interesting thing is that, on another windows 10 pro computer with an on board Intel HD Graphics 630 card which has also had the same updates, everything work as it should in 64bit.
So what are the dependencies of KNLMeansCL in 64bit?
Any help would be appreciated.

[Groucho2004]

So what are the dependencies of KNLMeansCL in 64bit?Run AVSMeter64 (https://forum.doom9.org/showthread.php?t=173259) with the switch "-avsinfo", it will report any missing dependencies.

[goorawin]

From my experence reinstalling GeForce driver after Win10 major updates solves this issue. YMMV.

That did the trick thank you. Interesting that it only caused an issue with 64bit scripts.

[SaurusX]

Great filter, khanatilla. Non-local means is an excellent noise reduction theory and with OpenCL its actually usable in my computer. Thanks for your hard work!

[SaurusX]

Well, I found a way to break the filter while encoding some cartoons. Here's the pseudo-script:

MPEG2SOURCE("myclip")
TComb(mode=0, blah, blah) #for dot-crawl removal
TFM()
TDecimate()
the masked and motion-compensating derainbowing procedure found here (http://forum.doom9.org/showthread.php?t=171715)
Bifrost(interlaced=false) #1.1 version for final de-rainbowing
KNLMeansCL(d=2,a=3,h=1.2,device_type="GPU") #Using Intel HD4000 graphics
maa2() #anti-alias
msharpen() #resharpen
dehalo_alpha() #de-halo
awarpsharp2() #sharpen chroma

It seems like too much temporal analysis will cause mask pops, frame flashing, and mistimed chroma in the final encode. I can remove TComb or remove Bifrost or change the d parameter to 1 and the problems disappear. This may not be a specific problem to KNLMeansCL, but I thought you'd like to know about it.

[MysteryX]

This doesn't work in 16-bit


ConvertToYV24()
ConvertBits(16)
KNLMeansCL(D=2, A=2, h=1.5, channels="YUV")

KNLMeansCL: 'channels = YUV' require a YV24 pixel format!

I am very busy so I do not have time now, but I notice your reports :goodpost:

This one is an easy bug, any plans to fix it soon?

[tuanden0]

Sorry Sir, but after update my VGA card driver, the KNLMeansCL fillter make my PC reboot because the GPU load over 100%.
I tried to reinstall old driver but the issue still there.
I'm using NVIDIA GeForce GTX 650Ti.

[Ben_Nicholls]

Sorry Sir, but after update my VGA card driver, the KNLMeansCL fillter make my PC reboot because the GPU load over 100%.
I tried to reinstall old driver but the issue still there.
I'm using NVIDIA GeForce GTX 650Ti.

Have you confirmed that your GPU is 100% stable?

Running MSI Kombustor (http://www.geeks3d.com/20150710/msi-kombustor-v3-5-2-64-bit-v2-6-0-32-bit-download-gpu-stress-test-benchmark/) or FurMark (http://www.ozone3d.net/benchmarks/fur/) should let you know: if either one crashes, your GPU isn't stable and probably needs replacing.

[tuanden0]

Have you confirmed that your GPU is 100% stable?

Running MSI Kombustor (http://www.geeks3d.com/20150710/msi-kombustor-v3-5-2-64-bit-v2-6-0-32-bit-download-gpu-stress-test-benchmark/) or FurMark (http://www.ozone3d.net/benchmarks/fur/) should let you know: if either one crashes, your GPU isn't stable and probably needs replacing.

Thank for your reply,
My problem solved, my VGA card is dirty, so they crashed because it's too hot :mad:
I cleaned my VGA so it's OK :p

[MysteryX]

Thank for your reply,
My problem solved, my VGA card is dirty, so they crashed because it's too hot :mad:
I cleaned my VGA so it's OK :p
Just cleaned my laptop too. Removing dust fixes a few things :)

[Khanattila]

Roughly what does NLMeansCL2(h=radius*1.5,u=3,v=3) convert to for KNLMeansCL()?

KNLMeansCL(d=0), but the results are different.
NLMeansCL2 worked on a smaller image.