KNLMeansCL: OpenCL NLMeans de-noising algorithm [2018-01-29]

[DarkSpace]

Every pixel is restored by the weighted average of all pixels in its search window.
And the weight is inversely proportional to the square of distance. It gives better result, usually. ;)
Ahh, so that omission was on purpose. I see.

In any case the distance between two vectors is: sqrt((p0.x-p1.x)^2 + (p0.y-p1.y)^2 + ....).
You can simplify to abs(p0-p1) only if p is a number.
I only said that Y-only distance could be simplified that way. And Y-only distance only uses one number of each p (the Y value).

[tormento]

Is somebody using KNLMeans under Windows10_x64?

It worked perfectly under Windows8.1_x64 but in Windows10_x64 the preview of MeGUI stays frozen on a "grey" screen.

SetMTMode(5)
SetMemoryMax(2048)
LoadPlugin("D:\eseguibili\media\DGDecNV\DGDecodeNV.dll")
DGSource("E:\in\2_01 favoloso mondo di Amelie, Il\favoloso.dgi")
CompTest(1)
ChangeFPS(last,last,true)
SetMTMode(2)
SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6)

PreFilter=4 means KNL, with Prefilter=2 (software) everything is ok.

[Xebika]

Is somebody using KNLMeans under Windows10_x64?

It worked perfectly under Windows8.1_x64 but in Windows10_x64 the preview of MeGUI stays frozen on a "grey" screen.

SetMTMode(5)
SetMemoryMax(2048)
LoadPlugin("D:\eseguibili\media\DGDecNV\DGDecodeNV.dll")
DGSource("E:\in\2_01 favoloso mondo di Amelie, Il\favoloso.dgi")
CompTest(1)
ChangeFPS(last,last,true)
SetMTMode(2)
SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6)

PreFilter=4 means KNL, with Prefilter=2 (software) everything is ok.
Install this version of NVIDIA driver:

http://developer.download.nvidia.com/assets/gameworks/downloads/regular/oculus07driver/355.83/355.83_geforce_win10_64bit_international.exe

[tormento]

Install this version of NVIDIA driver

Thanks but no change at all.

In GPU-Z I can see a little bump in Video Engine and then nothing. :(

[Xebika]

Do you use desktop or notebook??

[tormento]

Do you use desktop or notebook??

I use the same hardware I had before, a desktop with 17-2600K and a GTX760. On Windows8.1_x64 ok, on Windows10_x64, ko.

Have you done a successful encode with Win10_x64?

[Xebika]

Yes i have some successful test clips, but can only get it to working with the version of the driver i posted.

[tormento]

Yes i have some successful test clips, but can only get it to working with the version of the driver i posted.

I have tried to run some OpenCL benchmarks to see if the 760 is correctly working, such as LuxMark.

It identifies the card as OpenCL 1.2 CUDA 7.5.0 device and runs benchmarks smootly.

No idea.

Neither simple KNLMeans() works.

[Xebika]

Ok I have no more ideas right now.

[Khanattila]

I use the same hardware I had before, a desktop with 17-2600K and a GTX760. On Windows8.1_x64 ok, on Windows10_x64, ko.

Have you done a successful encode with Win10_x64?

Same configuration here: http://www.techpowerup.com/gpuz/details.php?id=5p569.

Anyway, MT is NOT recommended with KNLMeansCL. Try mode=5 or mode=6.

[tormento]

I have disabled MT and now gives me a green screen with CUDA ERROR! in the upper left corner.

[tormento]

Are you sure your DGDecNV works fine in your Windows 10? How about just use the following simple script and see what you get?

Yes, a simple encoding with software filter works perfectly.

Update: the following script works, it must be a incompatibility between DGDecNV and KNLMeans together

LoadPlugin("D:\eseguibili\media\MeGUI\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("F:\raw\favoloso mondo di Amelie, Il.mkv")
#deinterlace
crop(0, 128, 0, -134)
#resize
#denoise
CompTest(1)
ChangeFPS(last,last,true)
SetMTMode(2)
SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6)

[videoh]

How much memory do you have on your video card?

[tormento]

How much memory do you have on your video card?

Look here (http://www.techpowerup.com/gpuz/details.php?id=vu7kg)

[tormento]

Even

SetMTMode(5)
SetMemoryMax(2048)
LoadPlugin("D:\eseguibili\media\DGDecIM\DGDecodeIM.dll")
dgsourceim("E:\in\2_01 favoloso mondo di Amelie, Il\favoloso.dgi",engine=1)
crop(0, 128, 0, -134)
CompTest(1)
ChangeFPS(last,last,true)
SetMTMode(2)
SMDegrain (tr=6,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6)

Works perfectly.

[Khanattila]

Yes, a simple encoding with software filter works perfectly.

Update: the following script works, it must be a incompatibility between DGDecNV and KNLMeans together

LoadPlugin("D:\eseguibili\media\MeGUI\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("F:\raw\favoloso mondo di Amelie, Il.mkv")
#deinterlace
crop(0, 128, 0, -134)
#resize
#denoise
CompTest(1)
ChangeFPS(last,last,true)
SetMTMode(2)
SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6)

I got it working now, because a new driver is released today but just a beta driver.

As another user reported, could be a problem of the nvidia driver.

[tormento]

As another user reported, could be a problem of the nvidia driver.

Xebika gave me the nvidia driver link. No fix.

[tormento]

Ok I have no more ideas right now.
Can you please post your avs script?

[Xebika]

Here:
LoadPlugin("C:\Video\MeGUI\tools\dgindexnv\DGDecodeNV.dll")
DGSource("C:\Video\Video.dgi",fieldop=0, deinterlace=1)
crop(24, 0, -8, -6)
Spline36Resize(768,576)
Levels(5, 1, 255, 0, 255)
SmoothContrast(contrast=0.2, sat=0.1)
LSFmod(defaults="slow")
KNLMeansCL(D=1, A=1, h=4.5)

[tormento]

Here:

Would you please try with a 1080p source?

[Xebika]

It worked fine.

[Khanattila]

v0.6.1 release: https://github.com/Khanattila/KNLMeansCL/releases.

It corrects an error that I had introduced in the previous version.

[tormento]

v0.6.1 release
Even this version doesn't work for me with DGDecNV. It works for every other SW/HW decoder plugin.

Same green screen in preview with message "CUDA ERROR!".

[MysteryX]

This version gives me this error:
KNLMeansCL: AviSynthCreate Error (clBuildProgram!)

[detmek]

This version gives me this error:
KNLMeansCL: AviSynthCreate Error (clBuildProgram!)

Same here. I use it with StaxRip x64 and Avisynth+. Same error with 0.6.0. Builds up to 0.5.9 work just fine.

[MysteryX]

I'm using AviSynth 2.6

Also, KNLMeans is giving me better results than FF3DFilter on most videos, but why is it giving me such bad output on this video? The lines in the background end up being very jerky (I know, the source video is bad quality), but FF3DFilter is doing a MUCH better job in this particular case. In most other videos with similarly bad quality, however, KNLMeans gives a better output.

Here's the video
https://mega.nz/#!CNICBS5S!56wGjk8gtKzc9HMrPfc8ZZuCCxdGgk4ms8mPXFr0nkY

KNLMeansCL(D=2, A=1, h=3, device_type="GPU")

[Sparktank]

Win7 (x64) SP1
GTX 750 ti
GeForce 355.82 (newest; although I don't think much changed as far as video processing goes, mostly just new games)

KNLMeansCL v0.6.1
SetMemoryMax(512)
LoadPlugin("C:\AVS\dgdecnv2049\DGDecodeNV.dll")
DGSource("E:\Demux\Scream\Scream.dgi", crop_b=140, crop_t=132)
#KNLMeansCL(D=2, A=1, h=3, device_type="GPU", info=true)

"jerky lines in the background"

I don't see "jerky lines" in the sample video.
I grabbed a demux of the poorly mastered BD for Scream, since it's noisy and messy.
(Canadian BD Region A, identical to USA BD)

(all images are "large thumbnail", click to follow to original size (1920x816)
Here's original Frame #7282:
Focus: the white door frame behind Drew
http://i.imgur.com/wlbax1el.png (http://imgur.com/wlbax1e)

Here's with the settings used by MysteryX:
KNLMeansCL(D=2, A=1, h=3, device_type="GPU", info=true)
Focus: the white door frame behind Drew
http://i.imgur.com/0sRssNzl.png (http://imgur.com/0sRssNz)

Looks it denoises a lot of the main flat areas, but any edges are nearly the same.
When you scan forward, you can see this jerkiness it produces to due to unfiltered edges.
Some frames, the edges are cleaned out right.
And a large portion of (proceeding) frames, the noise varies, giving it a jerky look when played back.
At those setings.

Here's same settings, an earlier frame where the door frame is entirely denoised and clean edges.
Frame #7275:
http://i.imgur.com/yEf8hPml.png (http://imgur.com/yEf8hPm)

Frame #7276 is pretty clean too.
From there, it starts to degrade back to a noisy edge.

BUT
I changed parameters and found this:

Frame #7313
KNLMeansCL(D=3, A=0, h=3, device_type="GPU", info=true)
http://i.imgur.com/hHkWFFwl.png (http://imgur.com/hHkWFFw)
Jerky-ish lines.

Are those the kind of jerky lines you got?
I couldn't get those lines on the sample video with previous settings.
I haven't checked with new settings to see if jerky lines appear.

[feisty2]

Win7 (x64) SP1
GTX 750 ti
GeForce 355.82 (newest; although I don't think much changed as far as video processing goes, mostly just new games)

KNLMeansCL v0.6.1
SetMemoryMax(512)
LoadPlugin("C:\AVS\dgdecnv2049\DGDecodeNV.dll")
DGSource("E:\Demux\Scream\Scream.dgi", crop_b=140, crop_t=132)
#KNLMeansCL(D=2, A=1, h=3, device_type="GPU", info=true)

"jerky lines in the background"

I don't see "jerky lines" in the sample video.
I grabbed a demux of the poorly mastered BD for Scream, since it's noisy and messy.
(Canadian BD Region A, identical to USA BD)

(all images are "large thumbnail", click to follow to original size (1920x816)
Here's original Frame #7282:
Focus: the white door frame behind Drew
http://i.imgur.com/wlbax1el.png (http://imgur.com/wlbax1e)

Here's with the settings used by MysteryX:
KNLMeansCL(D=2, A=1, h=3, device_type="GPU", info=true)
Focus: the white door frame behind Drew
http://i.imgur.com/0sRssNzl.png (http://imgur.com/0sRssNz)

Looks it denoises a lot of the main flat areas, but any edges are nearly the same.
When you scan forward, you can see this jerkiness it produces to due to unfiltered edges.
Some frames, the edges are cleaned out right.
And a large portion of (proceeding) frames, the noise varies, giving it a jerky look when played back.
At those setings.

Here's same settings, an earlier frame where the door frame is entirely denoised and clean edges.
Frame #7275:
http://i.imgur.com/yEf8hPml.png (http://imgur.com/yEf8hPm)

Frame #7276 is pretty clean too.
From there, it starts to degrade back to a noisy edge.

BUT
I changed parameters and found this:

Frame #7313
KNLMeansCL(D=3, A=0, h=3, device_type="GPU", info=true)
http://i.imgur.com/hHkWFFwl.png (http://imgur.com/hHkWFFw)
Jerky-ish lines.

Are those the kind of jerky lines you got?
I couldn't get those lines on the sample video with previous settings.
I haven't checked with new settings to see if jerky lines appear.

NLMeans degrades to something like "Motion Flow" (pixel wise motion compensation) by setting A=0 (spatial processing disabled)
and the jerky lines are just common motion flow artifacts
solution: enable spatial processing, pick an A value greater than 0

[MysteryX]

I might as well post images of the video having bad results.

Original
http://s4.postimg.org/j7n92sj4p/Original.jpg (http://postimg.org/image/j7n92sj4p/)

KNLMeansCL(D=2, A=2, h=3, device_type="GPU")
http://s4.postimg.org/6ez52v7ix/KNLMeans.jpg (http://postimg.org/image/6ez52v7ix/)

fft3dfilter(sigma=3, bt=5, bw=48, bh=48, ow=24, oh=24, sharpen=0.15)
http://s4.postimg.org/8m3dqsct5/FF3_DFilter.jpg (http://postimg.org/image/8m3dqsct5/)

[Khanattila]

This version gives me this error:
KNLMeansCL: AviSynthCreate Error (clBuildProgram!)

Same here. I use it with StaxRip x64 and Avisynth+. Same error with 0.6.0. Builds up to 0.5.9 work just fine.

Please report KNLMeansCL.txt.

Each driver has its own OpenCL compiler. I can not test all possible cases. :p

[Khanattila]

NLMeans degrades to something like "Motion Flow" (pixel wise motion compensation) by setting A=0 (spatial processing disabled)
and the jerky lines are just common motion flow artifacts
solution: enable spatial processing, pick an A value greater than 0

:thanks:

[Khanattila]

I'm using AviSynth 2.6

Also, KNLMeans is giving me better results than FF3DFilter on most videos, but why is it giving me such bad output on this video? The lines in the background end up being very jerky (I know, the source video is bad quality), but FF3DFilter is doing a MUCH better job in this particular case. In most other videos with similarly bad quality, however, KNLMeans gives a better output.

Here's the video
https://mega.nz/#!CNICBS5S!56wGjk8gtKzc9HMrPfc8ZZuCCxdGgk4ms8mPXFr0nkY

KNLMeansCL(D=2, A=1, h=3, device_type="GPU")

I might as well post images of the video having bad results.

Original
http://s4.postimg.org/j7n92sj4p/Original.jpg (http://postimg.org/image/j7n92sj4p/)

KNLMeansCL(D=2, A=2, h=3, device_type="GPU")
http://s4.postimg.org/6ez52v7ix/KNLMeans.jpg (http://postimg.org/image/6ez52v7ix/)

fft3dfilter(sigma=3, bt=5, bw=48, bh=48, ow=24, oh=24, sharpen=0.15)
http://s4.postimg.org/8m3dqsct5/FF3_DFilter.jpg (http://postimg.org/image/8m3dqsct5/)
KNLMeansCL works with similar patches. If similar patches are noise, output will have more noise.

[YAFU]

Hello.
With this script by feisty2:
http://forum.doom9.org/showpost.php?p=1736920&postcount=234

vsedit shows me the following message:
Failed to evaluate the script:
Python exception: knlm.KNLMeansCL: VapourSynthCreate error (clBuildProgram)!
Traceback (most recent call last):
File "vapoursynth.pyx", line 1469, in vapoursynth.vpy_evaluateScript (src/cython/vapoursynth.c:23703)
File "/media/TEMPORAL/scrypt.vpy", line 60, in <module>
spt = core.std.CropRel (core.knlm.KNLMeansCL(padding(clp,24,24,24,24), 0, 24, 4, h=3.2),24,24,24,24)
File "vapoursynth.pyx", line 1368, in vapoursynth.Function.__call__ (src/cython/vapoursynth.c:22272)
vapoursynth.Error: knlm.KNLMeansCL: VapourSynthCreate error (clBuildProgram)!

I use Kubuntu 64 bits. I installed KNLMeansCL from github. I needed to install the "opencl-headres" package to compile in Kubuntu.
My graphics card is a GTX 960, using NVIDIA 355.11 driver. "nvidia_uvm" seems loaded into the kernel. nVidia OpenCL in Blender 3D seems to be working properly.

What could be causing the problem?

EDIT:
I have found that a file "KNLMeansCL.txt" is generated in the same folder as the script:
http://pastebin.com/4Cvep6w8

Another question. I have two devices that can use OpenCL, Nvidia and Intel. How to indicate in the script which device KNLMeansCL should use?

[Khanattila]

Hello.
With this script by feisty2:
http://forum.doom9.org/showpost.php?p=1736920&postcount=234

vsedit shows me the following message:
Failed to evaluate the script:
Python exception: knlm.KNLMeansCL: VapourSynthCreate error (clBuildProgram)!
Traceback (most recent call last):
File "vapoursynth.pyx", line 1469, in vapoursynth.vpy_evaluateScript (src/cython/vapoursynth.c:23703)
File "/media/TEMPORAL/scrypt.vpy", line 60, in <module>
spt = core.std.CropRel (core.knlm.KNLMeansCL(padding(clp,24,24,24,24), 0, 24, 4, h=3.2),24,24,24,24)
File "vapoursynth.pyx", line 1368, in vapoursynth.Function.__call__ (src/cython/vapoursynth.c:22272)
vapoursynth.Error: knlm.KNLMeansCL: VapourSynthCreate error (clBuildProgram)!

I use Kubuntu 64 bits. I installed KNLMeansCL from github. I needed to install the "opencl-headres" package to compile in Kubuntu.
My graphics card is a GTX 960, using NVIDIA 355.11 driver. "nvidia_uvm" seems loaded into the kernel. nVidia OpenCL in Blender 3D seems to be working properly.

What could be causing the problem?

EDIT:
I have found that a file "KNLMeansCL.txt" is generated in the same folder as the script:
http://pastebin.com/4Cvep6w8

Another question. I have two devices that can use OpenCL, Nvidia and Intel. How to indicate in the script which device KNLMeansCL should use?

The plugin use the first device available of type device_type. You can check it with info = true.
Or you can manually set the default opencl device in your system environment.

EDIT. v0.6.2 is required.
Tomorrow I correct everything. Others have had problems. :p

[YAFU]

Ok, I'll be tuned waiting to v0.6.2
Thank you.

[Khanattila]

https://github.com/Khanattila/KNLMeansCL/releases/tag/v0.6.2

This release should fix all problems.

[YAFU]

Hi.
I updated from git (git pull), compiled and installed (Kubuntu Linux 64 bits, nvidia 355.11 driver)

Still having similar error in vsedit, and this is the log:
http://pastebin.com/eiWtS3ws

I uninstalled intel OpenCL driver, so nvidia is the only OpenCL device.

[sl1pkn07]

i think is problem by vsedit (have a similar problem (http://sl1pkn07.wtf/paste/view/dff3ef19))

try:

vspipe -y script.vpy -i -

or

vspipe -y script.vpy - | mpv -

[YAFU]

Hey, with these commands my graphics card started making small weird treble sounds (they were no sounds of the fan). I stopped with Ctrl + C. What does that mean? Are there risks to break my card?

[sl1pkn07]

should not. i think

http://wstaw.org/m/2015/09/04/Screenshot_20150904_204833.png


maybe is a Coil Whine (https://www.youtube.com/watch?v=834oj3d7_jo) (?)

[Are_]

Depending on your locale you may need to change it to use this filter with vsedit:

LANG=C vsedit scritp.py

[sl1pkn07]

Depending on your locale you may need to change it to use this filter with vsedit:

LANG=C vsedit scritp.py

yep. this fix the(my) problem with vsedit

[YAFU]

@sl1pkn07
Yes, quite similar to that sound in this video. The strange thing is that it is the first time I hear that sound. I had my graphics card in long sessions rendering with Blender 3D/CUDA and this sound does not appear at all (it's a pretty quiet card). Perhaps bad optimized for OpenCL?

I'm testing this script with a set of PNG (script in message #234 and png images in message #241):
http://forum.doom9.org/showthread.php?t=171207&page=12

I run:
vspipe --y4m script.vpy - | ffmpeg -i pipe: -vcodec libx264 -crf 10 encoded.mkv

My nvidia card starts making these weird sounds, but I do not get messages in the terminal indicating that the script is working and the file "encoded.mkv" is never written to disk (I cancel at 15 seconds, I do not like that sound at all).

Edit:
Ok, I tried again and in Blender (CUDA or OpenCL) also makes some of that sound, but much less (I had to open the Case cover and close to it to hear the sound). Perhaps this plugin in vapoursynth is entering some kind of loop with OpenCL and my card?

[sl1pkn07]

the script is TOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOoooooooooooooooooooooooooooooooooooooooooo........... slow

http://wstaw.org/m/2015/09/04/Screenshot_20150904_220515.png

/me pats feisty2 XD

[Are_]

Also something is wrong somewhere with that script because... pink output, but yeah, what you experienced was your computer trying to die.

And now that we are at it, Khanattila, do you know if the problems with the locales in Linux is within your plug-in or with vsedit?
It's strange, because I only experienced this with knlmeanscl + vsedit, everything else is fine.

[YAFU]

So any of you could recommend me some script with which I can reduce noise in these 16 bits png images?

The story start here:
http://forum.doom9.org/showpost.php?p=1736889&postcount=227

With that script noise reduction is very good, but color banding appear. So there they recommended me to work with 16-bit color depth images, but Fft3d does not support 16 bits.

the idea is to save some time rendering in Blender 3D with low quality, and then reducing noise with vapoursynth.

[detmek]

Please report KNLMeansCL.txt.

Each driver has its own OpenCL compiler. I can not test all possible cases. :p

With 0.6.2 (same as with 0.6.0 and .1.

:133:34: error: use of logical '||' with constant operand
:133:34: note: use '|' for a bitwise operation
:161:34: error: use of logical '||' with constant operand
:161:34: note: use '|' for a bitwise operation

error: front end compiler failed build.

I only have Intel iGPU. No dedicated GPU. Latest Intel drivers for Windows 8.1 x64.

Up to 0.5.9 everything works just fine.

[Khanattila]

With 0.6.2 (same as with 0.6.0 and .1.



I only have Intel iGPU. No dedicated GPU. Latest Intel drivers for Windows 8.1 x64.

Up to 0.5.9 everything works just fine.

It is not possible. Try to re-download the file.

[Khanattila]

Also something is wrong somewhere with that script because... pink output, but yeah, what you experienced was your computer trying to die.

And now that we are at it, Khanattila, do you know if the problems with the locales in Linux is within your plug-in or with vsedit?
It's strange, because I only experienced this with knlmeanscl + vsedit, everything else is fine.

Maybe is my problem.

EDIT. Is my problem.
123.456 float example in my locate.
123,456 float example in your locate.

[Khanattila]

So any of you could recommend me some script with which I can reduce noise in these 16 bits png images?

The story start here:
http://forum.doom9.org/showpost.php?p=1736889&postcount=227

With that script noise reduction is very good, but color banding appear. So there they recommended me to work with 16-bit color depth images, but Fft3d does not support 16 bits.

the idea is to save some time rendering in Blender 3D with low quality, and then reducing noise with vapoursynth.

If you are in this thread, you probably want to try KNLMeansCL.