KNLMeansCL: OpenCL NLMeans de-noising algorithm [2018-01-29]

[Khanattila]

However this is a bug. Thanks for the tip.

>>>> KNLMeansCL_v0.5.4.zip <<<< (http://www.mediafire.com/download/qoop11de7u95na3/KNLMeansCL_v0.5.4.zip)

ChangeLog

v0.5.4 (2015-06-06)
- Fixed: AviSynth temporal support with 16-bits clip.

[noisyfart]

Thanks for the fix!

[Pulp Catalyst]

Hi, version 0.5.0 is fine,

but version 0.5.x is always giving me same error.

opencl device not found,

if i go back to version 0.5.0, all is good.

the script is as follows

KNLMeansCL (D=2, A=1, h=1.2, device_type="GPU")

was the format changed or something, i have read the read me....but it's unfortunate that there are no included examples inside the text document.

it's strange that 0.5.0 everything is good, but all other versions the openCL device never gets found???

Regards,

p.s going back to 0.5.0

[Khanattila]

1) What graphics card do you have?
2) Have you tried device_type="default"?
2a) If it works, share a screen with info=true?
Thanks!

[Pulp Catalyst]

hi, i have a gtx 760, the default gpu is the internal intel hd4000, the GPU is the gtx....

i till wait until further versions come out, i have tried different situations, all versions after 0.5.0 all do the same.

i use MeGUI and my default script is as follows,

global MeGUI_darx = 40
global MeGUI_dary = 17
SetMemoryMax(1024)
SetMTMode(5,2)
LoadPlugin("D:\Program Files (x86)\MeGUInew\tools\dgindexnv\DGDecodeNV.dll")
DGSource("E:\MeGUI\rsxyxu0x.yip\05 You only live twice (1967).dgi",fieldop=0)
SetMTMode(5,0)
KNLMeansCL(D=2, A=1, h=1.2, device_type="GPU")
SetMTMode(2)
#deinterlace
#crop
#resize
#denoise


when i replace KNLMeansCL with newer versions i get errors.

i do also use DGDecodeNV.... not sure if have any bearings???

thanks,

[Khanattila]

Imo this gives you problems: SetMTMode(5,0).
* Mode 5 is slowest (slower than not using SetMTMode) but should work with all filters that don't require linear frameserving (that is, the frames come in order (frame 0,1,2 ... last)).
* Threads. Set to 0 to set it to the number of processors available.

So you're calling four or eight times KNLMeansCL. Set threads=1.
Really, you do not need to use MT for OpenCL / CUDA plugin.

[Groucho2004]

Imo this gives you problems: SetMTMode(5,0).
* Mode 5 is slowest (slower than not using SetMTMode) but should work with all filters that don't require linear frameserving (that is, the frames come in order (frame 0,1,2 ... last)).
* Threads. Set to 0 to set it to the number of processors available.

So you're calling four or eight times KNLMeansCL. Set threads=1.
Really, you do not need to use MT for OpenCL / CUDA plugin.
Nope. The first call of SetMTMode(x, n) sets the number of threads (n). Any subsequent change of "n" will be ignored.

He should try a simple script without MT calls, post the "info" screen as you requested and be more verbose than "with newer versions i get errors".

[Khanattila]

Nope. The first call of SetMTMode(x, n) sets the number of threads (n). Any subsequent change of "n" will be ignored.

He should try a simple script without MT calls, post the "info" screen as you requested and be more verbose than "with newer versions i get errors".

Ah! It's even worse than I thought! Really.... why use MT?

[Groucho2004]

Really.... why use MT?
Beats me.

[Pulp Catalyst]

the script was created very carefully a long time ago now using the amazing tool "AVSMeter" (currently using 1.98)

in the learning process, i found some things to follow.

1. to use as little of the CPU as possible so to maximize available resources to the encoder
2. to stabilize the CPU usage of 1 or 2 cores so the pipeline going to the encoder is constant and consistent (i don't want flucutations of the CPU)
3. to use as little threads as possible, this helps with (1) and (2)
4. to make sure the GPU is feeding enough data (decoding video) quick enough to maximize the encoder FPS (this was a little tricky now i'm using x265, but fortunately my GPU can just about decode enough video for x265 to use max CPU 9accept for the 1core that needs to be given up for nvidia).

if i don't use MT, i get an avereage FPS in AVSMeter 16.76
with MT i get 18.92

now if i use
global MeGUI_darx = 40
global MeGUI_dary = 17
SetMemoryMax(1024)
SetMTMode(5,2)
LoadPlugin("D:\Program Files (x86)\MeGUInew\tools\dgindexnv\DGDecodeNV.dll")
DGSource("E:\MeGUI\rsxyxu0x.yip\05 You only live twice (1967).dgi",fieldop=0)
SetMTMode(5,1)
KNLMeansCL(D=2, A=1, h=1.2, device_type="GPU")
SetMTMode(2)
#deinterlace
#crop
#resize
#denoise

my FPS goes to 20.82 (faster),
however what i found a long time ago is the thread count goes from 13 to 19, and AVSMeter shows the CPU usage of the 1st core becomes erratic, bouncing from 8% to 17% which is strange because i don't ever get over 13% when using the script i have above (with hyper-threading i have 8 cores)

my real live results showed that the x264 or now x265 encoder gives much inferior results when this behaviour is exhibited, however when using the script that utilizes only 13 threads and the CPU usage remains nice and steady at 13%.... the encoder gives far better performance.

i remember spending a very long time finding a really good balance between CPU/GPU (my specific hardware) and getting this to work smoothly with the encoder (this was a tricky part).

AVSMeter helped me stabilize the efficiency of the flow chain, x264/x265 FPS showed me actual performance gains (sometimes getting more speed from AVSMeter actually hurt performance from the encoder, i believe this is because of the erratic behaviour that happens when i take the thread count from 13 to 19)

i won't pretend to understand what is going on behind the scenes with AVISynth MT, because i don't at all. However real live results and analysis of performance told me when i found the "SWEET SPOT", the script may look odd, and to some not make any sense...clearly, but it gives me the best performance and balance on my RIG... and this is much more important to me then things making sense...

anyway.... moving forward, i'm still at a lost why

Version 0.5.1
Version 0.5.2
Version 0.5.3
Version 0.5.4

all gives me the same error, OpenCL device not found, however Version 0.5.0 works perfectly???

But i'm sure time will answer this dilemma...

anyway thank you all for your feed back, i always appreciate feedback off all kind.

[Groucho2004]

global MeGUI_darx = 40
global MeGUI_dary = 17
SetMemoryMax(1024)
SetMTMode(5,2)
LoadPlugin("D:\Program Files (x86)\MeGUInew\tools\dgindexnv\DGDecodeNV.dll")
DGSource("E:\MeGUI\rsxyxu0x.yip\05 You only live twice (1967).dgi",fieldop=0)
SetMTMode(5,1)
KNLMeansCL(D=2, A=1, h=1.2, device_type="GPU")
SetMTMode(2)
#deinterlace
#crop
#resize
#denoise
That script doesn't even work, it does not return a clip because "SetMTMode(2)" is the last statement.

The "SetMTMode(5,1)" is completely pointless (since you can't change the number of threads after the first call) as is "SetMTMode(2)" at the end.

I tried your script with and without MT (adding "return last" at the end to get it to work), the speed difference in AVSMeter is virtually zero. If you understood how AVS MT works or read the documentation you would know that this is the expected result.

[Khanattila]

anyway.... moving forward, i'm still at a lost why

Version 0.5.1
Version 0.5.2
Version 0.5.3
Version 0.5.4

all gives me the same error, OpenCL device not found, however Version 0.5.0 works perfectly???

But i'm sure time will answer this dilemma...

anyway thank you all for your feed back, i always appreciate feedback off all kind.

Because Version 0.5.* != 0.5.0. In Version 0.5.1 I rewrote a lot of things...
Anyway I have a GTX 760, then it should work.

[Khanattila]

>>>> KNLMeansCL_v0.5.5.zip <<<< (http://www.mediafire.com/download/0h0cuii38ecoyh4/KNLMeansCL_v0.5.5.zip)

ChangeLog

v0.5.5 (2015-06-25)
- New: AviSynth RGB32 support.


VS takes longer.

[Khanattila]

#RGB32
Colorbars(1920, 1080, "RGB32").KillAudio().KNLMeansCL()

#YV24
Colorbars(1920, 1080, "YV24").KillAudio()
y = KNLMeansCL()
u = UToY8().KNLMeansCL()
v = VToY8().KNLMeansCL()
YToUV(u, v, y)

#YV12
Colorbars(1920, 1080, "YV12").KillAudio()
y = KNLMeansCL()
u = UToY8().KNLMeansCL()
v = VToY8().KNLMeansCL()
YToUV(u, v, y)

#Y8
Colorbars(1920, 1080, "YV12").KillAudio()


RGB32 ----- 46.36 FPS
YV24 ------ 22.49 FPS
YV12 ------ 43.58 FPS
Y8--------- 68.98 FPS

GPU architecture. Lovable.

[RazorBurn]

Does KNLMeansCL support Temporal in Vapoursynth?

Got Error on this

clip = core.knlm.KNLMeansCL (clip = clip, d=1, a=0, h=5.4, device_type="GPU") ##KNLMeans Temporal Medium


clip = core.knlm.KNLMeansCL (clip = clip, d=1, a=1, h=4.5, device_type="GPU") ##KNLMeans Spatio-Temporal Medium


Yet no Error on this

clip = core.knlm.KNLMeansCL (clip = clip, d=0, a=4, h=3.6, device_type="GPU") ##KNLMeans Spatial Medium

[Khanattila]

Does KNLMeansCL support Temporal in Vapoursynth?

Got Error on this

clip = core.knlm.KNLMeansCL (clip = clip, d=1, a=0, h=5.4, device_type="GPU") ##KNLMeans Temporal Medium


clip = core.knlm.KNLMeansCL (clip = clip, d=1, a=1, h=4.5, device_type="GPU") ##KNLMeans Spatio-Temporal Medium


Yet no Error on this

clip = core.knlm.KNLMeansCL (clip = clip, d=0, a=4, h=3.6, device_type="GPU") ##KNLMeans Spatial Medium



I have to investigate, some more details?

[tormento]

Latest version finding OpenCL device (Nvidia GTX 760) is 0.5.0, latest driver version on Win 8.1_x64.

[RazorBurn]

I have to investigate, some more details?

Script ==>
import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(r'C:\Apps\Encoders\StaxRip\Apps\Plugins\both\ffms2\ffms2.dll')
clip = core.ffms2.Source(source = r'C:\Temp\Media\SampleClip.mpg', cachefile = r'C:\Temp\Media\SampleClip temp files\SampleClip.ffindex')
clip = core.knlm.KNLMeansCL (clip = clip, d=1, a=1, h=4.5, device_type="GPU")
clip.set_output()


CMD Line ==

vspipe.exe" "C:\Temp\Media\SampleClip temp files\SampleClip.vpy" - --y4m | x264.exe --crf 22 --demuxer y4m --frames 59 --output "C:\Temp\Media\SampleClip temp files\SampleClip_out.h264" "-"

Error Log ==

Negative frame request by: Cache1
x264 [error]: could not open input file `-'

SampleClip (From NeatVideo) ==

https://www.neatvideo.com/files4/testkit.zip

[Khanattila]

It should fix everything.

>>>> KNLMeansCL_v0.5.6.zip <<<< (http://www.mediafire.com/download/plzsx3i46yp3d0g/KNLMeansCL_v0.5.6.zip)

ChangeLog

v0.5.6 (2015-06-26)
- Fixed: VapourSynth temporal support (regression v0.5.2)
- Fixed: VapourSynth device_type ignored.
- Fixed: Possible opencl device not available.

[Monarc]

Hi,

i tried to use your filter 0.5.6 with vapoursynth under linux.

It compiles after some small changes, but vapoursynth stops with errors.

vs script:


import vapoursynth as vs
core = vs.get_core()

file="/home/monarc/Downloads/test.mp4"
clip = core.ffms2.Source(source=file)

clip = core.knlm.KNLMeansCL (clip = clip, d=0, a=4, h=3.6, info=True)
clip.set_output()


vapoursynth error:


Failed to evaluate the script:
Python exception: knlm.KNLMeansCL: VapourSynthCreate error (1)!
Traceback (most recent call last):
File "vapoursynth.pyx", line 1469, in vapoursynth.vpy_evaluateScript (src/cython/vapoursynth.c:23703)
File "/home/monarc/filters/filters.vpy", line 50, in <module>
clip = core.knlm.KNLMeansCL (clip = clip, d=0, a=4, h=3.6, info=True)
File "vapoursynth.pyx", line 1368, in vapoursynth.Function.__call__ (src/cython/vapoursynth.c:22272)
vapoursynth.Error: knlm.KNLMeansCL: VapourSynthCreate error (1)!


KNLMeansCL.txt

<kernel>:90:40: error: invalid digit 'f' in decimal constant
(NLMK_WMODE == 2) ? fmax(1.0f -sum * NLMK_H2_INV_NORM, 0.0f) * fmax(1.0f -sum * NLMK_H2_INV_NORM, 0.0f) :
^
<built-in>:31:35: note: expanded from here
#define NLMK_H2_INV_NORM 61,942730f
^
<kernel>:90:83: error: invalid digit 'f' in decimal constant
(NLMK_WMODE == 2) ? fmax(1.0f -sum * NLMK_H2_INV_NORM, 0.0f) * fmax(1.0f -sum * NLMK_H2_INV_NORM, 0.0f) :
^
<built-in>:31:35: note: expanded from here
#define NLMK_H2_INV_NORM 61,942730f
^
<kernel>:91:41: error: invalid digit 'f' in decimal constant
(NLMK_WMODE == 1) ? native_exp(-sum * NLMK_H2_INV_NORM) :
^
<built-in>:31:35: note: expanded from here
#define NLMK_H2_INV_NORM 61,942730f
^
<kernel>:92:29: error: invalid digit 'f' in decimal constant
native_recip(1.0f + sum * NLMK_H2_INV_NORM);
^
<built-in>:31:35: note: expanded from here
#define NLMK_H2_INV_NORM 61,942730f
^
^@^@


I am not sure, if this is even possible with my card:


clinfo --help
Number of platforms: 1
Platform Profile: FULL_PROFILE
Platform Version: OpenCL 1.2 CUDA 7.5.9
Platform Name: NVIDIA CUDA
Platform Vendor: NVIDIA Corporation
Platform Extensions: cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts


Platform Name: NVIDIA CUDA
Number of devices: 1
Device Type: CL_DEVICE_TYPE_GPU
Device ID: 4318
Max compute units: 8
Max work items dimensions: 3
Max work items[0]: 1024
Max work items[1]: 1024
Max work items[2]: 64
Max work group size: 1024
Preferred vector width char: 1
Preferred vector width short: 1
Preferred vector width int: 1
Preferred vector width long: 1
Preferred vector width float: 1
Preferred vector width double: 1
Native vector width char: 1
Native vector width short: 1
Native vector width int: 1
Native vector width long: 1
Native vector width float: 1
Native vector width double: 1
Max clock frequency: 1760Mhz
Address bits: 64
Max memory allocation: 268222464
Image support: Yes
Max number of images read arguments: 128
Max number of images write arguments: 8
Max image 2D width: 16384
Max image 2D height: 16384
Max image 3D width: 2048
Max image 3D height: 2048
Max image 3D depth: 2048
Max samplers within kernel: 16
Max size of kernel argument: 4352
Alignment (bits) of base address: 4096
Minimum alignment (bytes) for any datatype: 128
Single precision floating point capability
Denorms: Yes
Quiet NaNs: Yes
Round to nearest even: Yes
Round to zero: Yes
Round to +ve and infinity: Yes
IEEE754-2008 fused multiply-add: Yes
Cache type: Read/Write
Cache line size: 128
Cache size: 131072
Global memory size: 1072889856
Constant buffer size: 65536
Max number of constant args: 9
Local memory type: Local
Local memory size: 49152
Error correction support: 0
Unified memory for Host and Device: 0
Profiling timer resolution: 1000
Device endianess: Little
Available: Yes
Compiler available: Yes
Execution capabilities:
Execute OpenCL kernels: Yes
Execute native function: No
Queue properties:
Out-of-Order: Yes
Profiling : Yes
Platform ID: 0x234f7f0
Name: GeForce GTX 560 Ti
Vendor: NVIDIA Corporation
Device OpenCL C version: OpenCL C 1.1
Driver version: 352.21
Profile: FULL_PROFILE
Version: OpenCL 1.1 CUDA
Extensions: cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64

[Khanattila]

Can you tell me what you had to change? So I implement them.
However, it is a stupid mistake. Just do not use TAB with snprintf. Or just do not break line ("\").

if (channel == CL_FLOAT) {
snprintf(options, 2048, "-Werror -D H_BLOCK_X=%i -D H_BLOCK_Y=%i -D V_BLOCK_X=%i -D V_BLOCK_Y=%i -D NLMK_RGB=%i -D NLMK_S=%i -D NLMK_WMODE=%i -D NLMK_TEMPORAL=%i -D NLMK_H2_INV_NORM=%ff",
H_BLOCK_X, H_BLOCK_Y, V_BLOCK_X, V_BLOCK_Y, 0, d.s, d.wmode, d.d, 65025.0 / (d.h*d.h*(2 * d.s + 1) * (2 * d.s + 1)));
} else {
snprintf(options, 2048, "-cl-denorms-are-zero -cl-fast-relaxed-math -Werror -D H_BLOCK_X=%i -D H_BLOCK_Y=%i -D V_BLOCK_X=%i -D V_BLOCK_Y=%i -D NLMK_RGB=%i -D NLMK_S=%i -D NLMK_WMODE=%i -D NLMK_TEMPORAL=%i -D NLMK_H2_INV_NORM=%ff",
H_BLOCK_X, H_BLOCK_Y, V_BLOCK_X, V_BLOCK_Y, 0, d.s, d.wmode, d.d, 65025.0 / (d.h*d.h*(2 * d.s + 1) * (2 * d.s + 1)));
}

[Monarc]

Does not change much ... maybe i do something wrong.

Even with all changes, vapoursynth error remains the same.

KNLMeansCL.txt:

<kernel>:90:40: error: invalid digit 'f' in decimal constant
(NLMK_WMODE == 2) ? fmax(1.0f -sum * NLMK_H2_INV_NORM, 0.0f) * fmax(1.0f -sum * NLMK_H2_INV_NORM, 0.0f) :
^
<built-in>:31:35: note: expanded from here
#define NLMK_H2_INV_NORM 61,942730f
^
<kernel>:90:83: error: invalid digit 'f' in decimal constant
(NLMK_WMODE == 2) ? fmax(1.0f -sum * NLMK_H2_INV_NORM, 0.0f) * fmax(1.0f -sum * NLMK_H2_INV_NORM, 0.0f) :
^
<built-in>:31:35: note: expanded from here
#define NLMK_H2_INV_NORM 61,942730f
^
<kernel>:91:41: error: invalid digit 'f' in decimal constant
(NLMK_WMODE == 1) ? native_exp(-sum * NLMK_H2_INV_NORM) :
^
<built-in>:31:35: note: expanded from here
#define NLMK_H2_INV_NORM 61,942730f
^
<kernel>:92:29: error: invalid digit 'f' in decimal constant
native_recip(1.0f + sum * NLMK_H2_INV_NORM);
^
<built-in>:31:35: note: expanded from here
#define NLMK_H2_INV_NORM 61,942730f



to compile under linux, i tried this line:


clang++-3.7 -o ../KNLMeansCL.so -std=c++11 -O3 -march=native -mtune=native -I/usr/local/include/vapoursynth -shared -fPIC -lOpenCL KNLMeansCL.cpp


in KNLMeansCL.h:

i wrapped #include "avisynth.h" with #ifdef _WIN32
and added #include "math.h" for ceil

in KNLMeansCL.cpp:
added a space between " Version " VERSION in line 454


diff -r -u3 KNLMeansCL/src/KNLMeansCL.cpp KNLMeansCL.patch/src/KNLMeansCL.cpp
--- KNLMeansCL/src/KNLMeansCL.cpp 2015-06-26 16:31:48.000000000 +0200
+++ KNLMeansCL.patch/src/KNLMeansCL.cpp 2015-06-26 21:37:16.362359041 +0200
@@ -324,7 +324,7 @@
int pitch = dst->GetPitch(PLANAR_Y);
char buffer[2048], str[2048], str1[2048];
DrawString(frm, pitch, 0, y++, "KNLMeansCL");
- DrawString(frm, pitch, 0, y++, " Version "VERSION);
+ DrawString(frm, pitch, 0, y++, " Version " VERSION);
DrawString(frm, pitch, 0, y++, " Copyright(C) Khanattila");
snprintf(buffer, 2048, " D:%i A:%ix%i S:%ix%i", 2 * D + 1, 2 * A + 1, 2 * A + 1, 2 * S + 1, 2 * S + 1);
DrawString(frm, pitch, 0, y++, buffer);
@@ -451,7 +451,7 @@
int pitch = vsapi->getStride(dst, 0);
char buffer[2048], str[2048], str1[2048];
DrawString(frm, pitch, 0, y++, "KNLMeansCL");
- DrawString(frm, pitch, 0, y++, " Version "VERSION);
+ DrawString(frm, pitch, 0, y++, " Version " VERSION);
DrawString(frm, pitch, 0, y++, " Copyright(C) Khanattila");
snprintf(buffer, 2048, " D:%i A:%ix%i S:%ix%i", 2 * d->d + 1, 2 * d->a + 1, 2 * d->a + 1, 2 * d->s + 1, 2 * d->s + 1);
DrawString(frm, pitch, 0, y++, buffer);
@@ -703,15 +703,9 @@
d.program = clCreateProgramWithSource(d.context, 1, &source_code, NULL, NULL);
char options[2048];
if (channel == CL_FLOAT) {
- snprintf(options, 2048, "-Werror \
- -D H_BLOCK_X=%i -D H_BLOCK_Y=%i -D V_BLOCK_X=%i -D V_BLOCK_Y=%i \
- -D NLMK_RGB=%i -D NLMK_S=%i -D NLMK_WMODE=%i -D NLMK_TEMPORAL=%i -D NLMK_H2_INV_NORM=%ff",
- H_BLOCK_X, H_BLOCK_Y, V_BLOCK_X, V_BLOCK_Y, 0, d.s, d.wmode, d.d, 65025.0 / (d.h*d.h*(2 * d.s + 1) * (2 * d.s + 1)));
+ snprintf(options, 2048, "-Werror -D H_BLOCK_X=%i -D H_BLOCK_Y=%i -D V_BLOCK_X=%i -D V_BLOCK_Y=%i -D NLMK_RGB=%i -D NLMK_S=%li -D NLMK_WMODE=%li -D NLMK_TEMPORAL=%li -D NLMK_H2_INV_NORM=%ff",H_BLOCK_X, H_BLOCK_Y, V_BLOCK_X, V_BLOCK_Y, 0, d.s, d.wmode, d.d, 65025.0 / (d.h*d.h*(2 * d.s + 1) * (2 * d.s + 1)));
} else {
- snprintf(options, 2048, "-cl-denorms-are-zero -cl-fast-relaxed-math -Werror \
- -D H_BLOCK_X=%i -D H_BLOCK_Y=%i -D V_BLOCK_X=%i -D V_BLOCK_Y=%i \
- -D NLMK_RGB=%i -D NLMK_S=%i -D NLMK_WMODE=%i -D NLMK_TEMPORAL=%i -D NLMK_H2_INV_NORM=%ff",
- H_BLOCK_X, H_BLOCK_Y, V_BLOCK_X, V_BLOCK_Y, 0, d.s, d.wmode, d.d, 65025.0 / (d.h*d.h*(2 * d.s + 1) * (2 * d.s + 1)));
+ snprintf(options, 2048, "-cl-denorms-are-zero -cl-fast-relaxed-math -Werror -D H_BLOCK_X=%i -D H_BLOCK_Y=%i -D V_BLOCK_X=%i -D V_BLOCK_Y=%i -D NLMK_RGB=%i -D NLMK_S=%li -D NLMK_WMODE=%li -D NLMK_TEMPORAL=%li -D NLMK_H2_INV_NORM=%ff",H_BLOCK_X, H_BLOCK_Y, V_BLOCK_X, V_BLOCK_Y, 0, d.s, d.wmode, d.d, 65025.0 / (d.h*d.h*(2 * d.s + 1) * (2 * d.s + 1)));
}
ret = clBuildProgram(d.program, 1, &d.deviceID, options, NULL, NULL);
if (ret != CL_SUCCESS) {

diff -r -u3 KNLMeansCL/src/KNLMeansCL.h KNLMeansCL.patch/src/KNLMeansCL.h
--- KNLMeansCL/src/KNLMeansCL.h 2015-06-26 16:32:42.000000000 +0200
+++ KNLMeansCL.patch/src/KNLMeansCL.h 2015-06-26 18:52:07.686612969 +0200
@@ -31,7 +31,10 @@
#include <CL/cl.h>
#include "kernel.h"
#include "startchar.h"
+#ifdef _WIN32
#include "avisynth.h"
+#endif
+#include "math.h"
#include "VapourSynth.h"
#include "VSHelper.h"



then it compiles (with warnings):


clang++-3.7 -ferror-limit=0 -o ../KNLMeansCL.so -std=c++11 -O3 -march=native -mtune=native -I/usr/local/include/vapoursynth -shared -fPIC -lOpenCL KNLMeansCL.cpp
KNLMeansCL.cpp:397:60: warning: implicit conversion of NULL constant to 'cl_uint' (aka 'unsigned int') [-Wnull-conversion]
vsapi->getStride(src, 0), 0, vsapi->getReadPtr(src, 0), NULL, NULL, NULL);
^~~~
0
KNLMeansCL.cpp:405:36: warning: implicit conversion of NULL constant to 'cl_uint' (aka 'unsigned int') [-Wnull-conversion]
0, vsapi->getReadPtr(src, 0), NULL, NULL, NULL);
^~~~
0
KNLMeansCL.cpp:439:35: warning: implicit conversion of NULL constant to 'cl_uint' (aka 'unsigned int') [-Wnull-conversion]
0, vsapi->getWritePtr(dst, 0), NULL, NULL, NULL);
^~~~
0
KNLMeansCL.cpp:456:54: warning: format specifies type 'int' but the argument has type 'long' [-Wformat]
snprintf(buffer, 2048, " D:%i A:%ix%i S:%ix%i", 2 * d->d + 1, 2 * d->a + 1, 2 * d->a + 1, 2 * d->s + 1, 2 * d->s + 1);
~~ ^~~~~~~~~~~~
%li
KNLMeansCL.cpp:456:68: warning: format specifies type 'int' but the argument has type 'long' [-Wformat]
snprintf(buffer, 2048, " D:%i A:%ix%i S:%ix%i", 2 * d->d + 1, 2 * d->a + 1, 2 * d->a + 1, 2 * d->s + 1, 2 * d->s + 1);
~~ ^~~~~~~~~~~~
%li
KNLMeansCL.cpp:456:82: warning: format specifies type 'int' but the argument has type 'long' [-Wformat]
snprintf(buffer, 2048, " D:%i A:%ix%i S:%ix%i", 2 * d->d + 1, 2 * d->a + 1, 2 * d->a + 1, 2 * d->s + 1, 2 * d->s + 1);
~~ ^~~~~~~~~~~~
%li
KNLMeansCL.cpp:456:96: warning: format specifies type 'int' but the argument has type 'long' [-Wformat]
snprintf(buffer, 2048, " D:%i A:%ix%i S:%ix%i", 2 * d->d + 1, 2 * d->a + 1, 2 * d->a + 1, 2 * d->s + 1, 2 * d->s + 1);
~~ ^~~~~~~~~~~~
%li
KNLMeansCL.cpp:456:110: warning: format specifies type 'int' but the argument has type 'long' [-Wformat]
snprintf(buffer, 2048, " D:%i A:%ix%i S:%ix%i", 2 * d->d + 1, 2 * d->a + 1, 2 * d->a + 1, 2 * d->s + 1, 2 * d->s + 1);
~~ ^~~~~~~~~~~~
%li
KNLMeansCL.cpp:458:46: warning: format specifies type 'int' but the argument has type 'long' [-Wformat]
snprintf(buffer, 2048, " Iterations: %i", ((2 * d->d + 1)*(2 * d->a + 1)*(2 * d->a + 1) - 1) / (d->d ? 1 : 2));
~~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
%li
KNLMeansCL.cpp:460:55: warning: format specifies type 'int' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
snprintf(buffer, 2048, " Global work size: %ix%i", global_work[0], global_work[1]);
~~ ^~~~~~~~~~~~~~
%zu
KNLMeansCL.cpp:460:71: warning: format specifies type 'int' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
snprintf(buffer, 2048, " Global work size: %ix%i", global_work[0], global_work[1]);
~~ ^~~~~~~~~~~~~~
%zu
KNLMeansCL.cpp:664:52: warning: implicit conversion of NULL constant to 'cl_uint' (aka 'unsigned int') [-Wnull-conversion]
ret |= clGetDeviceIDs(temp_platforms[i], device, NULL, NULL, &num_devices);
~~~~~~~~~~~~~~ ^~~~
0
KNLMeansCL.cpp:694:16: warning: 'clCreateImage2D' is deprecated [-Wdeprecated-declarations]
d.mem_in[0] = clCreateImage2D(d.context, CL_MEM_READ_ONLY, &image_format, d.image_dimensions[0], d.image_dimensions[1], 0, NULL, NULL);
^
/usr/include/CL/cl.h:1170:1: note: 'clCreateImage2D' has been explicitly marked deprecated here
clCreateImage2D(cl_context /* context */,
^
KNLMeansCL.cpp:695:25: warning: 'clCreateImage2D' is deprecated [-Wdeprecated-declarations]
if (d.d) d.mem_in[1] = clCreateImage2D(d.context, CL_MEM_READ_ONLY, &image_format, d.image_dimensions[0], d.image_dimensions[1], 0, NULL, NULL);
^
/usr/include/CL/cl.h:1170:1: note: 'clCreateImage2D' has been explicitly marked deprecated here
clCreateImage2D(cl_context /* context */,
^
KNLMeansCL.cpp:696:14: warning: 'clCreateImage2D' is deprecated [-Wdeprecated-declarations]
d.mem_out = clCreateImage2D(d.context, CL_MEM_WRITE_ONLY, &image_format, d.image_dimensions[0], d.image_dimensions[1], 0, NULL, NULL);
^
/usr/include/CL/cl.h:1170:1: note: 'clCreateImage2D' has been explicitly marked deprecated here
clCreateImage2D(cl_context /* context */,
^
15 warnings generated.

[Khanattila]

Try to add "-cl-single-precision-constant" in clBuildProgram options.
And / Or change "-D NLMK_H2_INV_NORM=%ff" to "-D NLMK_H2_INV_NORM=%f".

[feisty2]

fast yv24
y=converttoy8 ()
u=utoy8 ()
v=vtoy8 ()
fakergb=mergergb (y,u,v).converttorgb32 ()
nlm=fakergb.knlmeanscl (xxx)
y=nlm.showred ("y8")
u=nlm.showgreen ("y8")
v=nlm.showblue ("y8")
ytouv (u,v,y)

[Khanattila]

fast yv24
y=converttoy8 ()
u=utoy8 ()
v=vtoy8 ()
fakergb=mergergb (y,u,v).converttorgb32 ()
nlm=fakergb.knlmeanscl (xxx)
y=nlm.showred ("y8")
u=nlm.showgreen ("y8")
v=nlm.showblue ("y8")
ytouv (u,v,y)

This however is not gamma-correct. (Even if colour perception is not exactly orthogonal...)

Example:
Red RGB(255, 0, 0) YUV(76, 84, 255)
Violet RGB(127, 0, 255) YUV(67, 234, 170)

Squared Euclidean distance between two points p(r, g, b) and q(r, g, b).

d^2(p, q) = (p.r -q.r)^2 + (p.g - q.g)^2 + (p.b - q.b)^2
d^2(p, q) = (255 - 127)^2 + (0 - 0)^2 + (0 - 255)^2
d^2(p, q) = 81409

Squared Euclidean distance between two points p(y, u, v) and q(y, u, v).

d^2(p, q) = (p.y - q.y)^2 + (p.u - q.u) ^2 + (p.v - q.u)^2
d^2(p, q) = (76 - 64)^2 + (84 - 234)^2 + (255 - 170)^2
d^2(p, q) = 29869

EDIT.
Some people suggested a weighted Euclidean distance in RGB, example:
d^2(p, q) = 3*(p.r -q.r)^2 + 4*(p.g - q.g)^2 + 2*(p.b - q.b)^2 = 179202
d^2(p, q) = 2*(p.r -q.r)^2 + 4*(p.g - q.g)^2 + 3*(p.b - q.b)^2 = 227843
Future investigations are needed.

[Monarc]

#define NLMK_H2_INV_NORM 61,942730f


its snprintf and my local language settings (german)

In german floats are written in plain text with a comma : e.g. 3,14
In english with point: 3.14

if i start vapoursynth with LC_ALL=C, the error is gone.

[Khanattila]

#define NLMK_H2_INV_NORM 61,942730f


its snprintf and my local language settings (german)

In german floats are written in plain text with a comma : e.g. 3,14
In english with point: 3.14

if i start vapoursynth with LC_ALL=C, the error is gone.

Easier than expected

[Groucho2004]

I want to try Dithertools to denoise with 16 bit precision. Would the following code be correct?

Dither_convert_8_to_16()

y = KNLMeansCL(D = 1, A = 1, h = 1.5, lsb_inout = true)
u = UToY8().KNLMeansCL(D = 1, A = 1, h = 1.5, lsb_inout = true)
v = VToY8().KNLMeansCL(D = 1, A = 1, h = 1.5, lsb_inout = true)
YToUV(u, v, y)

f3kdb(sample_mode = 2, keep_tv_range = true, input_mode = 1, output_mode = 1)

DitherPost()

[Khanattila]

Yes of course.

[feisty2]

This however is not gamma-correct. (Even if colour perception is not exactly orthogonal...)

Example:
Red RGB(255, 0, 0) YUV(76, 84, 255)
Violet RGB(127, 0, 255) YUV(67, 234, 170)

Squared Euclidean distance between two points p(r, g, b) and q(r, g, b).

d^2(p, q) = (p.r -q.r)^2 + (p.g - q.g)^2 + (p.b - q.b)^2
d^2(p, q) = (255 - 127)^2 + (0 - 0)^2 + (0 - 255)^2
d^2(p, q) = 81409

Squared Euclidean distance between two points p(y, u, v) and q(y, u, v).

d^2(p, q) = (p.y - q.y)^2 + (p.u - q.u) ^2 + (p.v - q.u)^2
d^2(p, q) = (76 - 64)^2 + (84 - 234)^2 + (255 - 170)^2
d^2(p, q) = 29869

EDIT.
Some people suggested a weighted Euclidean distance in RGB, example:
d^2(p, q) = 3*(p.r -q.r)^2 + 4*(p.g - q.g)^2 + 2*(p.b - q.b)^2 = 179202
d^2(p, q) = 2*(p.r -q.r)^2 + 4*(p.g - q.g)^2 + 3*(p.b - q.b)^2 = 227843
Future investigations are needed.

guess b should have the least weight, so 3r+4g+2b kinda stuff, if the image comes from an actually camera, not CGI, blue channel will be VERY NOISY (compared to R,G), so not good to make that as a reference of the noise estimation at all

[Khanattila]

Another possibility is to use mahalanobis distance but noise covariance matrix is required.

[feisty2]

I'd just get green channel to do the weighting stuff, cuz green channel is generally the channel got the highest quality in real life images, least noisy, and got more details than RB.

[feisty2]

request: can you add an extra reference clip option (default = input clip) to do the weighting calculation?
like "prefilter" for mvtools, you can use it to do a lot of things, like, use a blunt and aggressive filter to remove nasty artifacts like dot crawl, then get the dif between the filtered clean clip and the original noisy clip, then filter the dif with delicate filters like nlmeans, weights calculated from the clean clip, then add the dif back to the clean clip and you can remove the artifacts and save a lot of details from this

[Khanattila]

request: can you add an extra reference clip option (default = input clip) to do the weighting calculation?
like "prefilter" for mvtools, you can use it to do a lot of things, like, use a blunt and aggressive filter to remove nasty artifacts like dot crawl, then get the dif between the filtered clean clip and the original noisy clip, then filter the dif with delicate filters like nlmeans, weights calculated from the clean clip, then add the dif back to the clean clip and you can remove the artifacts and save a lot of details from this

I don't know if you can get good results, but I will add this feature.

[feisty2]

I don't know if you can get good results, but I will add this feature.

amazing! :)
well, we'll have to test it first to get the answers :p

[sl1pkn07]

Hi

EDIT3: I've open a git repo with the code and a small configure/makefile. with the patches applied and some more things

https://github.com/sl1pkn07/KNLmeansCL

greetings

[GMJCZP]

Hello friends:

He wanted to know if there is a possibility to use this filter without a video card. I have a PC Foxconn G41MX-F 2.0 with an Intel Core 2 Duo E4400 processor with Windows XP SP3 installed, so the only way to have OpenCL was installed here:

http://www.wings3d.com/forum/showthread.php?tid=174

I installed the a) package (ATI Stream SDK v2) but get the following error message:

"opencl device not available!"

And I tried all the alternatives (DEFAULT, CPU, GPU, ACCELERATOR) and nothing.

Does the SDK package are not drivers or the correct version? What am I doing wrong? Do you simply can not use this plugin without video card? Thanks.

[feisty2]

remove winxp and get win10 :)

[sl1pkn07]

I tried to install 2 OpenCL vendors, one for Nvidia GPU and Intel only for CPU

http://sl1pkn07.wtf/paste/view/696b9533

when set 'device_type="CPU"', the "decoding" is slowwwwwwwww like ass ("spected", in dual Xeon X5650 (24Threads))

but when set 'device_type="GPU"', the VSpipe get segfault

[967221.403446] vspipe[8062]: segfault at 7f949359cb88 ip 00007f949359cb88 sp 00007ffd9162c448 error 15 in libc-2.21.so[7f949359c000+2000]

but when no set 'device_type=', knlmeans use the GPU.

how to debug this?

knlmeans is builded with

CXXFLAGS = -O2 -std=c++11 -Wall -Wno-unused-local-typedefs -I. -I./include -march=native -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/vapoursynth
LDFLAGS = -shared -fPIC -Wl,-O1,--sort-common,--as-needed,-z,relro

in linux with
gcc 5.1.0
ocl-icd 2.2.7 (OpenCL loader)
nvidia 352.21 beta drivers
Cuda 7.0.28
Intel OpenCL for CPU 15.1 (5.0.0.57)

[Khanattila]

Hello friends:

He wanted to know if there is a possibility to use this filter without a video card. I have a PC Foxconn G41MX-F 2.0 with an Intel Core 2 Duo E4400 processor with Windows XP SP3 installed, so the only way to have OpenCL was installed here:

http://www.wings3d.com/forum/showthread.php?tid=174

I installed the a) package (ATI Stream SDK v2) but get the following error message:

"opencl device not available!"

And I tried all the alternatives (DEFAULT, CPU, GPU, ACCELERATOR) and nothing.

Does the SDK package are not drivers or the correct version? What am I doing wrong? Do you simply can not use this plugin without video card? Thanks.

1) Sure. You can use it without a graphics card.
2) You should install the Intel driver: https://software.intel.com/en-us/articles/opencl-drivers

[Khanattila]

I tried to install 2 OpenCL vendors, one for Nvidia GPU and Intel only for CPU

http://sl1pkn07.wtf/paste/view/696b9533

when set 'device_type="CPU"', the "decoding" is slowwwwwwwww like ass ("spected", in dual Xeon X5650 (24Threads))

but when set 'device_type="GPU"', the VSpipe get segfault

[967221.403446] vspipe[8062]: segfault at 7f949359cb88 ip 00007f949359cb88 sp 00007ffd9162c448 error 15 in libc-2.21.so[7f949359c000+2000]

but when no set 'device_type=', knlmeans use the GPU.

how to debug this?

knlmeans is builded with

CXXFLAGS = -O2 -std=c++11 -Wall -Wno-unused-local-typedefs -I. -I./include -march=native -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/vapoursynth
LDFLAGS = -shared -fPIC -Wl,-O1,--sort-common,--as-needed,-z,relro

in linux with
gcc 5.1.0
ocl-icd 2.2.7 (OpenCL loader)
nvidia 352.21 beta drivers
Cuda 7.0.28
Intel OpenCL for CPU 15.1 (5.0.0.57)

1) Floating Point Performance: 57 Gflops (Xeon X5650) vs 2258 (GTX 760) :)
2) VSpipe get segfault... Maybe it's a problem in the code, I need to check.

[Khanattila]

Hi

EDIT3: I've open a git repo with the code and a small configure/makefile. with the patches applied and some more things

https://github.com/sl1pkn07/KNLmeansCL

greetings

EDIT2. Taken. https://github.com/Khanattila/KNLMeansCL

[GMJCZP]

1) Sure. You can use it without a graphics card.
2) You should install the Intel driver: https://software.intel.com/en-us/articles/opencl-drivers

Those drivers are for W7 SP1 and later, why did I mention all my PC settings and ATI package, which is supposedly the only one compatible with XP and Intel.

There will be a solution?

[Bloax]

You might want to just use the original TNLMeans instead of trying to make your CPU operate through OpenCL so that you can run an OpenCL remake of TNLMeans.

[Groucho2004]

You might want to just use the original TNLMeans instead of trying to make your CPU operate through OpenCL
Indeed. On a C2D 4400 alone, KNLMeansCL and TNLMeans would probably perform about the same. TNLMeans may even have the edge.
However, all this is rather pointless because NLMeans on this CPU will be too slow for any practical use.

[Khanattila]

Those drivers are for W7 SP1 and later, why did I mention all my PC settings and ATI package, which is supposedly the only one compatible with XP and Intel.

There will be a solution?

As suggested, you should try using the CPU. It's slow, but it should work.

[GMJCZP]

The problems that should be happening is that the ATI package is incompatible with the plugin or too old, that is why the KNLMeans does not work for me (CPU or another mode). Thanks anyway.

[feisty2]

any news about "reference clip" feature?
:)

[Khanattila]

any news about "reference clip" feature?
:)

It's almost ready.

[sl1pkn07]

please send me your github user

oh, you forked it. then i can remove from my repository?