KNLMeansCL: OpenCL NLMeans de-noising algorithm [2018-01-29]

[ReinerSchweinlin]

"Invalid Thread specified....."

[ReinerSchweinlin]

thank you :)

[tormento]

@pinterf

Have a look here (https://forum.doom9.org/showthread.php?p=2004419#post2004419).

Thank you.

[anton_foy]

Btw there is a faster drop in replacement for KNLMeansCL but written for cuda, only for vapoursynth https://github.com/AmusementClub/vs-nlm-cuda

Any possibility to rewrite this for Avs+?

[ChaosKing]

You have to ask the author of the plugin :-)

[kedautinh12]

You have to ask the author of the plugin :-)

WolframRhodium said "no more avisynth ported" :D

[anton_foy]

WolframRhodium said "no more avisynth ported" :D

Too bad, it is really fast and has so much better temporal stability compared to knlmeansCL.

[tormento]

Too bad, it is really fast and has so much better temporal stability compared to knlmeansCL.


You tried newer DGDenoise?

[anton_foy]

You tried newer DGDenoise?

Yes and I get smearing/ghosting-like artifacts from the temporal setting even on low values.

[tormento]

Yes and I get smearing/ghosting-like artifacts from the temporal setting even on low values.


Tried to disable chroma? Anyway report the issue to the writer of the plugin.

[WolframRhodium]

Too bad, it is really fast and has so much better temporal stability compared to knlmeansCL.

The output of nlm-cuda should be closed to KNLMeansCL and should not result in an increase in image quality.

[anton_foy]

The output of nlm-cuda should be closed to KNLMeansCL and should not result in an increase in image quality.

My mistake, yes they seem to give identical results yet only the first 4 frames seems to differ from eachother even when both using the same settings. But that doesn't bother me. Great plugin :thanks:

@Tormento
Is the author here on Doom9 still? I cannot access his own forum.

[tormento]

Is the author here on Doom9 still? I cannot access his own forum.
I think he is lurking but not officially here.

You can’t access or you can’t register?

In the second case, perhaps some users here and on his forum can ask him to create you an account.

He is very capable but also a bit weird.

[JamesJohnston]

I reported this issue on GitHub, but I'm not sure if anyone's monitoring that, so posting it here as well. More details in the GitHub issue: https://github.com/pinterf/KNLMeansCL/issues/6

In short, if the image width is a multiple of 64, KNLMeansCL outputs garbage - as seen in this test case:


plugin_dir = "C:\VideoProject\Software\AviSynth-plugin\"
LoadPlugin(plugin_dir + "KNLMeansCL.dll")

function test(int width) {
input = ColorBars(width=width, height=480, pixel_type="YV24").Trim(0, 50).Subtitle(String(width), size=36).FadeOut(50).ConvertToY8()
output = input.KNLMeansCL(ocl_x=8, ocl_y=16, ocl_r=1)
StackVertical(input, output)
}

StackHorizontal( \
test(8*1), test(8*2), test(8*3), test(8*4), test(8*5), test(8*6), test(8*7), test(8*8), \
test(8*9), test(8*10), test(8*11), test(8*12), test(8*13), test(8*14), test(8*15), test(8*16), \
test(8*17), test(8*18), test(8*19), test(8*20), test(8*21), test(8*22), test(8*23), test(8*24), \
test(8*25), test(8*26), test(8*27), test(8*28), test(8*29), test(8*30), test(8*31), test(8*32), \
test(8*33), test(8*34), test(8*35), test(8*36), test(8*37), test(8*38), test(8*39), test(8*40), \
test(8*41), test(8*42), test(8*43), test(8*44), test(8*45), test(8*46), test(8*47), test(8*48) \
)


I posted an example of an output frame in the GitHub issue. But what happens is the bottom row (filter output) is missing outputs (they are all black) whenever width is a multiple of 64.

In real-world videos, I've seen where the filter output gets stuck at a specific frame, and doesn't change when I move the AvsPmod slider bar to other frames. Other times, it will show partial or total garbage pixels, as if it was displaying uninitialized memory.

In practice, I'm trying to crop a standard definition 720x480 video down by a multiple of 16, which is coming out with a width of 704. That seems like it would be a fairly common scenario...? but that's a multiple of 64, so the rendered output comes out as garbage.

[real.finder]

with intel igpu in 13th gen
ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL()
https://i.postimg.cc/02HPRr4f/Untitled.png (https://postimages.org/)
it's intel problem or KNLMeansCL?

[tormento]

it's intel problem or KNLMeansCL?
OpenCL should be GPU agnostic.

You tried with other GPUs as well?

[real.finder]

OpenCL should be GPU agnostic.

You tried with other GPUs as well?

there are problems with KNLMeansCL and amd gpu already, only nvidia seems work fine with KNLMeansCL

I think CPU is the only standard (Especially X86-X64 since in apple m1-m4 is kind of different from another arm from what I've heard, I know someone use x86 port of his program to make it work in m1-m4), so maybe it's good idea to make cpu port of KNLMeansCL (opencl in cpu is dropped in modren cpus) edit: there are https://forum.doom9.org/showthread.php?t=172966 already but not as KNLMeansCL, xNLMeans maybe got things that not in KNLMeansCL but there are also things in KNLMeansCL that not in xNLMeans, so maybe someone update xNLMeans to make it ultimate NLMeans :)

[qyot27]

with intel igpu in 13th gen
ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL()
https://i.postimg.cc/02HPRr4f/Untitled.png (https://postimages.org/)
it's intel problem or KNLMeansCL?
I can't reproduce that here with an Arc A770, Ubuntu 24.10.

[tormento]

it's good idea to make cpu port of KNLMeansCL
Or not. ;)

There are plugins that works really well under cpu, others really good on GPU. I think KNLMeansCL could be a real performance hit on a CPU.

Why don’t you simply use the CPU version of BM3D? I find it in many ways superior.

[Z2697]

CPU KNLM Choices:
1) CPU "port": https://github.com/AmusementClub/vs-nlm-ispc
2) CPU OpenCL runtime: https://www.intel.com/content/www/us/en/developer/articles/technical/intel-cpu-runtime-for-opencl-applications-with-sycl-support.html

[Jamaika]

there are problems with KNLMeansCL and amd gpu already, only nvidia seems work fine with KNLMeansCL

I think CPU is the only standard (Especially X86-X64 since in apple m1-m4 is kind of different from another arm from what I've heard, I know someone use x86 port of his program to make it work in m1-m4), so maybe it's good idea to make cpu port of KNLMeansCL (opencl in cpu is dropped in modren cpus) edit: there are https://forum.doom9.org/showthread.php?t=172966 already but not as KNLMeansCL, xNLMeans maybe got things that not in KNLMeansCL but there are also things in KNLMeansCL that not in xNLMeans, so maybe someone update xNLMeans to make it ultimate NLMeans :)

KNLMeansCL is old __AVISYNTH_8_H__ and openCL 1.2.
plugins/Khanattila/NLMAvisynth.cpp: In constructor 'NLMAvisynth::NLMAvisynth(PClip, int, int, int, double, const char*, int, double, PClip, const char*, int, int, int, int, bool, bool, int, IScriptEnvironment*)':
plugins/Khanattila/NLMAvisynth.cpp:215:38: warning: converting to non-pointer type 'cl_channel_type' {aka 'unsigned int'} from NULL [-Wconversion-null]
215 | cl_channel_type channel_type_u = NULL, channel_type_p = NULL;
| ^~~~
plugins/Khanattila/NLMAvisynth.cpp:215:61: warning: converting to non-pointer type 'cl_channel_type' {aka 'unsigned int'} from NULL [-Wconversion-null]
215 | cl_channel_type channel_type_u = NULL, channel_type_p = NULL;
| ^~~~
plugins/Khanattila/NLMAvisynth.cpp:216:42: warning: converting to non-pointer type 'cl_channel_type' {aka 'unsigned int'} from NULL [-Wconversion-null]
216 | cl_channel_type channel_type_p_out = NULL;
| ^~~~

ffplay_avx2.exe KNLMeansCL.avs =
ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL()

https://forum.doom9.org/showthread.php?p=2010721#post2010721

[real.finder]

I can't reproduce that here with an Arc A770, Ubuntu 24.10.

maybe windows driver bug? like this https://forum.doom9.org/showthread.php?p=1963373#post1963373

Or not. ;)

There are plugins that works really well under cpu, others really good on GPU. I think KNLMeansCL could be a real performance hit on a CPU.

Why don’t you simply use the CPU version of BM3D? I find it in many ways superior.

the cpu plugins of NLMeans in avs can't be used as drop-in replacement for KNLMeansCL
BM3D is another story

CPU KNLM Choices:
1) CPU "port": https://github.com/AmusementClub/vs-nlm-ispc
2) CPU OpenCL runtime: https://www.intel.com/content/www/us/en/developer/articles/technical/intel-cpu-runtime-for-opencl-applications-with-sycl-support.html
that for vs only?
anyway thanks for CPU OpenCL runtime link, I thought Intel no longer supported that

KNLMeansCL is old __AVISYNTH_8_H__ and openCL 1.2.
plugins/Khanattila/NLMAvisynth.cpp: In constructor 'NLMAvisynth::NLMAvisynth(PClip, int, int, int, double, const char*, int, double, PClip, const char*, int, int, int, int, bool, bool, int, IScriptEnvironment*)':
plugins/Khanattila/NLMAvisynth.cpp:215:38: warning: converting to non-pointer type 'cl_channel_type' {aka 'unsigned int'} from NULL [-Wconversion-null]
215 | cl_channel_type channel_type_u = NULL, channel_type_p = NULL;
| ^~~~
plugins/Khanattila/NLMAvisynth.cpp:215:61: warning: converting to non-pointer type 'cl_channel_type' {aka 'unsigned int'} from NULL [-Wconversion-null]
215 | cl_channel_type channel_type_u = NULL, channel_type_p = NULL;
| ^~~~
plugins/Khanattila/NLMAvisynth.cpp:216:42: warning: converting to non-pointer type 'cl_channel_type' {aka 'unsigned int'} from NULL [-Wconversion-null]
216 | cl_channel_type channel_type_p_out = NULL;
| ^~~~

ffplay_avx2.exe KNLMeansCL.avs =
ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL()

don't know, but avs should be backward compatible unless there are bugs in KNLMeansCL, anyway it still work with nvidia for me (in my home pc, the igpu bug using remote server)

[Jamaika]

You tried my version. Is it the same or not?

[real.finder]

You tried my version. Is it the same or not?

where? your link is for ffmpeg

[Jamaika]

I added it above. Yes, you run it via ffplay_avx2
https://www.sendspace.com/file/mou7uc

[real.finder]

I added it above. Yes, you run it via ffplay_avx2
https://www.sendspace.com/file/mou7uc

no need to do it in this case

anyway, it worked with
ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL(device_type="cpu")
after install w_opencl_runtime_p_2024.2.0.980.exe so it's gpu problem (maybe the driver)

[Jamaika]

I will just add that I have installed latest opencl.
https://github.com/KhronosGroup/OpenCL-Headers/commit/4ea6df132107e3b4b9407f903204b5522fdffcd6

[real.finder]

I will just add that I have installed latest opencl.
https://github.com/KhronosGroup/OpenCL-Headers/commit/4ea6df132107e3b4b9407f903204b5522fdffcd6

it's same as https://i.postimg.cc/02HPRr4f/Untitled.png

[pinterf]

with intel igpu in 13th gen
ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL()
it's intel problem or KNLMeansCL?
Reproduced.

device_id = 0 (Nvidia driver)
no problems, this setting is using NVidia 1660 on my machine.

device_id = 1: (Intel driver)
artifacts.
This id is using my Intel UHD 750 driver. It looks like if there were internal format or endian-ness problems (e.g. how a 32 bit data is ordered YUVX or XVUY)

Hint: with info=true you can display the actual driver config.

ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL(channels="auto", device_id=1, info=true)


My processor is 11th Gen Intel(R) Core(TM) i7-11700, but obviously, there must be a hidden driver setting which has to be checked to figure out that a pixel format which is good for one driver why is not good for the other.

[Jamaika]

Hint: with info=true you can display the actual driver config.

ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL(channels="auto", device_id=1, info=true)


My processor is 11th Gen Intel(R) Core(TM) i7-11700, but obviously, there must be a hidden driver setting which has to be checked to figure out that a pixel format which is good for one driver why is not good for the other.

ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL(channels="auto", device_id=1, info=true)
[avisynth @ 0000023e209d9f60] KNLMeansCL: no compatible opencl platforms available!
(AudioBoost.avs, line 2)
AudioBoost.avs: Unknown error occurred

ColorBars(width=640, height=480, pixel_type="yv12")
KNLMeansCL(channels="auto", device_id=0, info=true)
KNLMeansCL
Version 1.1.1e
Copyright(C) Khanattila
Bits per sample: 8
Search window: 5x5x3
Similarity neightborhood: 9x9
Num of ref pixels: 74
Local work size: 16x16 - 16x8x3
Platform info
Name: NVIDIA CUDA
Version: OpenCL 3.0 CUDA 12.6.65
Device info
Name: NVIDIA Geforce RTX 3050
Vendor: NVIDIA Corporation
Version: OpenCL 3.0 CUDA 560.94

It works.

[real.finder]

Hint: with info=true you can display the actual driver config.


https://i.imgur.com/1qIbUY7.png

[pinterf]

Same for me, except that I have 750. And the driver is a bit older, 32.0.101.6078.
Anyway, I narrowed down the problem, which is in one or more of the opencl program kernel functions which do the actual horizontal and vertical processing. Seems that next week(s) I'm going to entertain myself with the millions of trial and error modifications.
You have to wait a bit further, I will only appear when I'm ready.

[real.finder]

Same for me, except that I have 750. And the driver is a bit older, 32.0.101.6078.
Anyway, I narrowed down the problem, which is in one or more of the opencl program kernel functions which do the actual horizontal and vertical processing. Seems that next week(s) I'm going to entertain myself with the millions of trial and error modifications.
You have to wait a bit further, I will only appear when I'm ready.

no news since months, forgotten? :)

[pinterf]

no news since months, forgotten? :)
It's just not finalized. Most possibly a driver error, on the 4th crazy day and after the fifth possible workaround, I got it. I have almost finished and assembled a meaningful bug report (the smallest possible code which still presents the bug), then the holidays came then a huge pile of Avisynth and other plugin related stuffs. And I had to arrange and nicely commit the modifications I've done to KNL since 2020. So please wait a little bit more I hope not that much.

[tormento]

I find BM3D to be superior to KNLMeansCL almost in every aspect.

Is there any niche application where it could shine?

[pinterf]

Bug reported to Intel. At least I consider it a bug:

https://community.intel.com/t5/GPU-Compute-Software/Intel-OpenCL-on-UHD750-770-artifacts-at-specific-block-sizes/m-p/1673206#M1764
EDIT: Intel (or Intel's bot) almost instantly categorized my bug report as spam, causing it to disappear from the flow of posts. I mentioned this to the moderator... We'll see what happens. It's not easy to communicate with them.

There may be a possible workaround: manually specifying parameters ocl_x and ocl_y to 16.
Report back if it works.

[tormento]

Bug reported to Intel. At least I consider it a bug
The link you posted brings to:

The message you are trying to access is not available.

[real.finder]

Bug reported to Intel. At least I consider it a bug:

https://community.intel.com/t5/GPU-Compute-Software/Intel-OpenCL-on-UHD750-770-artifacts-at-specific-block-sizes/m-p/1673206#M1764
EDIT: Intel (or Intel's bot) almost instantly categorized my bug report as spam, causing it to disappear from the flow of posts. I mentioned this to the moderator... We'll see what happens. It's not easy to communicate with them.

There may be a possible workaround: manually specifying parameters ocl_x and ocl_y to 16.
Report back if it works.

KNLMeansCL(info=true, ocl_x=16, ocl_y = 16, ocl_r=16)
seems work, dont know if it act like KNLMeansCL(info=true) with other non bugged gpu drivers

[pinterf]

KNLMeansCL(info=true, ocl_x=16, ocl_y = 16, ocl_r=16)
seems work, dont know if it act like KNLMeansCL(info=true) with other non bugged gpu drivers
Probably, the 32x8 configuration was chosen for performance reasons on Intel GPUs (I'm not sure if this was based on tests or manufacturer recommendations). The 16x16 configuration should work as well. Anyway, a lot of time has passed since these values were established in the code, and the optimal setting may have changed since then.

[Katie Boundary]

Has anyone compiled a version of this that doesn't rely on external DLL bullshit? Visual C++ 2013 isn't installing on one of my computers for some stupid bullshit reason or another and I think that's screwing up a bunch of my filters.

UPDATE: I was able to find slightly older versions (12.0.30501) of Visual C++ 2013, and they installed correctly, but KNLmeansCL and two other filters are still whining about a missing DLL file.

Dynamic linking is one of the worst ideas in computing history.

[Jamaika]

I have a statically linked library using gcc/mingw.
https://github.com/FFmpeg/FFmpeg/commit/afe6c1238ac4119ec9c9dedc42220c4595f6a33c
https://github.com/AviSynth/AviSynthPlus/commit/2e34f5daba7f223e8172e28ed7900b0997749207
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/b43f75fab4a6f7330e24855c9f7c9c971cd626d9
https://github.com/Khanattila/KNLMeansCL/commit/ca424fa91d1e16ec011f7db9c3ba0d1e76ed7850
https://github.com/KhronosGroup/OpenCL-ICD-Loader/commit/86448ce4e47bc59688c58e7542ebfa4d7568c0de
https://github.com/KhronosGroup/OpenCL-Headers/commit/bb1dc2e5882378817831ae17ba7391b3e23efa81

LibavSource2("input.avi",fpsnum=30000,fpsden=1001)
KNLMeansCL(device_type="gpu",device_id=0,info=true)

https://www.sendspace.com/file/k2xj7v

Edit:
I will only add that all add-ons should be for ffms2 and not ffmpeg. If we want to use ffmpeg, the library should be modified, e.g. for l-smash. There aren't modifications for the latest avisynth.c 3.7.4.
https://github.com/HomeOfAviSynthPlusEvolution/FFmpeg/commits?author=qyot27

[Katie Boundary]

I have a statically linked library using gcc/mingw.
https://github.com/FFmpeg/FFmpeg/commit/afe6c1238ac4119ec9c9dedc42220c4595f6a33c
https://github.com/AviSynth/AviSynthPlus/commit/2e34f5daba7f223e8172e28ed7900b0997749207
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/b43f75fab4a6f7330e24855c9f7c9c971cd626d9
https://github.com/Khanattila/KNLMeansCL/commit/ca424fa91d1e16ec011f7db9c3ba0d1e76ed7850
https://github.com/KhronosGroup/OpenCL-ICD-Loader/commit/86448ce4e47bc59688c58e7542ebfa4d7568c0de
https://github.com/KhronosGroup/OpenCL-Headers/commit/bb1dc2e5882378817831ae17ba7391b3e23efa81

LibavSource2("input.avi",fpsnum=30000,fpsden=1001)
KNLMeansCL(device_type="gpu",device_id=0,info=true)

https://www.sendspace.com/file/k2xj7v

Okay... which of those 20 or 30 links is it?

EDIT: after a little bit of digging, it turns out that version 0.6.11 was compiled with Visual C++ 2013 instead of 2015, and therefore does not require UCRT. It was kept around for compatibility with older versions of OpenCL, but XP compatibility was an accidental side effect. I can use this for now.

[Khanattila]

Bug reported to Intel. At least I consider it a bug:

https://community.intel.com/t5/GPU-Compute-Software/Intel-OpenCL-on-UHD750-770-artifacts-at-specific-block-sizes/m-p/1673206#M1764
EDIT: Intel (or Intel's bot) almost instantly categorized my bug report as spam, causing it to disappear from the flow of posts. I mentioned this to the moderator... We'll see what happens. It's not easy to communicate with them.

There may be a possible workaround: manually specifying parameters ocl_x and ocl_y to 16.
Report back if it works.

Great job with the bug report.

Probably, the 32x8 configuration was chosen for performance reasons on Intel GPUs (I'm not sure if this was based on tests or manufacturer recommendations). The 16x16 configuration should work as well. Anyway, a lot of time has passed since these values were established in the code, and the optimal setting may have changed since then.

Testing, and a general understanding of how the Intel integrated GPU architecture was designed (memory alignment, cache size, compute units per group). Those values obviously depend on the architecture and are not optimal forever.

[GeoffreyA]

Not related, but I recently discovered a slight performance loss with KNLMeansCL on the Arc B580's recent drivers versus one from January.

https://github.com/IGCIT/Intel-GPU-Community-Issue-Tracker-IGCIT/issues/1134

[takla]

FFMPEG has merged a rewrite of nlmeans vulkan (https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20689), which offers increased processing speed.

Newest FFMPEG download (https://github.com/BtbN/FFmpeg-Builds/releases)

Usage example:

ffmpeg -y -benchmark -init_hw_device vulkan -i INPUT.mkv -vf "format=nv12,hwupload,nlmeans_vulkan,hwdownload,format=nv12" -c:v libx264 OUTPUT.mkv


On my system (Ryzen 7600X, RTX 5070 Ti) I get 39.2 FPS with 1920x1080 BluRay footage.
Edit: And 44.4 FPS with +300/+3000 core/memory overclock on the GPU.

[GeoffreyA]

FFMPEG has merged a rewrite of nlmeans vulkan (https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20689), which offers increased processing speed.

Newest FFMPEG download (https://github.com/BtbN/FFmpeg-Builds/releases)

Usage example:

ffmpeg -y -benchmark -init_hw_device vulkan -i INPUT.mkv -vf "format=nv12,hwupload,nlmeans_vulkan,hwdownload,format=nv12" -c:v libx264 OUTPUT.mkv


On my system (Ryzen 7600X, RTX 5070 Ti) I get 39.2 FPS with 1920x1080 BluRay footage.
Edit: And 44.4 FPS with +300/+3000 core/memory overclock on the GPU.

On the 4600G and B580, I get 12 fps on an old build, and 14 fps on the new one. Notably, though, is that the pervasive artefacts are gone.

[tormento]

Now we just need to wait (and hope) that someone will port that to AVS+.

[takla]

On the 4600G and B580, I get 12 fps on an old build, and 14 fps on the new one. Notably, though, is that the pervasive artefacts are gone.

Thats a 16.6% speedup, nice.