KNLMeansCL: OpenCL NLMeans de-noising algorithm [2018-01-29]

[real.finder]

bug when using rclip and prefetch at least with https://github.com/pinterf/KNLMeansCL (didn't test the others since they not work with last avs+ now)

ColorBars(width=640, height=480, pixel_type="yv24").Trim(0,1000).Subtitle("Testing",align=5,size=200)
Animate(0, 500, "Crop",
\ 0, 0, 300,400,
\ 316, 0, 300,400)
KNLMeansCL(rclip=Blur(1))
prefetch(8)

using avspmod and seek to frame 120 (or any other) and refresh with f5 you will noted that image is blended with another frame and it same or worst when you go to the next frames (121, 122, ...) until you reach clean frames after some frames

the problem is there even without rclip, but it's worst with rclip

and it also showed with d=0

using SetFilterMTMode("KNLMeansCL",3,true) or SetFilterMTMode("KNLMeansCL",2,true) will fix it

edit: with real encoding, even SetFilterMTMode("KNLMeansCL",3,true) is not fully fix it, it only make it showed in less frames

edit: I tried with another device with only opencl on cpu, and it work fine! the one above was using intel gpu
edit: seems it also showed in opencl on cpu device, but not in the frame that I refreshed but in next 2 or 3

[real.finder]

also I note that KNLMeansCL (at least with mcKNLMeansCL) in very complex scripts even without prefetch case random deadlock but the cpu usage will stay high in x264/avspmod with no progress

[pinterf]

bug when using rclip and prefetch at least with https://github.com/pinterf/KNLMeansCL (didn't test the others since they not work with last avs+ now)

ColorBars(width=640, height=480, pixel_type="yv24").Trim(0,1000).Subtitle("Testing",align=5,size=200)
Animate(0, 500, "Crop",
\ 0, 0, 300,400,
\ 316, 0, 300,400)
KNLMeansCL(rclip=Blur(1))
prefetch(8)

using avspmod and seek to frame 120 (or any other) and refresh with f5 you will noted that image is blended with another frame and it same or worst when you go to the next frames (121, 122, ...) until you reach clean frames after some frames

Bug fixed. For high bit depth (9-15 bits) support and other changes see change log. (mods done and tested for both VS and Avs)
https://github.com/pinterf/KNLMeansCL/releases/tag/v1.1.1e

[real.finder]

Bug fixed. For high bit depth (9-15 bits) support and other changes see change log. (mods done and tested for both VS and Avs)
https://github.com/pinterf/KNLMeansCL/releases/tag/v1.1.1e

thanks! everything seems work now

[Khanattila]

Bug fixed. For high bit depth (9-15 bits) support and other changes see change log. (mods done and tested for both VS and Avs)
https://github.com/pinterf/KNLMeansCL/releases/tag/v1.1.1e

Arg! You have succumbed to the dark side of the force!
I'm joking, of course.

EDIT. Seriously, I have never added support for 9-15 bits formats because I think it is a waste of memory.
In the best of scenarios, they are as fast as the 16-bit format... so why bother?

[kedautinh12]

You still developer KNLMeansCL or abandoned, Khanattila??

[real.finder]

Arg! You have succumbed to the dark side of the force!
I'm joking, of course.

EDIT. Seriously, I have never added support for 9-15 bits formats because I think it is a waste of memory.
In the best of scenarios, they are as fast as the 16-bit format... so why bother?

I think it's better to support them and leave the choice to the user, also here https://forum.doom9.org/showthread.php?p=1906009#post1906009

You still developer KNLMeansCL or abandoned, Khanattila??

I have same question and also https://forum.doom9.org/showthread.php?t=182126

[feisty2]

users don't know shit, the internal processing precision has nothing to do with either the bitdepth of the source file, or the bitdepth of the encoded file, or the display bitdepth.

[feisty2]

Even first going up to int16 or directly to fp32, then simply applying truncation to convert the bitdepth to something like int10 before encoding would be better than an int10 processing line. clueless users should be banned from touching filters before they understand how error propagation works.

[Khanattila]

I think it's better to support them and leave the choice to the user, also here https://forum.doom9.org/showthread.php?p=1906009#post1906009

I wrote it to be sharp and transparent and then use AvS or VS to smooth the corners.
Anyway, you are probably right; I wrote the program 6 years ago, while I was studying C and OpenCL, and anyone can notice it by style in which I wrote it.

I have same question and also https://forum.doom9.org/showthread.php?t=182126

To be honest, I didn't think anyone cared. I wrote it myself just to test one thing. It is a project that I could think of resuming if there is interest.

To talk about KNLMeansCL ... I don't think it's worth it. Let me be clear, I am glad that there is someone who takes care of it but OpenCL support is a complete disaster, and will not improve. If today I work on it actively, I will try to do it again in C ++ and Vulkan.

"Tensorflow on open source GPUs" - David Airlie (LCA 2020) (https://youtu.be/KfDQb6xOkXg?t=740)

[ChaosKing]

It's time for KNLMeansVK :cool:

[real.finder]

To be honest, I didn't think anyone cared. I wrote it myself just to test one thing. It is a project that I could think of resuming if there is interest.


I think there are many care, there were https://forum.doom9.org/showthread.php?p=1806299&highlight=KPassFilterCL#post1806299 even if KPassFilterCL was not announced back then and seems there are care in posts here https://forum.doom9.org/showthread.php?t=182126 too

[real.finder]

To talk about KNLMeansCL ... I don't think it's worth it. Let me be clear, I am glad that there is someone who takes care of it but OpenCL support is a complete disaster, and will not improve. If today I work on it actively, I will try to do it again in C ++ and Vulkan.

"Tensorflow on open source GPUs" - David Airlie (LCA 2020) (https://youtu.be/KfDQb6xOkXg?t=740)

well, even if Vulkan seems better, OpenCL still has Features on Vulkan like it can run on CPUs and old GPUs

KNLMeansCL still has problems that didn't fixed yet even by pinterf update, like dark image https://forum.doom9.org/showthread.php?p=1927287#post1927287 and this VS problem https://github.com/Khanattila/KNLMeansCL/issues/43

[ChaosKing]

AMD basically dropped OpenCL support anyway. GPU driver support is not that good on windows. Opencl in Ryzen is not supported, on windows. Why focus on old stuff if Vulkan is the future...

[Reel.Deel]

Khanattila nice to see you back around!

If possible can you please share the source code to KPassFilterCL?

[Khanattila]

Khanattila nice to see you back around!

If possible can you please share the source code to KPassFilterCL?
Fun fact, I don't think I have KPassFilterCL anymore.
Furthermore, PassFilter was something else than KPassFilterCL.

EDIT. It's your lucky day

[StainlessS]

Fun fact, I don't think I have KPassFilterCL anymore.
This is ? :- https://github.com/Khanattila/KPassFilterCL/tree/master/KPassFilterCL

EDIT: I Got from here:- https://sites.google.com/site/khanattilasfilters/

OOps, you already just posted that link in the other thread.

[Khanattila]

This is ? :- https://github.com/Khanattila/KPassFilterCL/tree/master/KPassFilterCL

EDIT: I Got from here:- https://sites.google.com/site/khanattilasfilters/

OOps, you already just posted that link in the other thread.

I was really sure I didn't have it anymore :D

[ChaosKing]

Bug fixed. For high bit depth (9-15 bits) support and other changes see change log. (mods done and tested for both VS and Avs)
https://github.com/pinterf/KNLMeansCL/releases/tag/v1.1.1e

In Vapoursynth I get this
https://i.imgur.com/OAPJweD.png
sometimes like this
https://i.imgur.com/HekyxaH.png

clip = core.lsmas.LWLibavSource(source=r"f:\VTS_01_CHAPTER_03_1.VOB")# DVD NTSC 720x480
clip=mvf.Depth(clip, 16)
clip = core.knlm.KNLMeansCL(clip, d=2, a=3, h=1)

Only 16bit seems to be affected
Vapoursynth R52, Win10 64

[pinterf]

In Vapoursynth I get this
https://i.imgur.com/OAPJweD.png
sometimes like this
https://i.imgur.com/HekyxaH.png

clip = core.lsmas.LWLibavSource(source=r"f:\VTS_01_CHAPTER_03_1.VOB")# DVD NTSC 720x480
clip=mvf.Depth(clip, 16)
clip = core.knlm.KNLMeansCL(clip, d=2, a=3, h=1)

Only 16bit seems to be affected
Vapoursynth R52, Win10 64
Right now I have an R48 and was not able to reproduce. Tried YUV420, 422, 444, 8-9-10-12-14-16 bits, channels="YUV" and without it.
EDIT: same with R52: I'm seeing no artifacts.

Does it work with the original one?

[ChaosKing]

The original has the same problem :D

I switched recently to a AMD Vega64 card. On my previous GTX 1070 it was fine, at least the original version.

RGB24-48 is fine
YUV8-15 is fine, 16 bit = FAIL

What info should/can I provide in order to help fixing it?


EDIT
Avisynth 3.7 64bit looks ok

convertbits(16)
KNLMeansCL(d=2, a=3, h=1)

[real.finder]

AMD GPU then maybe this has to do with it https://forum.doom9.org/showthread.php?p=1927287#post1927287

but since it work in avs, maybe this (https://github.com/Khanattila/KNLMeansCL/issues/43)?

[Selur]

using:

clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, range_s="limited")
clip = core.knlm.KNLMeansCL(clip=clip, d=2, a=3, s=1, h=1.00)

with YUV444P16, YUV422P16, YUV420P16 works fine here. (using a Geforce GTX 1070ti)
could be an issue of 'clip=mvf.Depth(clip, 16)' or with the amd card,..

Cu Selur

[pinterf]

Could you please try this test build:
https://drive.google.com/uc?export=download&id=1xR5RPUUwJbF8ZNbY1P_6p3VxSq3S8ESU

Theoretically the same thing could happen in VapourSynth as in Avisynth did (and was fixed there): frame buffer would be freed up before the actual content is copied for OpenCL.

[ChaosKing]

Same broken output.
Usually the first preview is very dark, I see red and some blue colors. Then I try it with Depth(10) and it looks ok. Back to 16 again and it looks like the second image I posted here https://forum.doom9.org/showthread.php?p=1935108#post1935108

https://i.imgur.com/tA86OhO.png
left broken, right original

I tried it in vseditor and Vdub2, same result.

[pinterf]

Check if only plane 0 (Luma) is affected or chroma as well?
Default is luma-only working mode so chroma planes should be copied as-is.
EDIT:
further tests:
- OK? 10 bit input, channels "YUV"
- All is OK? 10 bit input, channels "YUV", mode_9_to_15bits parameter 0 and 1 and 2
- screenshot with info=1 parameter (unfortunately this works only for 8 bit modes)

[ChaosKing]

info:
https://i.imgur.com/0TL5lS0.png

knlm.KNLMeansCL(clip, channels="Y") # 8-15 bit OK - 16 broken (14 seems not to be supported?)
clip = clip.fmtc.resample(css="444")
clip=mvf.Depth(clip, 16)
clip = core.knlm.KNLMeansCL(clip, channels="YUV") 1-16 OK !

mode_9_to_15bits = 0,1,2 is only broken with 16bits Y channel, 1-15 are ok. YUV 16bits is also ok

=> So 16bit with only Y channel seems to be affected

I tested with your the latest test-version.

[pinterf]

Nothing special, just 'info' works for any formats (except 16-bit float "half"):
https://drive.google.com/uc?export=download&id=1hfd6K6Q4hOEqlDMwPrte6maqPs3DxElU

[Khanattila]

AMD GPU then maybe this has to do with it https://forum.doom9.org/showthread.php?p=1927287#post1927287

but since it work in avs, maybe this (https://github.com/Khanattila/KNLMeansCL/issues/43)?

Fun fact, Vega's availability has always been so scarce that I bought a Navi directly.

[ChaosKing]

Fun fact, Vega's availability has always been so scarce that I bought a Navi directly.

Crypto mining is back! I sold my Vega for a good price. GPU prices are insane right now...

I'm on a cheap GTX 660 now and 16 bit output looks ok here.

[FranceBB]

I'm on a cheap GTX 660 now and 16 bit output looks ok here.

For what it's worth, 4:2:2 16bit planar looks fine on Avisynth x64 3.7 on my NVIDIA Quadro P4000 as well.

[Soichiro]

I hope this doesn't count as a necro, but since I can't post issues on the Github fork, here seems like the best place.

I'm having the same issue you are. I narrowed down the triggers quite a bit.

It only happens on my Windows PC, which is running an AMD Radeon 6700 XT (latest drivers). My two Linux PCs, one with a Geforce 2060 and one with a 2070M, both produce correct outputs in all tested cases. (Unfortunately, I haven't narrowed it down to whether Windows is the trigger or AMD is the trigger.)

It only occurs on 8-bit or 16-bit inputs where the width of the video in bytes is a multiple of 32 (i.e. width in pixels is mod-32 for 8-bit or mod-16 for 16-bit).

I've tested both using a precompiled binary from the internet, as well as one compiled myself using the meson script in the repo, on clang 13, x86_64-pc-windows-msvc. This is all using pinterf's fork.

What seems to be happening is that every pixel in the luma plane is getting set to some very low value (1?) and it is causing the image to appear extremely dark like this.

info:
https://i.imgur.com/0TL5lS0.png

knlm.KNLMeansCL(clip, channels="Y") # 8-15 bit OK - 16 broken (14 seems not to be supported?)
clip = clip.fmtc.resample(css="444")
clip=mvf.Depth(clip, 16)
clip = core.knlm.KNLMeansCL(clip, channels="YUV") 1-16 OK !

mode_9_to_15bits = 0,1,2 is only broken with 16bits Y channel, 1-15 are ok. YUV 16bits is also ok

=> So 16bit with only Y channel seems to be affected

I tested with your the latest test-version.

[takla]

Why not use MVtools degrain or BM3D? They are a lot faster and can look just as good if not better.

[Soichiro]

BM3D is not faster than KNLMeansCL. Not even close. Unless you don't have a GPU.

Also that's the least helpful response ever, it's completely irrelevant to the issue I mentioned.

[kedautinh12]

I seen BM3DCUDA faster than KNLMEANSCL

[Soichiro]

In my case, it's not. On my 2060, BM3DCuda is (surprisingly) even slower than CPU BM3D. Which is, on the fastest preset, half the speed of KNLMeansCL.

And it's irrelevant anyway. All I came here to do is research and report more details on a bug. Not be dismissed for using KNLMeansCL, in the thread dedicated to KNLMeansCL.

[tormento]

In my case, it's not. On my 2060, BM3DCuda is (surprisingly) even slower than CPU BM3D. Which is, on the fastest preset, half the speed of KNLMeansCL.
Put the 2060 on the Windows PC and give DGDenoise a try.

It's as fast as being almost transparent and works really well with anime contents.

[takla]

In my case, it's not. On my 2060, BM3DCuda is (surprisingly) even slower than CPU BM3D. Which is, on the fastest preset, half the speed of KNLMeansCL.

And it's irrelevant anyway. All I came here to do is research and report more details on a bug. Not be dismissed for using KNLMeansCL, in the thread dedicated to KNLMeansCL.

Well in my testings KNLMeansCL was insanely slow (0.1fps) so I just moved on. I'm very surprised BM3D Cuda is slower then BM3D cpu...
What CPU do you have?
Also, why shouldn't I recommend denoisers which are better?

[Dogway]

Put the 2060 on the Windows PC and give DGDenoise a try.

Wow, so much better, I'm going to fully replace ex_KNLmeansCL with that.

[tormento]

Wow, so much better, I'm going to fully replace ex_KNLmeansCL with that.
Much better to place side by side with other prefilter modes than to fully replace, just to see how it performs on speed and noise reduction, also because it's CUDA only.

[Dogway]

Even these past days I was already pondering on removing KNLMeansCL or at least make it spatial only, it's motion estimation is horrible besides anime works better with spatial only. To me KNLMeansCL now looks like poor man's BM3D (when this one gets fixed). I will keep the function though.

[Dogway]

By the way I noticed that DGDenoise() is broken when chroma is denoised : ( Looks like we can't have a fully working GPU denoiser without jumping through hoops.

[videoh]

That was fixed just days ago. Re-download DGDecNV 239 and update DGDecodeNV.dll. If you still have a problem, let me know and I'll fix it straightaway. Working fine for me and others.

[Dogway]

Thanks videoh! Need to integrate latest version but felt intimidated by having to reindex all the source clips

[videoh]

Depending on your previous version you may be able to appropriately edit the dgi file instead of re-indexing.

Maybe since DG stuff is free now, I should unbundle the CUDA filters. Then you wouldn't have this dilemma.

[anton_foy]

Even these past days I was already pondering on removing KNLMeansCL or at least make it spatial only, it's motion estimation is horrible besides anime works better with spatial only. To me KNLMeansCL now looks like poor man's BM3D (when this one gets fixed). I will keep the function though.

Yes I agree even D=6 and MCknlmeanscl does not rid my footage of wobbly lines due to the noise. Did you guys get good results with Dgdenoise? When I used it with UHD material it looks just blurry and unusable and I cannot find any good settings. I guess this will be trial and error just as it was with knlmeanscl for me to find good settings which was also just blurry until I cranked up to s=6.

[Dogway]

@anton_foy: Yes it's too blurry so it might not be good for regular denoising but for prefiltering it's perfect. Normally you don't want to denoise spatially except for exceptional (yeah) cases.

@videoh: Worked now. I think when I tried a few months back I was keeping the major version "2053.0.0.239 X64" while now it's "239.0.0.0 X64"

[real.finder]

I hope this doesn't count as a necro, but since I can't post issues on the Github fork, here seems like the best place.

I'm having the same issue you are. I narrowed down the triggers quite a bit.

It only happens on my Windows PC, which is running an AMD Radeon 6700 XT (latest drivers). My two Linux PCs, one with a Geforce 2060 and one with a 2070M, both produce correct outputs in all tested cases. (Unfortunately, I haven't narrowed it down to whether Windows is the trigger or AMD is the trigger.)

It only occurs on 8-bit or 16-bit inputs where the width of the video in bytes is a multiple of 32 (i.e. width in pixels is mod-32 for 8-bit or mod-16 for 16-bit).

I've tested both using a precompiled binary from the internet, as well as one compiled myself using the meson script in the repo, on clang 13, x86_64-pc-windows-msvc. This is all using pinterf's fork.

What seems to be happening is that every pixel in the luma plane is getting set to some very low value (1?) and it is causing the image to appear extremely dark like this.

I think someone should report it to AMD so they fix it in windows, it was fixed in Linux https://github.com/Khanattila/KNLMeansCL/issues/26#issuecomment-660256817 (note that it not only KNLMeansCL problem (https://github.com/Khanattila/KNLMeansCL/issues/26#issuecomment-609772502) since Morpheus1123 said: "I also find similar issue occurs on VapourSynth-NNEDI3CL when I set planes=[0] or [1] or [2] or [0,1]........ . Planes=[0, 1, 2] is no problem.")

Edit: https://forum.doom9.org/showthread.php?p=1873988&highlight=dark#post1873988

Yes I agree even D=6 and MCknlmeanscl does not rid my footage of wobbly lines due to the noise. Did you guys get good results with Dgdenoise? When I used it with UHD material it looks just blurry and unusable and I cannot find any good settings. I guess this will be trial and error just as it was with knlmeanscl for me to find good settings which was also just blurry until I cranked up to s=6.

@anton_foy: Yes it's too blurry so it might not be good for regular denoising but for prefiltering it's perfect. Normally you don't want to denoise spatially except for exceptional (yeah) cases.

maybe xNLMeans worth a try https://forum.doom9.org/showthread.php?t=172966 , there are also https://github.com/pinterf/TNLMeans

[kedautinh12]

I remember xNLMeans very slow

[real.finder]

I remember xNLMeans very slow

it's cpu only and no simd (c only) so that normal