KNLMeansCL: OpenCL NLMeans de-noising algorithm [2018-01-29]

[Khanattila]

I'm also looking for official document for new version...

Update: oh the next version will be documented, that's great!

I have to finish to update the wiki ;)

[WolframRhodium]

I have to finish to update the wiki ;)

That wiki helps me a lot, thank you.

Anyway, isn't the default value of "wmode" is 1(DFT_WMODE)? The wiki says by default the value is 0. And there are 3 kind of Bisquare weighting function but the wiki just mention 2.

[dipje]

Easiest is to say, that while the 1.0 branch is in 'beta' the doc and wiki will be outdated or have a chance to be incorrect. It's a beta for a reason, a quick release to find problems, not to have a complete finished and documented release :P.

Then again, there are a lot of plugins and products around here that never leave the beta phase so. mweh :).

[tormento]

A beta can't be really tested until parameters are known :)

[Khanattila]

A beta can't be really tested until parameters are known :)

Done (https://github.com/Khanattila/KNLMeansCL/wiki/Filter-description).

[tormento]

Done (https://github.com/Khanattila/KNLMeansCL/wiki/Filter-description).

Great. Thanks.

[tormento]

Maybe because "wmode" has changed? For v1.0.0-beta.1, you should set it to 0 (Welsch weighting function), which is used in v0.7.7 by default.

It seems it is the default value for both releases.

So, I dunno why output bitrate changed.

[WolframRhodium]

It seems it is the default value for both releases.

So, I dunno why output bitrate changed.

I just read the source code and perhaps I made a mistake.

The update log mentions "Improved de-noising quality around image edge." Maybe that's the reason?

Update:
I have made another test of v1.0.0-beta.1 on vapoursynth just now.
Yes, the default value of "wmode" is 1 for both version, but in v0.7.7, it indicates "Welsch weighting function", while in v1.0.0-beta.1, it indicates "Modified Bisquare weighting function", so what I have suggested you before to set it to 0 for v1.0.0-beta.1 is to return to use "Welsch weighting function".

I'm sorry for my poor English expression. I'm not a native English speaker.

[Khanattila]

I just read the source code and perhaps I made a mistake.

The update log mentions "Improved de-noising quality around image edge." Maybe that's the reason?

Update:
I have made another test of v1.0.0-beta.1 on vapoursynth just now.
Yes, the default value of "wmode" is 1 for both version, but in v0.7.7, it indicates "Welsch weighting function", while in v1.0.0-beta.1, it indicates "Modified Bisquare weighting function", so what I have suggested you before to set it to 0 for v1.0.0-beta.1 is to return to use "Welsch weighting function".

I'm sorry for my poor English expression. I'm not a native English speaker.

There were several problems in the release of v1.0.0-beta1, so before I make other changes I release beta2.

https://github.com/Khanattila/KNLMeansCL/releases/tag/v1.0.0-beta.2

[tormento]

Yes, the default value of "wmode" is 1 for both version, but in v0.7.7, it indicates "Welsch weighting function", while in v1.0.0-beta.1, it indicates "Modified Bisquare weighting function", so what I have suggested you before to set it to 0 for v1.0.0-beta.1 is to return to use "Welsch weighting function".
Nope. Welsch is the default wmode with both even with different wmode numbers, i.e. 1 for 0.7.7 and 0 for 1.0.

Different noise treatment should be somewhere else. Khanattila, can you please tell us what is under the hood? :)

[WolframRhodium]

Nope. Welsch is the default wmode with both even with different wmode numbers, i.e. 1 for 0.7.7 and 0 for 1.0.

Different noise treatment should be somewhere else. Khanattila, can you please tell us what is under the hood? :)

khanattila just pushed the last commit of v1.0.0-beta.2, which is "Change default value of wmode" (https://github.com/Khanattila/KNLMeansCL/commit/0a1af6c9b1710f9a723adf0a03573bd0e5ccedf1)

[Khanattila]

Nope. Welsch is the default wmode with both even with different wmode numbers, i.e. 1 for 0.7.7 and 0 for 1.0.

Different noise treatment should be somewhere else. Khanattila, can you please tell us what is under the hood? :)

v0.7.7 wmode=1 -> Welsch
v1.0.0 beta1 wmode=1 (my fault) -> Bisquare
v1.0.0 beta2 wmode=0 -> Welsch

[feisty2]

https://www.mathworks.com/help/stats/robustfit.html
just in case if anyone's curious about the mathematic definition of each weighting function...

[tormento]

v0.7.7 wmode=1 -> Welsch
v1.0.0 beta1 wmode=1 (my fault) -> Bisquare
v1.0.0 beta2 wmode=0 -> Welsch

+3/5% bitrate difference from 0.7.7.

I invite other members to test some samples and report here.

[WolframRhodium]

I find strange stains at the border when using v1.0.0-beta.2, especially when "a" is large.

examples:
http://screenshotcomparison.com/comparison/193750
Just look at the top border of the filtered image. There are also some stains at the left/right border, but is nearly invisible.

The problems happens on both "channels".

[tormento]

How is possible to denoise a normal DGDecNV output on both Y and UV? If I left auto, it uses Y only, if I try YUV it tells me

error: KNLMeansCL: 'channels = YUV' require a YV24 pixel format!

[Khanattila]

I find strange stains at the border when using v1.0.0-beta.2, especially when "a" is large.

examples:
http://screenshotcomparison.com/comparison/193750
Just look at the top border of the filtered image. There are also some stains at the left/right border, but is nearly not visible.

The problems happens on both "channels".

I would say that the "Improved denoising image quality around edge" must be fixed :thanks:

[Khanattila]

How is possible to denoise a normal DGDecNV output on both Y and UV? If I left auto, it uses Y only, if I try YUV it tells me

error: KNLMeansCL: 'channels = YUV' require a YV24 pixel format!

As has always been...

a) Processes each channel independently.
b) Process "Y" and "UV" separately.
c) Converting to RGB, processing, and them back to YUV.
d) Other more elaborate methods.

[feisty2]

I would say that the "Improved denoising image quality around edge" must be fixed :thanks:

do some padding before the actual process, trivial to fix

[tormento]

As has always been...

a) Processes each channel independently.
b) Process "Y" and "UV" separately.
c) Converting to RGB, processing, and them back to YUV.
d) Other more elaborate methods.

e) ask kindly the programmer to add the feature :)

[Khanattila]

do some padding before the actual process, trivial to fix
It would be a step back, and that's what I want to avoid

[real.finder]

How is possible to denoise a normal DGDecNV output on both Y and UV? If I left auto, it uses Y only, if I try YUV it tells me

error: KNLMeansCL: 'channels = YUV' require a YV24 pixel format!

you can use SMDegrain_prefilters(4) and play with it settings ;)

[tormento]

you can use SMDegrain_prefilters(4) and play with it settings ;)

I was playing with plain beta before feeding it to SMDegrain.

[real.finder]

I was playing with plain beta before feeding it to SMDegrain.

even so

SMDegrain_prefilters =\\= SMDegrain

with SMDegrain_prefilters(4) you will get only KNLMeansCL output clip, and there is knlm_params to use other KNLMeansCL params

and you can feed that clip in SMDegrain if you want

like this

knlclip=SMDegrain_prefilters(4, knlm_params=", wmode=0")
SMDegrain(prefilter=knlclip)

[WolframRhodium]

Feature request:
Extra "mask" clip option
For each pixel in the output clip, if the value of corresponding pixel in "mask" is 0 (black), just copy the value from input clip; else, use NLM normally.

Hope this will reduce computation complexity a lot in some cases, althought the can also be achieved by mt_merge(source, KNLMeansCL(source), mask) but that's too slow when there's only few pixels need to be processed

[tormento]

knlclip=SMDegrain_prefilters(4, knlm_params=", wmode=0")
SMDegrain(prefilter=knlclip)
WOW! Never noticed SMDegrain_prefilters

[feisty2]

floating point support is corrupted in v1.0 beta2

[Khanattila]

Feature request:
Extra "mask" clip option
For each pixel in the output clip, if the value of corresponding pixel in "mask" is 0 (black), just copy the value from input clip; else, use NLM normally.

Hope this will reduce computation complexity a lot in some cases, althought the can also be achieved by mt_merge(source, KNLMeansCL(source), mask) but that's too slow when there's only few pixels need to be processed

It could be interesting.

floating point support is corrupted in v1.0 beta2

What kind of problem? Loss of precision or anything else?

[feisty2]

It could be interesting.



What kind of problem? Loss of precision or anything else?

an error message bitching about "input format not supported"

[feisty2]

can you fix the floating point support soon?
wanna give that new wmode a shot.

[Khanattila]

can you fix the floating point support soon?
wanna give that new wmode a shot.

Ok, I release the beta.3 in the next days.

[dipje]

@khanattila: What about the results of that 'benchmark only' build you did?

Are you going to put 'optimal' settings inside the plugin based on detected gpu / generation. are or you going to open up parameters like those so we can find our own optimal distribution with the final release?

[Khanattila]

@khanattila: What about the results of that 'benchmark only' build you did?

Are you going to put 'optimal' settings inside the plugin based on detected gpu / generation. are or you going to open up parameters like those so we can find our own optimal distribution with the final release?

Good point, the parameters will be already pre-calibrated, but you can overwrite them.

[tormento]

Good point, the parameters will be already pre-calibrated, but you can overwrite them.

I think a --preset value, such as in x264, with 4/5 max setting value groups would be nice and noob proof, better with too many parameters based on video card.

Speed is determined by total script, not only a plugin. I could trade some speed for better prefiltering or anything else.

[dipje]

But the parameters have nothing to do with quality or speed settings for the plugin. More a way to optimize the workload so it performs better for certain video cards.

Certain AMD generations need other parameters than other AMD cards or something. My otherwise speedy GTX 1060 needed really different (low) settings to gain maximum speed than the latest RX480 cards for instance.

This has nothing to do with quality or what the plugin does. More the way the plugin gives commands to the GPU as far as I get it .

[hydra3333]

Hello. A newbie type question, however worth asking since I find I am in need of guidance.

I wonder if you could clarify whether and when KNLMeansCL is appropriate to use as a plain denoiser by itself ? I noticed it's been used in SMDegrain however I am unclear if KNLMeansCL is appropriate for use as a denoiser in its own right, for example on a range of OTA TV captures for moderate denoising a la mdegrain1/2/3. Any advice or links to comparisons somewhere ?

Another objective is to attempt to identify GPU (eg OpenCL) based filters for the times when a fast workflow is OK, ie where some improved quality output is hoped for but not paramount and speed is valued. (Tools = ffmpeg and x264, portable vapoursynth_x64, win10_x64.)

So far I have only seen these GPU based filters

DGDecodeNV for GPU decoding / deinterlacing (nvidia PureVideo) / resizing - in vapoursynth
unsharp - an ffmpeg internal filter which uses OpenCL to sharpen
KNLMeansCL for denoising - OpenCL in vapoursynth


Do you know of any other GPU filters usable in vapoursynth, or ffmpeg, especially sharpeners ? Is there already a list somewhere ?

Thanks.

[Khanattila]

Hello. A newbie type question, however worth asking since I find I am in need of guidance.

I wonder if you could clarify whether and when KNLMeansCL is appropriate to use as a plain denoiser by itself ? I noticed it's been used in SMDegrain however I am unclear if KNLMeansCL is appropriate for use as a denoiser in its own right, for example on a range of OTA TV captures for moderate denoising a la mdegrain1/2/3. Any advice or links to comparisons somewhere ?

Another objective is to attempt to identify GPU (eg OpenCL) based filters for the times when a fast workflow is OK, ie where some improved quality output is hoped for but not paramount and speed is valued. (Tools = ffmpeg and x264, portable vapoursynth_x64, win10_x64.)

So far I have only seen these GPU based filters

DGDecodeNV for GPU decoding / deinterlacing (nvidia PureVideo) / resizing - in vapoursynth
unsharp - an ffmpeg internal filter which uses OpenCL to sharpen
KNLMeansCL for denoising - OpenCL in vapoursynth


Do you know of any other GPU filters usable in vapoursynth, or ffmpeg, especially sharpeners ? Is there already a list somewhere ?

Thanks.

I believe that my filter is the exception, not the rule.
Do not focus yourself on gpu based filters.

[feisty2]

Hello. A newbie type question, however worth asking since I find I am in need of guidance.

I wonder if you could clarify whether and when KNLMeansCL is appropriate to use as a plain denoiser by itself ? I noticed it's been used in SMDegrain however I am unclear if KNLMeansCL is appropriate for use as a denoiser in its own right, for example on a range of OTA TV captures for moderate denoising a la mdegrain1/2/3. Any advice or links to comparisons somewhere ?

Another objective is to attempt to identify GPU (eg OpenCL) based filters for the times when a fast workflow is OK, ie where some improved quality output is hoped for but not paramount and speed is valued. (Tools = ffmpeg and x264, portable vapoursynth_x64, win10_x64.)



KNLMeansCL itself is an "appropriate" and very high quality (theoretically better quality than motion compensation based filters like MDeGrain since motion compensation matches macroblocks in temporal dimension only and NLMeans does that in both spatial and temporal dimensions) denoiser, it has been misused to do other things thanks to folks like me, but that doesn't mean it has lost its original purpose, to work as a plain denoiser!

[feisty2]

Also it's a big mystery that how the hell NLMeans ended up acting as a "pre filter" for MDeGrain? You do realize that's like doing a motion compensation pre filtering for RemoveGrain, right?

EDIT: simple rule, the fancier filter gets to be the main filter, so MDeGrain should be the pre filter (rclip) and NLMeans should be the main filter.

[real.finder]

Also it's a big mystery that how the hell NLMeans ended up acting as a "pre filter" for MDeGrain? You do realize that's like doing a motion compensation pre filtering for RemoveGrain, right?

EDIT: simple rule, the fancier filter gets to be the main filter, so MDeGrain should be the pre filter (rclip) and NLMeans should be the main filter.

ask dogway, anyway maybe he want a pure temporal, and the knlmeans have blend artfact in temporal dimension

[feisty2]

ask dogway, anyway maybe he want a pure temporal, and the knlmeans have blend artfact in temporal dimension

I wouldn't be surprised about the "blending" artifacts since a lot of people that use this filter got an "a" value of 2 or 3 or so.
Apparently the term "Non-Local" is so rocket science and incomprehensible to them and ironically, it is the "Non-Local Means" filter they are using.

Set "a" to 32 and if you can still observe any blending artifacts, post a sample

[Khanattila]

https://github.com/Khanattila/KNLMeansCL/releases/tag/v1.0.0-beta.3

KNLMeansCL v1.0.0-beta.3
New:
* Improved performance with CPU.
* Improved performance with AMD GCN Architecture.
* Reduced CPU overload in some system.
* Two modified bisquare weighting functions.
* Advanced OpenCL parameters for fine tuning.

Changed:
* Replaced 'cmode' with 'channels' and added the options to only process the chroma.
* Increased the maximum 's' value to 8.
* Updated to VapourSynth R35.

Removed:
* Cauchy weighting function.

Fixed:
*Second clip 'rclip' in some circumstances.


It corrects the errors of the previous beta.

[feisty2]

http://i.imgur.com/MxVdYfp.png
something is still wrong..

[feisty2]

I located the error to "rclip", it will crash if rclip is not None

[Khanattila]

I located the error to "rclip", it will crash if rclip is not None

The bugs do not exist until someone discovers them, I am sure.

[feisty2]

The bugs do not exist until someone discovers them, I am sure.

I only test for floating point inputs, couldn't care less about all that integer crap.

http://i.imgur.com/rjGES9f.png

script works fine with v0.7.7

[cork_OS]

Beta3 give this error (beta2 work ok):
OpenCL Platform
------------------------------------------------------------
CL_PLATFORM_VENDOR: Advanced Micro Devices, Inc.
CL_PLATFORM_NAME: AMD Accelerated Parallel Processing
CL_PLATFORM_VERSION: OpenCL 2.0 AMD-APP (2236.10)
CL_PLATFORM_PROFILE: FULL_PROFILE

OpenCL Device
------------------------------------------------------------
CL_DEVICE_VENDOR: Advanced Micro Devices, Inc.
CL_DEVICE_NAME: Pitcairn
CL_DRIVER_VERSION: 2236.10
CL_DEVICE_VERSION: OpenCL 1.2 AMD-APP (2236.10)
CL_DEVICE_PROFILE: FULL_PROFILE
CL_DEVICE_IMAGE_SUPPORT: 1
CL_DEVICE_IMAGE2D_MAX_WIDTH: 16384
CL_DEVICE_IMAGE2D_MAX_HEIGHT: 16384
CL_DEVICE_IMAGE_MAX_ARRAY_SIZE: 2048

Program Build
------------------------------------------------------------
CL_PROGRAM_BUILD_OPTIONS: -cl-single-precision-constant
-cl-denorms-are-zero
-cl-fast-relaxed-math
-Werror
-D NLM_CLIP_TYPE_UNORM
-D NLM_CLIP_REF_LUMA
-D NLM_WMODE_WELSCH
-D VI_DIM_X=720
-D VI_DIM_Y=480
-D HRZ_RESULT=1
-D VRT_RESULT=1
-D HRZ_BLOCK_X=32
-D HRZ_BLOCK_Y=8
-D VRT_BLOCK_X=32
-D VRT_BLOCK_Y=8
-D NLM_D=1
-D NLM_S=4
-D NLM_H=1.200000
-D NLM_WREF=1.000000
CL_PROGRAM_BUILD_LOG:
"C:\Users\cork_OS\AppData\Local\Temp\OCL7900T8.cl", line 19: error: global
variable declaration is corrected by the compiler to have addrSpace
constant
const sampler_t nne = CLK_NORMALIZED_COORDS_FALSE | CLK_ADDRESS_NONE | CLK_FILTER_NEAREST;
^

"C:\Users\cork_OS\AppData\Local\Temp\OCL7900T8.cl", line 20: error: global
variable declaration is corrected by the compiler to have addrSpace
constant
const sampler_t clm = CLK_NORMALIZED_COORDS_FALSE | CLK_ADDRESS_CLAMP | CLK_FILTER_NEAREST;
^

2 errors detected in the compilation of "C:\Users\cork_OS\AppData\Local\Temp\OCL7900T8.cl".
Frontend phase failed compilation.


RETURN: 0

[jmac698]

Does this work with opencl 1.1? I have Fermi/Nvidia. Maybe before version 0.7?

[Groucho2004]

Does this work with opencl 1.1? I have Fermi/Nvidia. Maybe before version 0.7?
See here (https://github.com/Khanattila/KNLMeansCL/wiki/Requirements), last paragraph.

[jmac698]

Right, thanks. Testing.

Edit:
colorbars()
KNLMeansCL()
Plugin was designed for a later version of Avisynth
Am using Avisynth+ 0.1 (r1576, x86)

Also another question, if it did work with x64 version of Avs+, and I used 64bit version of plugin, would I then need 64bit version of vc2013?

Was hoping to use it with Avs+.

Will try to find a later version.

Edit 2:
Found latest build on page 145 of Avisynth+ thread.
http://www.mediafire.com/file/bazu8v58ab318j6/avsplus-r2397.7z

Edit 3:
Used Avs+ 0.1 (r2397, MT, i386)
KNLMeansCL: AviSynthCreate error (clBuildProgram)!
Please report Log-KNLMeansCL.txt
I can't find that .txt file.

Edit 4:
Got it to work :)
Avs 2.60
KNLMeansCL(device_type="GPU")
Also found the log file now (and maybe was previously) at same place as .avs file.

The errors were:
---------------------------------
*** Error in OpenCL compiler ***
---------------------------------

# Build Options
-cl-single-precision-constant -cl-denorms-are-zero -cl-fast-relaxed-math -Werror -D H_BLOCK_X=32 -D H_BLOCK_Y=4 -D V_BLOCK_X=32 -D V_BLOCK_Y=4 -D NLMK_TCLIP=76 -D NLMK_S=4 -D NLMK_WMODE=1 -D NLMK_TEMPORAL=0 -D NLMK_H2_INV_NORM=185.828175 -D NLMK_BIT_SHIFT=0

# Build Log
:119:47: error: double precision constant requires cl_khr_fp64, casting to single precision
:122:40: error: double precision constant requires cl_khr_fp64, casting to single precision
:125:43: error: double precision constant requires cl_khr_fp64, casting to single precision


So I had to set device type to GPU. Works with Nvidia Fermi and opencl 1.1.

Next: to test avs+ again, then benchmark.