Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
24th September 2021, 18:41 | #1 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
xClean 3-Pass Denoiser
xClean 3-Pass Denoiser for VapourSynth
Here's a denoising comparison video beta 7 (2021-10-27) by Etienne Charland Supported formats: YUV, RGB, GRAY Requires: rgsf, rgvs, fmtc, mv, mvsf, tmedian, knlm, bm3d, bm3dcuda_rtc, bm3dcpu, neo_f3kdb, akarin, nnedi3_resample, nnedi3cl xClean runs MVTools -> BM3D -> KNLMeans in that order, passing the output of each pass as the ref of the next denoiser. The objective is to remove noise while preserving as much details as possible. Removing noise is easy -- just blur out everything. The hard work is in preserving the details in a way that feels natural. Designed for raw camera footage to remove noise in dark areas while preserving the fine details. It works for most types of content. Performance-wise, BM3D pass is the heaviest and helps recover fine details, but this script runs 1 pass of BM3D whereas stand-alone BM3D runs twice. +++ Short Doc (TL;DR) +++ Default settings provide the best quality in most cases. Simply use xClean(sharp=..., outbits=...) If only darker areas contain noise, set strength=-50 For better performance, set m1=0 or m2=0, or set m1=.5 and m2=3.6 (downscale) BM3D performance can be greatly improved by setting radius=0, block_step=7, bm_range=7, ps_range=5 For 720p WebCam, optimal settings are: sharp=9.5, m1=.65, h=2.8 For 288p anime, optimal settings are: sharp=9.5, m1=.7, rn=0, optional depth=1 For 4-5K GoPro (with in-camera sharpening at Low), optimal settings are: sharp=7.7, m1=.5, m2=3.7, optional strength=-50 (or m1=.6, m2=3.8 if your computer can handle it) +++ Description +++ KNLMeans does a good job at denoising but can soften the image, lose details and give an artificial plastic look. I found that on any given source (tested 5K GoPro footage and noisy WebCam), denoising with less than h=1.4 looks too noisy, and anything above it blurs out the details. KNLMeans also keeps a lot of data from the clip passed as rclip, so doing a good prefilter highly impacts the output. Similarly, BM3D performs best with sigma=9. A lower value doesn't remove enough noise, and a higher value only makes the edges sharper. xClean is essentially KNLMeans with advanced pre-filtering and with post-processing to renoise & sharpen to make the image look more natural. One strange aspect of xClean is that denoising is automatic and there's very little room to configure denoising strength other than reducing the overall effect. It runs with BM3D sigma=9 and KNL h=1.4, and generally you shouldn't change that. One setting that can allow increasing denoising (and performance) is downscaling MVTools and BM3D passes. You can also set h=2.8 if the output remains too noisy. h = 1.4 or 2.8 are generally the best values. According to my tests, water & cliff 5K video with little noise preserves the details very well while removing subtle grain, and with same settings, very noisy 720p WebCam footage has HUGE noise reduction while preserving a surprising amount of natural details. The default settings are very tolerant to various types of clips. All processing is done in YUV444 format. When conv=True, processing is done in YCgCoR, and in OPP colorspace for BM3D. +++ Denoising Methods Overview +++ To provide the best output, processing is done in 3 passes, passing the output of one pass as the ref clip of the 2nd pass. Each denoiser has its strengths and weaknesses. Pass 1: MVTools (m1) Strength: Removes a lot of noise, good at removing temporal noise. Weakness: Can remove too much, especially with delicate textures like water. Ref: Impacts vectors analysis but low impact on outcome Pass 2: BM3D (m2) Strength: Good at preserving fine details! Weakness: Doesn't remove much grain. Ref: Moderate impact on outcome. A blurry ref will remove more grain while BM3D puts back a lot of details. Pass 3: KNLMeansCL (m3) Strength: Best general-purpose denoiser Weakness: Can blur out details and give an artificial plastic effect Ref: High impact the outcome. All prefilters benefit from running KNLMeans over it. +++ Denoising Pass Configuration (m1=.6, m2=2, m3=2) +++ Each pass (method) can be configured with m1 (MVTools), m2 (BM3D) and m3 (KNLMeansCL) parameters to run at desired bitdepth. This means you can fine-tune for quality vs performance. 0 = Disabled, 1 = 8-bit, 2 = 16-bit, 3 = 32-bit Note: BM3D always processes in 32-bit, KNLMeansCL always processes in 16-bit+, and post-processing always processes at least in 16-bit, so certain values such as m2=1, m3=1 will behave the same as m2=2, m3=2. Setting m2=2 instead of 3 will only affect BM3D post-processing (YUV444P16 instead of YUV444PS) MVTools (m1) and BM3D (m2) passes can also be downscaled for performance gain, and it can even improve quality! Values between .5 and .8 generally work best. Optional resize factor is set after the dot: m1 = .6 or 1.6 processes MVTools in 8-bit at 60% of the size. m2 = 3.6 processes BM3D in 16-bit at 60% of the size. You may want to downscale MVTools (m1) because of high CPU usage and low impact on outcome. You may want to downscale BM3D (m2) because of high memory usage. If you run out of memory, lower the size until you get no hard-drive paging. Note: Setting radius=0 greatly reduces BM3D memory usage! +++ Renoise and Sharpen (rn=14, sharp=9.5) +++ The idea comes from mClean by Burfadel (https://forum.doom9.org/showthread.php?t=174804) and the algorithm was changed by someone else while porting to VapourSynth, producing completely different results -- original Avisynth version blurs a lot more, VapourSynth version keeps a lot more details. It may sound counter-productive at first, but the idea is to combat the flat or plastic effect of denoising by re-introducing part of the removed noise. The noise is processed and stabilized before re-inserting so that it's less distracting. Renoise also helps reduce large-radius grain; but should be disabled for anime (rn=0). Using the same analysis data, it's also sharpening to compensate for denoising blur. Sharpening must be between 0 and 20. Actual sharpening calculation is scaled based on resolution. +++ Strength / Dynamic Denoiser Strength (strength=20) +++ A value of 20 will denoise normally. A value between 1 and 19 will reduce the denoising effect by partially merging back with the original clip. A value between 0 and -200 will activate Dynamic Denoiser Strength, useful when bright colors require little or no denoising and dark colors contain more noise. It applies a gradual mask based on luma. Specifying a value of -50 means that out of 255 (or 219 tv range), the 50 blackest values have full-reduction and the 50 whitest values are merged at a minimal strength of 50/255 = 20%. +++ Radius (radius=0) +++ BM3D radius. Low impact on individual frames. Pros: Helps stabilize temporal grain. Can significantly improve video compressability. Cons: High impact on performance and memory usage! May require downscaling BM3D for HD content with m2 between 3.6 and 3.8 For moving water, the temporal stabilization may be undesirable. +++ Depth (depth=0) +++ This applies a modified warp sharpening on the image that may be useful for certain things, and can improve the perception of image depth. Settings range up from 0 to 5. This function will distort the image, for animation a setting of 1 or 2 can be beneficial to improve lines. +++ Deband (deband=False) +++ This will perceptibly improve the quality of the image by reducing banding effect and adding a small amount of temporally stabilised grain to both luma and chroma. Default settings are suitable for most cases without having a large effect on compressibility. +++ Output (outbits, dmode=0) +++ Specifies the output bitdepth. If not specified it will be converted back to the bitdepth of the source clip using dithering method specified by dmode. You can set dmode=3 if you won't be doing any further processing for high-quality ditherig. +++ Chroma upsampling/downsamping (chroma=nnedi3, downchroma=True) +++ Chroma upsampling options: none = don't touch chroma bicubic = bicubic(0, .5) upsampling nnedi3 = NNEDI3 upsampling reconstructor = feisty2's ChromaReconstructor_faster v3.0 HBD mod downchroma: whether to downscale back to match source clip. Default is False for reconstructor and True for other methods. +++ Anime +++ For anime, set rn=0. Optionally, you can set depth to 1 or 2 to thicken the lines. +++ Advanced Settings +++ gpuid = 0: The GPU id to use for KNLMeans and BM3D, or -1 to use CPU. gpucuda = 0: The GPU id to use for BM3D, or -1 to use CPU. h = 1.4: KNLMeans strength, can increase slightly if the output is still too noisy. 1.4 or 2.8 generally work best. block_step = 4, bm_range = 16, ps_range = 8: BM3D parameters for performance vs quality. No impact on CPU and memory. Adjust based on GPU capability. Fast settings are block_step = 5, bm_range = 7, ps_range = 5 Normally you shouldn't have to touch these rgmode = 18: RemoveGrain mode used during post-processing. Setting this to 0 disables post-processing, useful to compare raw denoising. thsad = 400: Threshold used for MVTools analysis. d = 2: KNLMeans temporal radius. Setting 3 can either slightly improve quality or give a slight plastic effect. a = 2: KNLMeans spacial radius. sigma = 9: BM3D strength. bm3d_fast = False. BM3D fast. conv = True. Whether to convert to OPP format for BM3D and YCgCoR for everything else. If false, it will process in standard YUV444.
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 6th February 2022 at 20:43. |
24th September 2021, 18:53 | #2 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
Remains a few things to fix
- KNLMeansCL, what would be the optimal settings for dark scenes? Should chroma be processed with the same strength as luma? Would rclip be useful here? - BM3D, what would be the right way to use it? It could be added as method=2 - I'll do some improvement to the dark scene detection - You can test that strength, sharp and other parameters are doing what they're supposed to be doing It will be converted to Avisynth only when the VapourSynth script is completed. MvTools2 temporal analysis runs MUCH smoother in VapourSynth. I did very quick benchmark. Processing 5K video at 8-bit: 1.4 fps 16-bit: 0.8 fps 32-bit: 0.20 fps 8-bit with outbits=16: 1.2 fps
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 25th September 2021 at 05:37. |
24th September 2021, 19:47 | #3 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,821
|
Nice work.
The new Vapoursynth version (API4) does not support COMPAT anymore. mvsf.Degrain3, mvsf.Degrain4 means it's refers to the old mvtools-sf version. The new version only knows mvsf.Degrain.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
24th September 2021, 21:59 | #5 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,821
|
It seems you now need to specify a radius in Analyze and this is then basically Degrain3, 4, 24 etc. It works now with any radius.
https://forum.doom9.org/showthread.p...78#post1911978
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
25th September 2021, 02:46 | #6 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
With r10-prerelease, I cannot specify the 4 vector clips to Degrain, the syntax is completely different. For now I'll stay with r9.
8-bit runs at 1.4fps, 16-bit at 0.8fps, and 32-bit at 0.20fps. 32-bit output has a very slight colour shift. |
25th September 2021, 02:53 | #7 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
Fixed the sharpener. It works in 32-bit with mvsf r9.
I disabled boost by default. There's no good default value for it, as it depends for each video source. Many videos don't need it at all. If you have a troublesome video with dark scenes, best thing to do is to scan the video using this to find the right average luma threshold. Code:
clip = clip.std.PlaneStats().text.FrameProps(scale=4) Note that if I set boost=28 and a scene has 26% luma, it doesn't fully take the blurrier KNLMeansCL, but it's a partial merge based on how dark it goes below the threshold. I could do a fancier dark scene detection by discarding the brightest pixels if they cover less than 15%, to average everything else, but I'm not sure it's worth it.
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 25th September 2021 at 03:18. |
25th September 2021, 22:35 | #8 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
Added proper outbits support with dither at the end if necessary. Post-processing in 16-bit is worth it with minimal performance penalty, with 10-bit dithering if encoding in 10-bit.
Question: which dmode dithering method is recommended to use here? Curiously, bit-depth and dithering is impacting more the file size than I thought: - 8-bit processing: 7359kb - 8-bit processing with 16-bit post-processing with 10-bit dithering: 6992kb - 16-bit processing with 10-bit dithering: 6742kb - 32-bit processing with 10-bit dithering: 6686kb
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 25th September 2021 at 22:42. |
25th September 2021, 23:51 | #9 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
Fixed fullscreen detection that was reverse (full-range is 1 not 0)
Added method=2: BM3D (CUDA). Its effect seems to be very subtle and need to set sigma (p1) very high to see some difference. Since it's a conservative denoiser, not sure renoise/sharp will be so useful here but you can test it. For KNLMeansCL, what's the best way to process chroma? Right now it's processing luma first, then chroma separately. Should instead convert to YUV444 and process all planes at once? Or process chroma differently? |
26th September 2021, 14:56 | #10 | Link | ||
Registered User
Join Date: Jul 2021
Posts: 26
|
Quote:
That aside I am going to argue against this kind of option, controlling the depth of your clips is something very fundamental and something all users should know how to do without relying on the function to provide the option. It also adds needless complexity to the function. Quote:
BTW if possible, I think vcm is windows only and managing that on Linux is kind of annoying. Any chance a replacement could be found? I don't know of any other windows exclusive vapoursynth filters. Last edited by Quadratic; 26th September 2021 at 15:06. |
||
26th September 2021, 15:16 | #11 | Link | |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
Quote:
Only thing that would be optional is auto-dithering for 10bit, 12bit and 14bit outbits; but then supporting 10-bit outbits instead of 16-bit outbits is also optional. Output isn't the same. I believe I read that it's processing chroma based on Luma analysis or something, but can't find where I read that (or I imagined it) |
|
27th September 2021, 06:22 | #12 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
Added finalm parameter to run KNLMeansCL (finalm=1) or BM3d (finalm=2) with xClean as ref. Results are impressive.
This is the syntax to get the best results according to my tests. Code:
clean = xclean.xClean(clip, sharp=21, strength=-50, outbits=16, finalm=1, f1=1.4) |
28th September 2021, 01:18 | #14 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
You need to zoom into the 5K image to see it at full resolution to see the details. There's noise at the top, difficult details in the beard, moving trees in the background, and square textures on the seat. Small differences if you look closely.
If you just open the web page it doesn't allow zooming to 100%, need to drag that image into a new tab. Edit: the xClean image seemed to be wrong, I updated it. Haven't double-checked if there's any other mixup.
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 28th September 2021 at 03:27. |
29th September 2021, 06:16 | #15 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
xClean now fully support MVTools (0), KNLMeansCL (1) and BM3D (2) methods with renoising and sharpening, and can then apply KNLMeans (1) or BM3D (2) with the first method as ref using finalm.
I did extensive testing with screenshots here. Very interesting results! Remains a question. Renoise and sharpen run on the first denoise method, which then gets fed to the 2nd denoiser. Is it useful to re-apply renoise and/or sharpen again? The images already look pretty damn good. Re-applying them would require running RGTools to then calculate the sharpen and renoise masks. I do re-apply dynamic denoising strength; that's the only post-processing I'm re-applying the 2nd time. |
2nd October 2021, 01:57 | #16 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
Now all 3 methods can be used as final methods; but only KNLMeansCL(1) is useful. MVTools doesn't keep much from prefilter so it's a waste to feed complex denoising into it. BM3D doesn't take much from prefilter either and has poor temporal stability. As for feeding a prefilter into KNLMeans, I have yet to see cases where it produces worse results than prefilter itself.
Also adjusted sharpening to be consistent across methods. |
2nd October 2021, 16:20 | #18 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
As final method. I'm refactoring it so that you can enable/disable each component but they stay in the same order: MVTools -> BM3D -> KNLMeans. Plus configure in what bit-depth each will run (post-processing will be in 16-bit)
|
3rd October 2021, 01:46 | #19 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
Refactored to run in this order: MVTools -> BM3D -> KNLMeans, each with configurable bitdepth with m1 (MVTools), m2 (BM3D) and m3 (KNLMeans). 0=disabled, 1=8bit, 2=16bit, 3=32bit
Default is m1=1, m2=0, m3=2. All content look better with m1=1, m2=1, m3=2 (m2=1 or m2=2 is the same as it runs in 32-bit) I also removed vcm.Median as part of the output, now it was only pre-processing. Saw it was a HUGE performance hog so replaced it as MVTool prefilter with Convolution matrix. Here are some benchmark on 5K footage, running on Intel i7-10750H with GeoForce GTX 2060 "1 0 1" means m1=1, m2=0, m3=1. In order from lower quality to best quality. 0 0 1: 2.26fps (KNLMeans d=2, a=2) 1 0 1: 1.69fps 1 0 2: 1.77fps (faster??) 1 0 2 d=3: 0.95fps 0 1 2: 1.42fps 0 1 2 d=3 radius=1: 0.32fps 1 1 2: 0.82fps 1 1 2 d=3: 0.28fps 1 1 2 radius=1: 0.11fps -- 16GB RAM isn't enough Not sure why d=3 is absolutely killing performance when paired with BM3D. SMDegrain could probably be optimized for performance; if someone knows how it works and has ideas. Now the quality test is interesting. On a general clip, 1 0 2 d=3 (0.95fps) and 1 1 2 (0.82fps) have almost the exact same quality improvement over 1 1 2. They look nearly identical. The difference is that 1 1 2 will be more consistent across a variety of scenes. As for adding h=3 and radius=1 over 1 1 2... I looked very very closely and can't tell the difference. Not worth investing $4000 into a new Threadripper with 64GB RAM. Mode 1 1 2 thus seems to be the best option overall, and 1 0 2 for performance. Image comparison: Driving I challenge you to find a difference between 1 1 2 and 1 1 2 d3r1. It's also comparing with simple KNLMeans .When you add renoise & sharpen, file size goes up from 13.4MB to 15.5MB and it looks more natural. Since I removed Median at the start, it makes the image sharper... sharp=21 may be too high now. (screenshots are with sharp=21) Performance test on 720p WebCam: KNLMeansCL(d=2, a=2): 33.3fps 0 0 1: 33.1fps 1 0 1: 32.2fps 1 0 2: 32.25fps 1 0 2 d=3: 23.0fps 0 1 2: 27.65fps 0 1 2 d=3 radius=1: 20.6fps 1 1 2: 26.84fps 1 1 2 d=3: 20.43fps 1 1 2 d=3 radius=1: 17.46fps Wow, the highest setting runs only at half the performance of plain KNLMeans!? RAM really was the issue above. Image comparison: WebCam It's surprising the amount of noise that it's taking out and the amount of details it preserves with a natural feel. I've included an image with radius=1 and radius=2. Radius=1 helps very slightly but radius=2 seems counter-productive. BTW, 26.84fps means real-time camera denoising. Because it's running 3 different denoisers, it is very forgiving and works for any kind of content (that I tested so far). Finally... because of 16RAM limitations, my real options are A) 1 1 2 encoding with NVEnc (0.80fps) B) 1 0 2 or 0 1 2 depending on content, encoding with x265 (... fps) never mind 0.1fps even that isn't an option. x265 alone encodes at 1.5fps though, but using 9GB ram on its own.
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 3rd October 2021 at 04:39. |
3rd October 2021, 17:06 | #20 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,560
|
Wow found a weird quality/performance setting. My 2 main concerns were
1) Performance 2) Inability to increase denoising strength For the first pass with MVTools, if you resize it down, it barely impacts the output and may even increase quality by increasing the denoising strength. So now if m1 is between 0 and 1, I resize it by that factor. Performance on 5K: 1 1 2: 0.82fps .5 1 2: 1.23fps .6 1 2: 1.14fps .6 1 2 d=3: 0.74fps (best output) To compensate for very slight blur, increase sharp=10 to sharp=11 Now I can re-encode my raw GoPro 5K footage. With x265 preset=fast and .5 1 2, it goes at 0.38fps with good RAM margin left. Or, with NVEnc at 1fps.
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 3rd October 2021 at 20:37. |
Thread Tools | Search this Thread |
Display Modes | |
|
|