xClean 3-Pass Denoiser

[PatchWorKs]

@Selur just tested HINet (https://github.com/megvii-model/HINet), how does it performs (in terms of both speed and fidelity) compared to xClean ?

Authors claims:
With the help of HIN Block, HINet surpasses the state-of-the-art (SOTA) on various image restoration tasks. For image denoising, we exceed it 0.11dB and 0.28 dB in PSNR on SIDD dataset, with only 7.5% and 30% of its multiplier-accumulator operations (MACs), 6.8 times and 2.9 times speedup respectively. For image deblurring, we get comparable performance with 22.5% of its MACs and 3.3 times speedup on REDS and GoPro datasets. For image deraining, we exceed it by 0.3 dB in PSNR on the average result of multiple datasets with 1.4 times speedup.

Has anyone tested https://github.com/HolyWu/vs-hinet ?

Here are a few screen shots: (not sure what to make of them and for what content this is really useful)

Mode: Deblur GoPro
https://i.ibb.co/0MjBKN3/deblur-Go-Pro.png (https://ibb.co/QNF8kBG)
Mode: Deblur REDS
https://i.ibb.co/WyWdJyG/deblur-REDS.png (https://ibb.co/xjScTjX)
Mode: denoise
https://i.ibb.co/2SP8LJ7/denoise.png (https://ibb.co/pQZWsqv)
Mode: derain
https://i.ibb.co/zP0j9sH/derain.png (https://ibb.co/wpx3Gyr)

[Reclusive Eagle]

Bit of an older post but why not use Dfttest as a reference + BM3D?

I also find pre-sharpening the clip before the reference preserves more detail.
Using this combination preserves all but the most minute details in heavy noise clips.
I'm talking less than 1%-0.5% and only in the already faintest of details.

For anything that isn't SD compressed DVD grain, you will preserve all detail this way



In fact honestly after testing this I would recommend:

1: Double the resolution of your clip and then sharpen it. (This reduces the effect of sharpening noise)
2: Denoising with Dfttest
3: Using Dfftest for a BM3D Reference
4: Using the first BM3D Reference as a reference for a second BM3D
5: Downscale to original resolution

This will preserve massive amounts if not all detail, and because it relies on BM3D Cuda will result in 2fps + renders.
So +- 2 hours for 25 minutes of footage. If you have a better GPU than my 2014 750 TI you will get dramatically faster results.

Here is an example of a very noisy DVD IVTC:

Here is the original image after IVTC:
https://i.slow.pics/4c69Dt8B.png

Here is the same image after denoising:
https://i.slow.pics/jNtO6qyC.png

Even the clouds preserve all detail. Which is extremely, extremely hard to preserve in older anime backgrounds when denoising

To achieve this I: ​
1: pre-sharpened with LSFMod
2: Denoised with Dfftest
3: Denoised with BM3D using Dfftest as a reference
4: Denoised with a second BM3D using the first BM3D as a reference.



Here is the code I used if you or anyone else would like to replicate it.
Just remember this Dfftest's nlocation is custom made for this DVD to preserve extremely small details. It won't work on all clips.

#Upscale
clip = core.resize.Lanczos(clip, 1440,1080)

#Pre-sharpen clip
clip = haf.LSFmod(clip, preblur=3, strength=110, Smode=1,Smethod=3, soothe=True, edgemaskHQ=True, secure=True,soft=10)
clip=core.fmtc.bitdepth (clip,bits=32)

#3 Pass Denoise
ref = core.dfttest.DFTTest(clip, ftype=1, tbsize=3, nlocation=[35,0,0,10, 28,0,83,87, 95,0,0,100],sigma=10.0,sst=[0.0,2.0, 1.0,20.0], ssystem=1,opt=3)
ref2= core.bm3dcuda.BM3D(clip, ref=ref, sigma=[25,25,25], block_step=3)
clip= core.bm3dcuda.BM3D(clip, ref=ref2, sigma=[17,17,17], block_step=3)

clip=core.fmtc.bitdepth (clip,bits=8)
clip.set_output()

Notice how each denoise decreases in intensity. Also for BM3D Block step of 1 will reduce performance by 80%.
Having it at 3 is the difference between rendering at 5.5fps vs 0.6fps. And there is ZERO difference in quality (in this case)

[Reclusive Eagle]

In fact at this point the denoising is so good with BM3D if you wanted to you can just continue stacking BM3D references at less intensity.
You can have 8 pass noise reduction with 1 dfttest and 7 BM3D's with lightning speed compared to KLMeans.

And don't be afraid to add more sharpness after every few stacks.
For example, taking the above code I stacked it 4 times but this time added more sharpness to the original clip but because the references were so good-
I retained more detail, increased overall sharpness and gained zero new noise compared to the above example with this code:


#Upscale
clip = core.resize.Lanczos(clip, 1440,1080)

#Pre-sharpen
clip = haf.LSFmod(clip, preblur=3, strength=110, Smode=1,Smethod=3, soothe=True, edgemaskHQ=True, secure=True,soft=10)
clip=core.fmtc.bitdepth (clip,bits=32)

#3 Pass denoise
ref = core.dfttest.DFTTest(clip, ftype=1, tbsize=3, nlocation=[35,0,0,10, 28,0,83,87, 95,0,0,100],sigma=10.0,sst=[0.0,2.0, 1.0,20.0], ssystem=1,opt=3)
ref2= core.bm3dcuda.BM3D(clip, ref=ref, sigma=[25,25,25], block_step=3)
ref3= core.bm3dcuda.BM3D(clip, ref=ref2, sigma=[17,17,17], block_step=3)

#Post reference sharpen
clip = haf.LSFmod(clip, preblur=3, strength=40, Smode=1,Smethod=3, soothe=True, edgemaskHQ=True, secure=True,soft=10)

#Final 4th pass denoise
clip= core.bm3dcuda.BM3D(clip, ref=ref3, sigma=[13,13,13], block_step=3)

clip=core.fmtc.bitdepth (clip,bits=8)
clip.set_output()

The results are again, increased sharpness due to the post reference sharpen however with zero increase to the overall noise in the final 4th denoise pass.
Obviously you can see sharpening halos. But those can be entirely fixed by masking.

You can find a really detailed masking tutorial on kageru's blog:
https://blog.kageru.moe/legacy/edgemasks.html

and you can fix sharpening halos directly with this tutorial also by Kageru:
https://guide.encode.moe/encoding/masking-limiting-etc.html

Btw with this setup including Nnedi 3 and IVTC and upscaling and 4 pass denoising -
I am rendering at 2.8fps in prores 1440x1080 with an i5 9600k and GTX 750 Ti.

Just as a baseline for anyone who want's to compare future settings.


I also recommend tbilateral after your pre-sharpen but before your denoise references or-
Apply it to your final reference before final denoise (same affect either way).

This will give you an insanely clean image

[kedautinh12]

Wow, it's great :D

[MysteryX]

Reclusive Eagle, your reference clip is an anime while mine is raw camera footage -- great method, but completely different use-case.

It gives a nice polishing effect on your anime; but that will most likely give the plastic effect on other videos that I've been trying to avoid.

[MysteryX]

Here's a denoising comparison video for xClean (https://www.youtube.com/watch?v=Z0OrPP0VOUI)

[WarnerBrah]

Script Error
Script error: There is no function named 'nmod'.
TransformsPack - Main.avsi, line 456
xClean.avsi, line 232

any help?

[kedautinh12]

You lack this script
https://github.com/Dogway/Avisynth-Scripts/blob/master/ResizersPack.avsi

[WarnerBrah]

nnedi3_weights.bin is missing now

[kedautinh12]

You lack NNEDI3CL, download extra 1.0.3 to got .bin file and put same folder with .dll file from 1.0.5
https://github.com/Asd-g/AviSynthPlus-NNEDI3CL/releases

[WarnerBrah]

Script Error

System exception - Access Violation
xClean.avsi, line 516
xClean.avsi, line 535
xClean.avsi, line 550
xClean.avsi, line 233

[kedautinh12]

You need avs+ and vc++ latest ver
https://forum.doom9.org/showthread.php?p=1978415#post1978415
https://github.com/abbodi1406/vcredist

[WarnerBrah]

thank you kind sir! script is working in stax as prefilter to SMDegrain. Quite The Combo.

[asarian]

Odd. Installed it properly via "vsrepo install xclean", yet get following error:

File "", line 9, in
NameError: name 'xclean' is not defined

[_Al_]

did not tried that filter yet, but on Github the name is xClean.py, capital "C"

[asarian]

did not tried that filter yet, but on Github the name is xClean.py, capital "C"

It installed fine (and is called xClean.py too). And I invoked it like:

vid = xclean.xClean(vid, sharp=21, strength=-50, outbits=16, finalm=1, f1=1.4)

As per the author's example.

[Boulder]

It installed fine (and is called xClean.py too). And I invoked it like:

vid = xclean.xClean(vid, sharp=21, strength=-50, outbits=16, finalm=1, f1=1.4)

As per the author's example.
Did you import it in the script first?

[ChaosKing]

It installed fine (and is called xClean.py too). And I invoked it like:

vid = xclean.xClean(vid, sharp=21, strength=-50, outbits=16, finalm=1, f1=1.4)

As per the author's example.

You should then import like this first
import xClean as xclean

or (untested)from xClean import xClean
and the you should be able to simply call xClean(vid, sharp=21, strength=-50, outbits=16, finalm=1, f1=1.4) without any prefix

[asarian]

Did you import it in the script first?

To my great shame, I didn't. :( Works like a charm now.

[asarian]

You should then import like this first
import xClean as xclean

or (untested)from xClean import xClean
and the you should be able to simply call xClean(vid, sharp=21, strength=-50, outbits=16, finalm=1, f1=1.4) without any prefix

I came up with something like this

vid = xclean.xClean(vid, strength=20, outbits=10, dmode=3, gpuid = 0, gpucuda = 0, h=2.0, m3=2)
vid = core.neo_f3kdb.Deband (vid, preset="veryhigh", dither_algo=2)

Few questions remain. Can 'Deband=True' replace my last line? And I set 'gpucuda=0'. Seems to really use my GPU heftily (3080 Ti). Good choice, or will it slow down the render?


EDIT: memory hungry beast xCLean is. :) 4k process takes 51G ram (out of my 64G).

[tormento]

Moved to AVS+ thread (https://forum.doom9.org/showthread.php?p=2012430#post2012430).