Clay - Denoising script

[anton_foy]

Back again with yet another denoising function.

Clay tries to remove all noise while keeping the details intact. Simple, fast and temporally stable.
(loads of thanks to DTL, Dogway and StainlessS for the ideas and development etc.)

How it works:
Clay denoises bright areas in one mdegrain call and the dark areas in another then merging them together with a mask.
Dogway's brilliant ccd + iqmv + sbr combo for prefiltering.
To regain sharpness and detail it uses ex_unsharp before MSuper.
To smooth out bad blocks it uses vsdegrainmedian before mdegrain.
Neo VagueDenoiser in the dark areas for spatial smoothness.

(The same settings were used for all three images, see usage settings on the bottom of the page)
Before-After (ImgSli is compressing the images so banding occurs here)
House (Requested by Coolgit)
Test Clips (93mb)

#vsdegrainmedian = FAST!
#iqmv + ex_sbr = FAST!
#neo_vd = FAST!
#ex_unsharp = FAST!
#Clay - mdegrain = medium-fast.
#Results in combination = fine quality and fast!

requirements:
mvtools2, masktools2, neo vaguedenoiser

Additional(with my settings):
Vsdegrainmedian, deblock, neo_F3KDB, (Dogway's): extools, sharpeners pack, deblock pack


Function:

function clay(clip Input, clip "preclip", clip "superclip", clip "darkclip", clip "brightclip", int "DthSAD", int "DthSADC",
\ int "BthSAD", int "BthSADC", int "TR", int "BLKSize", int "Rb", int "Overlap", int "Pel", bool "Chroma",
\ float "VDTh", bool "tm", float "Falloff", int "UV", int "lo", int "hi", float "mid", int "mix", bool "fast")
{
DthSAD = default(DthSAD,  150)
DthSADC= default(DthSADC, 150)
BthSAD = default(BthSAD,   70)
BthSADC= default(BthSADC,  70)
TR = default(TR, 6)
BLKSize = default(BLKSize, 32)
Overlap = default(Overlap, 16)
Rb = default(Rb, 8)
Pel = default(Pel, 1)
Chroma = default(Chroma, true)
Tm = default(Tm, false)
Falloff = default(Falloff, 0.9)
VDth = default(VDth, 1.5)
lo = default(lo,     60)
mid = default(mid, 0.57)
hi = default(hi,    127)
mix = default(mix,  230)
UV = default(UV, 3)
Fast = default(Fast, false)
Bt= BitLShift(1,input.BitsPerComponent - 8)

preclip = Default(preclip,input)
superclip = Default(superclip,input)
darkclip = Default(darkclip,input)
brightclip = Default(brightclip,input)

super = MSuper(superclip, pel=pel, hpad = 0, vpad = 0, chroma=chroma, mt=false, levels=1, sharp=1)
superB = MSuper(superclip.merge(input), pel=pel, hpad = 0, vpad = 0, chroma=chroma, mt=false, levels=1, sharp=1)
superfilt = MSuper(preclip, pel=pel, hpad = 0, vpad = 0, mt=false,sharp=1)
Multi_Vector = Superfilt.MAnalyse(Multi=True, Delta=tr, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma, truemotion=tm ,global=true)
vmulti2 = Mrecalculate(superfilt,multi_vector,thsad=Dthsad,truemotion=tm,tr=tr,blksize=rb,overlap=rb/2,mt=false,chroma=true)

mv_bw1=SelectEvery(multi_vector, tr*2, 0)
mv_fw1=SelectEvery(multi_vector, tr*2, 1)
mv_bw2=SelectEvery(multi_vector, tr*2, 2)
mv_fw2=SelectEvery(multi_vector, tr*2, 3)

dark=darkclip.MDegrainN(Super, vmulti2, TR, thSAD=DthSAD, thSAD2=Int(DthSAD*Falloff), thSADC=DthSADC, thSADC2=Int(DthSADC*Falloff)).neo_vd(VDth,1,6,90)

bright = Fast ? input : brightclip.MDegrain2(SuperB, mv_bw1, mv_fw1, mv_bw2, mv_fw2, thSAD=BthSAD, thSADC=BthSADC)

mask=dark.extractY().levels(lo*bt, mid, hi*bt, 0, mix*bt)

mt_merge(dark, bright, mask, luma=true, u=uv, v=uv)

return last
}

Usage (my own settings):

convertbits(16)
converttoYUV444()

# My prefiltering (or use your own desired prefilter):
cd=convertbits(8,dither=-1).convertToYv12()
pre=cd.CCD(13).ex_median("IQMV",Y=3,UV=3).ex_sbr(2)

# Clay with preclip, superclip, darkclip and brightclip:
Clay(tr=7,dthsad=170,dthsadc=170,bthsad=100,preclip=pre,superclip=ex_unsharp(1.77,th=1.1,fc=width(),safe=true,UV=1),darkclip=vsdegrainmedian(),brightclip=last,chroma=true)

# Deblocking chroma and debanding luma:
deblock(planes="UV")
neo_F3KDB(grainY=0,grainC=0,Y=46,mt=true,output_depth=16,range=22,sample_mode=3).mergechroma(last)
Convertbits(10,dither=1)
prefetch(4)

Please give input
Enjoy!

Edit: changed "input" to "bright" in mt_merge call and inputD instead of darkclip, inputB instead of brightclip.
Edit2: updated function and usage with DTL's, StainlessS's and Gavino's input. Thanks guys!
Edit3: StainlessS cleanup for preclip, superclip, darkclip and brightclip, thanks!
Edit4: New "settings", changed params for neo_f3kdb for less smearing of details in dark areas.

[DTL]

For better MVs search in MAnalyse it may be recommended for 'slow/slowest/placebo' mode search=3, searchparam>2 and pelsearch>2, trymany=true. And may be some manual play with all penalties (non-zero values).

Also after yet another time reading the docs:
multi
When set to true, MAnalyse generates the motion vectors for all the reference frames −delta…−1 and +1…+delta. The resulting clip is made of single MAnalyse results interleaved in the delta order −1, +1, −2, +2, …, −delta, +delta, where negative values indicate backward searchs. This order is similar to the motion vector parameters in MDegrain3. The output clip is intended to be used directly in MDegrainN. Single motion vector clips can be extracted with a SelectEvery(delta*2, n).

So may be if it is required to use different MDegrainN with different tr-settings (for lower denoise and better details saving for example at one of the processing stream) - may be it is possible to extract required 'mv-frames' from single MAnalyse and reuse at different MDegrainN with different tr-settings ? With old stable 2.7.45 version from pinterf. It will make better performance of script because of AVS caching required mv-frames from single MAnalyse output to different MDegrainN.

At least before tr>6 the MDegrainX may be used as 'low-tr' degrain with extracted bv/fv mv-frames using SelectEvery ?

[anton_foy]

Originally Posted by DTL

For better MVs search in MAnalyse it may be recommended for 'slow/slowest/placebo' mode search=3, searchparam>2 and pelsearch>2, trymany=true. And may be some manual play with all penalties (non-zero values).

Thank you I will experiment with these params.

It will make better performance of script because of AVS caching required mv-frames from single MAnalyse output to different MDegrainN.

At least before tr>6 the MDegrainX may be used as 'low-tr' degrain with extracted bv/fv mv-frames using SelectEvery ?

Sounds great but I do not know how to script that to my function correctly, could you show me an example maybe? I am tired now and maybe I can understand better tomorrow.

[coolgit]

Nice one. The photo was a good example, the face doesn't look plasticity and the winkle lines are still there. How about a photo example of a building?

[anton_foy]

Originally Posted by coolgit

Nice one. The photo was a good example, the face doesn't look plasticity and the winkle lines are still there. How about a photo example of a building?

Thanks! Yes I actually have one old building with an uneven facade thats a bit noisy. Will do.

[DTL]

This idea:

tr = 6 # Temporal radius
super = MSuper ()
multi_vec = MAnalyse (super, multi=true, delta=tr)
big_tr=MDegrainN(super, multi_vec, tr, thSAD=400, thSAD2=150)

mv_bw1=SelectEvery(tr*2, 1) // or may be correct start from 0 ?
mv_fw1=SelectEvery(tr*2, 2)
mv_bw2=SelectEvery(tr*2, 3)
mv_fw2=SelectEvery(tr*2, 4)
mv_bw3=SelectEvery(tr*2, 5)
mv_fw4=SelectEvery(tr*2, 6)

mdgn3=MDegrain3(super, mvbw=mv_bw1, mvfw=mv_fw1,  mvbw2=mv_bw2, mvfw2=mv_fw2, mvbw3=mv_bw3, mvfw3=mv_fw3, ...)

Also may be mv_bw1...mv_fwN may be interleaved to feed other MDegrainN with different tr so only MDegrainN may be used with any tr up to max supported. The tr-converting of multi-MAnalyse output may be assembled into some function to use.

[anton_foy]

Originally Posted by DTL

This idea:

tr = 6 # Temporal radius
super = MSuper ()
multi_vec = MAnalyse (super, multi=true, delta=tr)
big_tr=MDegrainN(super, multi_vec, tr, thSAD=400, thSAD2=150)

mv_bw1=SelectEvery(tr*2, 1) // or may be correct start from 0 ?
mv_fw1=SelectEvery(tr*2, 2)
mv_bw2=SelectEvery(tr*2, 3)
mv_fw2=SelectEvery(tr*2, 4)
mv_bw3=SelectEvery(tr*2, 5)
mv_fw4=SelectEvery(tr*2, 6)

mdgn3=MDegrain3(super, mvbw=mv_bw1, mvfw=mv_fw1,  mvbw2=mv_bw2, mvfw2=mv_fw2, mvbw3=mv_bw3, mvfw3=mv_fw3, ...)

Also may be mv_bw1...mv_fwN may be interleaved to feed other MDegrainN with different tr so only MDegrainN may be used with any tr up to max supported. The tr-converting of multi-MAnalyse output may be assembled into some function to use.

You mean like this if using mdegrain2?

function clay(clip Input, clip "preclip", clip "superclip", clip "darkclip", clip "brightclip", int "DthSAD", int "DthSADC", int "BthSAD", int "BthSADC", int "TR", int "BLKSize", int "Rb", int "Overlap", int "Pel", bool "Chroma",
\ float "VDTh", int "Shrp", int "Rad", float "thr", bool "tm", float "Falloff", int "UV", int "lo", int "hi", float "mid", int "mix", bool "fast")
{
DthSAD = default(DthSAD,  150)
DthSADC= default(DthSADC, 150)
BthSAD = default(BthSAD,   70)
BthSADC= default(BthSADC,  70)
TR = default(TR1, 6)
#TR2 = default(TR2, 2) #not needed for mdg2
BLKSize = default(BLKSize, 32)
Overlap = default(Overlap, 16)
Rb = default(Rb, 8)
Pel = default(Pel, 1)
Chroma = default(Chroma, true)
Tm = default(Tm, false)
Falloff = default(Falloff, 0.9)
VDth = default(VDth, 1.5)
lo = default(lo,     60)
mid = default(mid, 0.57)
hi = default(hi,    127)
mix = default(mix,  230)
UV = default(UV, 3)
Fast = default(Fast, false)
Bt= BitLShift(1,input.BitsPerComponent - 8)

pre = defined(preclip)
inputP = pre ? preclip : input

sup = defined(superclip)
inputS = sup ? superclip : input

drk = defined(darkclip)
inputD = drk ? darkclip : input

brt = defined(brightclip)
inputB = brt ? brightclip : input

super = MSuper(inputS, pel=pel, hpad = 0, vpad = 0, chroma=chroma, mt=false, levels=1, sharp=1)
superB = MSuper(inputS.merge(input), pel=pel, hpad = 0, vpad = 0, chroma=chroma, mt=false, levels=1, sharp=1)
superfilt = MSuper(inputP, pel=pel, hpad = 0, vpad = 0, mt=false,sharp=1)
Multi_Vector = Superfilt.MAnalyse(Multi=True, Delta=tr, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma, truemotion=tm ,global=true)
vmulti2 = Mrecalculate(superfilt,multi_vector,thsad=Dthsad,truemotion=false,tr=tr,blksize=rb,overlap=rb/2,mt=false,chroma=true)

dark=darkclip.MDegrainN(Super, vmulti2, tr, thSAD=DthSAD, thSAD2=Int(DthSAD*Falloff), thSADC=DthSADC, thSADC2=Int(DthSADC*Falloff)).neo_vd(VDth,1,6,90)

mv_bw1=Multi_Vector.SelectEvery(tr*2, 1) // or 0?
mv_fw1=Multi_Vector.SelectEvery(tr*2, 2)
mv_bw2=Multi_Vector.SelectEvery(tr*2, 3)
mv_fw2=Multi_Vector.SelectEvery(tr*2, 4)

bright = Fast ? dark.merge(input, 0.7) : brightclip.MDegrain2(SuperB, mvbw=mv_bw1, mvfw=mv_fw1,  mvbw2=mv_bw2, mvfw2=mv_fw2, mvbw3=mv_bw3, mvfw3=mv_fw3, thSAD=BthSAD, thSAD2=Int(BthSAD*Falloff), thSADC=BthSADC, thSADC2=Int(BthSADC*Falloff))

mask=dark.extractY().levels(lo*bt, mid, hi*bt, 0, mix*bt)
mt_merge(dark, bright, mask, luma=true, u=uv, v=uv)

return last
}​

And also would it maybe be better to use something like:

bright=mdegrain2(...)
dark=bright.deblock(planes="yuv", quant=50).mdegrain5(...)
mask=...
mt_merge(dark,bright,mask...)

Thinking mdg2+mdg5 would equal mdegrain with strength of tr=7? EDIT: No I did not think clearly, tr is still max 5 then.

[StainlessS]

I have not tried your script, but this stuck out a bit for me.

dark=darkclip.MDegrainN(Super, vmulti2, tr, thSAD=DthSAD, thSAD2=Int(DthSAD*Falloff), thSADC=DthSADC, thSADC2=Int(DthSADC*Falloff)).neo_vd(VDth,1,6,90)

darkclip is an optional arg, and is not defaulted anywhere, so could be "UnDefined".

Suggest maybe try see if all works OK with ALL defaulted optional args.

I have not looked for additional problems.

[anton_foy]

Originally Posted by StainlessS

I have not tried your script, but this stuck out a bit for me.

dark=darkclip.MDegrainN(Super, vmulti2, tr, thSAD=DthSAD, thSAD2=Int(DthSAD*Falloff), thSADC=DthSADC, thSADC2=Int(DthSADC*Falloff)).neo_vd(VDth,1,6,90)

darkclip is an optional arg, and is not defaulted anywhere, so could be "UnDefined".

Suggest maybe try see if all works OK with ALL defaulted optional args.

I have not looked for additional problems.

Here it is defined:

function clay(clip Input, clip "preclip", clip "superclip", clip "darkclip", clip "brightclip", int "DthSAD", int "DthSADC", int "BthSAD", int "BthSADC", int "TR1", int "TR2", int "BLKSize", int "Rb", int "Overlap", int "Pel", bool "Chroma",
\ float "VDTh", int "Shrp", int "Rad", float "thr", bool "tm", float "Falloff", int "UV", int "lo", int "hi", float "mid", int "mix", bool "fast")
{
DthSAD = default(DthSAD,  150)
DthSADC= default(DthSADC, 150)
BthSAD = default(BthSAD,   70)
BthSADC= default(BthSADC,  70)
TR1 = default(TR1, 6)
TR2 = default(TR2, 2)
BLKSize = default(BLKSize, 32)
Overlap = default(Overlap, 16)
Rb = default(Rb, 8)
Pel = default(Pel, 1)
Chroma = default(Chroma, true)
Tm = default(Tm, false)
Falloff = default(Falloff, 0.9)
VDth = default(VDth, 1.5)
lo = default(lo,     60)
mid = default(mid, 0.57)
hi = default(hi,    127)
mix = default(mix,  230)
UV = default(UV, 3)
Fast = default(Fast, false)
Bt= BitLShift(1,input.BitsPerComponent - 8)

pre = defined(preclip)
inputP = pre ? preclip : input

sup = defined(superclip)
inputS = sup ? superclip : input

drk = defined(darkclip)
inputD = drk ? darkclip : input

brt = defined(brightclip)
inputB = brt ? brightclip : input

super = MSuper(inputS, pel=pel, hpad = 0, vpad = 0, chroma=chroma, mt=false, levels=1, sharp=1)
superB = MSuper(inputS.merge(input), pel=pel, hpad = 0, vpad = 0, chroma=chroma, mt=false, levels=1, sharp=1)
superfilt = MSuper(inputP, pel=pel, hpad = 0, vpad = 0, mt=false,sharp=1)
Multi_Vector = Superfilt.MAnalyse(Multi=True, Delta=max(tr1, tr2), BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma, truemotion=tm ,global=true)
vmulti2 = Mrecalculate(superfilt,multi_vector,thsad=Dthsad,truemotion=false,tr=max(tr1, tr2),blksize=rb,overlap=rb/2,mt=false,chroma=true)

dark=darkclip.MDegrainN(Super, vmulti2, TR1, thSAD=DthSAD, thSAD2=Int(DthSAD*Falloff), thSADC=DthSADC, thSADC2=Int(DthSADC*Falloff)).neo_vd(VDth,1,6,90)
bright = Fast ? dark.merge(input, 0.7) : brightclip.MDegrainN(SuperB, vmulti2, TR1, thSAD=BthSAD, thSAD2=Int(BthSAD*Falloff), thSADC=BthSADC, thSADC2=Int(BthSADC*Falloff))

mask=dark.extractY().levels(lo*bt, mid, hi*bt, 0, mix*bt)
mt_merge(dark, bright, mask, luma=true, u=uv, v=uv)

return last
}

Not correct?

[StainlessS]

Nope, drk and inputD are defined [EDIT: assigned to] there, darkclip is unchanged, ie potentially Undefined.

Suggest maybe try see if all works OK with ALL defaulted optional args.

Give it a try. [I expect Error Abort]

EDIT: Like (untested)

Colorbars(Pixel_type="YV12").KillAudio
clay() # expect "BANG !!!"

[anton_foy]

Originally Posted by StainlessS

Nope, drk and inputD are defined [EDIT: assigned to] there, darkclip is unchanged, ie potentially Undefined.


Give it a try. [I expect Error Abort]

EDIT: Like (untested)

Colorbars(Pixel_type="YV12").KillAudio
clay()

Oh I see but read my usage example in the first post.

Usage (my own settings):
Code:
convertbits(16)
converttoYUV444()

cd=convertbits(8,dither=-1).convertToYv12()
pre=cd.CCD(13).ex_median("IQMV",Y=3,UV=3).ex_sbr(2)

sup=ex_unsharp(1.77,th=1.1,fc=width(),safe=true,UV=1)

drk=deblock(planes="yuv",quant=50)

Clay(tr1=7,tr2=2,dthsad=160,dthsadc=160,bthsad=70,preclip=pre,superclip=sup,darkclip=drk,brightclip=last,chroma=true)

deblock(planes="UV")
neo_F3KDB(grainY=0,grainC=0,mt=false,output_depth=16,range=30).mergechroma(last)
Convertbits(10,dither=1)
prefetch(4)

[StainlessS]

Your usage examples are a bit irrelevant, you should never, ever, ever, have function with optional args which do not default to something, when not supplied. [ie, are NOT actually optional at all]

EDIT: The error is HERE (darkclip may still be undefined, you try call MDegrainN() on undefined clip arg. [there will be tears before bedtime]

dark=darkclip.MDegrainN(Super, vmulti2, tr, thSAD=DthSAD, thSAD2=Int(DthSAD*Falloff), thSADC=DthSADC, thSADC2=Int(DthSADC*Falloff)).neo_vd(VDth,1,6,90)

EDIT: Try the ALL optionals defaulted test, and keep fixing the function until it does not produce error abort. [I dont know if there are problems additional to the darkclip one]

[Gavino]

You should be using inputD instead of darkclip in the latter part of the script.
The same goes for inputB instead of brightclip.

[StainlessS]

Thanx Gavin, looks like that was the intention, but I did not want to make any assumptions. [All defaulted optionals test still a good idea, perhaps Gavin already tried that].

[anton_foy]

Originally Posted by Gavino

You should be using inputD instead of darkclip in the latter part of the script.
The same goes for inputB instead of brightclip.

Yes of course sorry both of you I was confused and interrupted. Just cannot figure out how my script is working since I copied it straight from my avsi-file into here. Will look and fix. Thanks!

you should never, ever, ever, have function with optional args which do not default to something, when not supplied. [ie, are NOT actually optional at all]

Sure, my intention was to use inputD if defined otherwise Input. Just copied the same lines I always use for prefilter in my other scripts.

[Gavino]

Originally Posted by StainlessS

[All defaulted optionals test still a good idea, perhaps Gavin already tried that].

Your advice is 100% sound, but I hadn't tried the script myself.
It simply seemed the obvious intention, as otherwise inputD and inputB were not used for anything.

[DTL]

"mv_bw1=Multi_Vector.SelectEvery(tr*2, 1)"

Yes - in the first example I forgot to set initial MAnalyse mv_multi clip as source for SelectEvery().

This is working example (and I still not sure about numbering from 0 or 1 for SelectEvery)

tr = 6 # Temporal radius
super = MSuper ()
multi_vec = MAnalyse (super, multi=true, delta=tr)
big_tr=MDegrainN(super, multi_vec, tr, thSAD=400, thSAD2=350)

mv_bw1=SelectEvery(multi_vec, tr*2, 0)
mv_fw1=SelectEvery(multi_vec, tr*2, 1)
mv_bw2=SelectEvery(multi_vec, tr*2, 2)
mv_fw2=SelectEvery(multi_vec, tr*2, 3)
mv_bw3=SelectEvery(multi_vec, tr*2, 4)
mv_fw3=SelectEvery(multi_vec, tr*2, 5)

MDegrain3(super, mv_bw1, mv_fw1, mv_bw2, mv_fw2, mv_bw3, mv_fw3, thSAD=300)

Also it looks MDegrain3 not accept mvclips as named arguments.

[takla]

https://imgsli.com/MTg3MjMz/0/1

Spline64ResizeMT(1920, 1080)
EZdenoise(Chroma=true, out16=true).ConvertBits(10, dither=1)

Downscaled, because 4k with such low quality is extremely pointless.
And personally, I wouldn't denoise the castle clip at all (very little noise).

On the moving water near the castle you can see EZdenoise preserves slightly more detail then Clay.
I used frame 20 for all 3 clips.

[anton_foy]

Originally Posted by takla

https://imgsli.com/MTg3MjMz/0/1

Spline64ResizeMT(1920, 1080)
EZdenoise(Chroma=true, out16=true).ConvertBits(10, dither=1)

Downscaled, because 4k with such low quality is extremely pointless.
And personally, I wouldn't denoise the castle clip at all (very little noise).

On the moving water near the castle you can see EZdenoise preserves slightly more detail then Clay.
I used frame 20 for all 3 clips.

Sorry but how can you compare this downscaled (please compare in full screen, before/after and with your downscaled images) it is extremely low in detail and the denoising on your part is showing alot of noise even still. How can you not see this? Yes the castle clip was denoised a tad too hard just to make the point that with the same settings for all the clips it still works pretty well.

Comparison

[takla]

Originally Posted by anton_foy

Sorry but how can you compare this downscaled (please compare in full screen, before/after and with your downscaled images) it is extremely low in detail and the denoising on your part is showing alot of noise even still. How can you not see this? Yes the castle clip was denoised a tad too hard just to make the point that with the same settings for all the clips it still works pretty well.

Comparison

I'm using a 55" native 4k display, calibrated to 203nits, 100% srgb (with an average deviation of 0.8 on 50 color test patterns) with 2.2 gamma.

I really don't want to know what kind of miscalibrated garbage display you use that overexposes everything.

How can you not see that your footage is bottom of the barrel, bitrate starved garbage? The chromatic aberration on that cameras footage is so bad, I'd straight up throw it into the trash.

Also, if the denoiser is set too high you'll lose too much detail in lossy encodes.