BASIQ Denoiser

[anton_foy]

Basic denoising most suitable for noisy DSLR/Camcorder footage (SLog2/SLog3 or other similar Log-formats) but can
also be used for other material.


### BASIQ DENOISER (by anton_foy) ###########################################################################
# Required: Mvtools2, neo_vaguedenoiser, AwarpSharp2, GradePack(Dogway), masktools2, #
# ApplyGradationCurves.avsi, GradationCurve, fastblur, neo_f3kdb, Extools, Avstp, #
# SMDegrain(Dogway) #
# #
### USAGE ###################################################################################################
# Choose Preset for denoising strength: Low / Medium / High #
# #
# Individual Parameters: #
# Adjust STR for strength of denoising in overall and especially in detailed areas. #
# Adjust BLK for Blocksize. #
# Adjust THRESH for luma and chroma threshold of denoising in noisy/lower detailed areas. #
# Adjust NSTEPS for the number of times the wavelet will decompose the picture. #
# Adjust PRC for partial of full denoising. #
# Adjust SHARP and WARP for sharpening/detail enhancement. #
# Adjust LIMIT lower for more smoothing overall or higher for less smoothing in brighter areas. #
# Adjust OPAC lower values gives an overall softer denoised image. #
# FMT = Multithreading for FastDegrain. #
# #
#############################################################################################################

function BASIQ(clip c, string "Preset", int "STR", int "BLK", float "THRESH", int "NSTEPS", float "SHARP",
\ float "PRC", int "WARP", float "LIMIT", int "OPAC", bool "FMT")
{

# Select presets:
Preset = default( Preset, "Medium" )
pNum = (Preset == "Low" ) ? 0 : \
(Preset == "Medium" ) ? 1 : \
(Preset == "High" ) ? 2 : \
Assert( FALSE , "'Preset' choice is invalid" )


# Preset defaults: Low Medium High
STR = default( STR, Select( pNum, 1 , 2 , 3 ) )
BLK = default( BLK, Select( pNum, 8 , 8 , 8 ) )
THRESH = default( THRESH, Select( pNum, 1, 2 , 2.2 ) )
NSTEPS = default( NSTEPS, Select( pNum, 2 , 4 , 6 ) )
PRC = default( PRC, Select( pNum, 76.0, 82.0, 94.0 ) )
SHARP = default( SHARP, Select( pNum, 0.16, 0.20, 0.24 ) )
WARP = default( WARP, Select( pNum, 4 , 5 , 6 ) )
LIMIT = default( LIMIT, Select( pNum, 1.0, 0.92, 0.60 ) )
OPAC = default( OPAC, Select( pNum, 240, 200, 100 ) )
FMT = default( FMT, Select( pNum, FALSE, FALSE, FALSE ) )

#fine denoise
f = c.VsCnr2("ooo",100,16,191,100,255,32,255,sceneChroma=false)
f = f.FastDegrain_k(degrain=str,blksize=blk,mt=fmt)
f = f.awarpsharp2(depth=warp)
f = f.sharpen(sharp)

#soft denoise
s = f.vdmc(THRESH=THRESH,NSTEPS=NSTEPS,PRC=PRC).ex_vibrance(1.06)

#finemask
fmask = c.coloryuv(autogain=true).levels(0,(LIMIT),255*256,0,255*256).
\ ApplyGradationCurves(lumaPoints="18,0, 54,20, 85,130, 112,216, 163,255, 255,255").grayscale().
\ mt_expand().fastblur(0.4).levels(0,1.0,255*256,0,(OPAC*256))

#merge
mt_merge(f,s,fmask.mt_invert(),Y=3,U=3,V=3, luma=true).neo_f3kdb(grainY=44,grainC=3)

return last
}

# FastDegrain - ex_KNLMEANSCL MOD #

function FastDegrain_k( clip src, int "degrain", int "blksize", int "overlap", int "pel",
\ int "thSAD", int "limit", bool "TrueM", bool "MT")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, blksize/2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)
TrueM = default( TrueM, false) # Enable/disable truemotion
MT = default( MT, false) # Internal multithreading (through avstp.dll)

# Create our Super clip for MV search. Then, search for motion vectors.
super = src.MSuper(pel=pel)
super8 = src.convertbits(8).ex_KNLMeansCL(D=0, A=1, h=6, s=4, device_type="gpu").MSuper(pel=pel)
bvec3 = (degrain>=3) ? super8.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP()
bvec2 = (degrain>=2) ? super8.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP()
bvec1 = (degrain>=1) ? super8.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP()
fvec1 = (degrain>=1) ? super8.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP()
fvec2 = (degrain>=2) ? super8.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP()
fvec3 = (degrain>=3) ? super8.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, thSAD=thSAD, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=limit) : src

return src
}

# VDMC - mocomped neo_vaguedenoiser #

function VDMC(clip input, int "tradius", int "mthresh", int "method", float "thresh", float "prc",
\ int "nsteps", int "blocksize",clip "auxclip", bool "pref", int "Y", int "UV", int "sY", int "sU", int "sV")
{

pref = Default(pref, true)
Y = Default(Y, 3)
UV = Default(UV, 3)
sY = Default(sY, 3)
sU = Default(sU, 3)
sV = Default(sV, 3)

t=Defined(tradius)
tradius=t ? tradius : 4
# temporal radius-number of frames analyzed before/after current frame.

m=Defined(mthresh)
mthresh=m ? mthresh : 180
# motion threshold-higher numbers denoise areas with higher motion.
#Anything above this number does not get denoised.

s=Defined(thresh)
thresh=s ? thresh : 4.0
#Threshold- Denoise pixels that match in surrounding frames.
#255 is the maximum and default. 0-255 are valid numbers.

n=Defined(nsteps)
nsteps=n ? nsteps : 6
# Number of times, the wavelet will decompose the picture.

m=Defined(method)
method=m ? method : 1
# Method

p=Defined(prc)
prc=p ? prc : 94.0
# Partial of full denoising.

b=Defined(blocksize)
blocksize=b ? blocksize : 16
#larger numbers = faster processing times

chroma = UV == 3
aux=Defined(auxclip)

w = width(input)
h = height(input)
isUHD = (w > 2599 || h > 1499)
nw = round(w/2.0)
nh = round(h/2.0)
inputA = input
inputA = aux ? auxclip : isUHD ? inputA.ConvertBits(8,dither=-1).BilinearResize(nw+nw%2, nh+nh%2) : inputA

super = MSuper(input, pel=1, hpad = 0, vpad = 0, chroma=true, mt=true, levels=1)
superfilt = MSuper(inputA,pel=1, hpad = 0, vpad = 0, chroma=true, mt=true) # bug can't disable chroma otherwise luma isn't processed

vmulti = Manalyse(superfilt,multi=true,delta=tradius,temporal=true,truemotion=true,blksize=blocksize,overlap=blocksize/2, mt=true, chroma=true)
vmulti2 = Mrecalculate(superfilt,vmulti,thsad=mthresh,truemotion=true,tr=tradius,blksize=blocksize/2,overlap=blocksize/4, mt=true, chroma=true)
vmulti2 = isUHD ? vmulti2.MScaleVect() : vmulti2
mocomp = Mcompensate(input,super,vmulti2,thsad=mthresh,tr=tradius,center=true,mt=true) # recursion=50 is bugged
dnmc = mocomp.neo_vd(threshold=thresh, method=method, nsteps=nsteps, percent=prc, Y=sY, U=sU, V=sV)
dec = selectevery(dnmc,tradius * 2 + 1,tradius)
Y != 3 ? input.mergechroma(dec) : dec
}

[StainlessS]

# Select presets:
Preset = default( Preset, "Medium" )
pNum = (Preset == "Low" ) ? 0 : \
(Preset == "Medium" ) ? 1 : \
(Preset == "High" ) ? 2 : \
Assert( pNum < 2, "'Preset' choice is invalid" )

Above, If Preset != 0,1,2, then In Assert(pNum, Has Not been Assigned, = bug [EDIT: Test, BASIQ(Preset="fart") ]
Also, maybe "pNum < 2" should have been "pNum <= 2", but maybe changing anyway.

Maybe

# Select presets:
Preset = default( Preset, "Medium" )
pNum = (Preset == "Low" ) ? 0 : \
(Preset == "Medium" ) ? 1 : \
(Preset == "High" ) ? 2 : \
-1

Assert( 0 <= pNum <= 2 , "'Preset' choice is invalid" )


NOTE, "0 <= pNum <= 2" same as "0 <= pNum && pNum <= 2".

None of above tested.

EDIT: You could have also used below

# Select presets:
Preset = default( Preset, "Medium" )
pNum = (Preset == "Low" ) ? 0 : \
(Preset == "Medium" ) ? 1 : \
(Preset == "High" ) ? 2 : \
Assert( FALSE , "'Preset' choice is invalid" )

Assert(False, ... throws error ALWAYS.

[anton_foy]

# Select presets:
Preset = default( Preset, "Medium" )
pNum = (Preset == "Low" ) ? 0 : \
(Preset == "Medium" ) ? 1 : \
(Preset == "High" ) ? 2 : \
Assert( pNum < 2, "'Preset' choice is invalid" )

Above, If Preset != 0,1,2, then In Assert(pNum, Has Not been Assigned, = bug [EDIT: Test, BASIQ(Preset="fart") ]
Also, maybe "pNum < 2" should have been "pNum <= 2", but maybe changing anyway.

Maybe

# Select presets:
Preset = default( Preset, "Medium" )
pNum = (Preset == "Low" ) ? 0 : \
(Preset == "Medium" ) ? 1 : \
(Preset == "High" ) ? 2 : \
-1

Assert( 0 <= pNum <= 2 , "'Preset' choice is invalid" )


NOTE, "0 <= pNum <= 2" same as "0 <= pNum && pNum <= 2".

None of above tested.

EDIT: You could have also used below

# Select presets:
Preset = default( Preset, "Medium" )
pNum = (Preset == "Low" ) ? 0 : \
(Preset == "Medium" ) ? 1 : \
(Preset == "High" ) ? 2 : \
Assert( FALSE , "'Preset' choice is invalid" )

Assert(False, ... throws error ALWAYS.

Thank you StainlessS I will try your suggestions out and change in the coming day and if you or anyone else has any advice on how to speed up the script or make it better somehow Im all ears!

[anton_foy]

I have now updated BASIQ for better default values and better temporal stability by replacing FFT3DFILTER with a Mocomped NEO_VAGUEDENOISER and gained speed and quality by replacing KNLMEANSCL with MCDEGRAIN.
As for the mask the ApplyGradationCurves seems slow and I think to replace it with an identical curve made with Expr but I don't seem to find much documentation or examples on how to make that, if someone has any links or ideas please share. Also I think coloryuv(autogain=true) seems a bit slow and I hope to also be able to replace this with something faster but with identical results. The mask only need to use the luma and maybe even go down to 8-bit on the mask would speed it up a bit without losing quality ( or even half res? ).

If anyone has any idea how to speed up this script and improve in any way Im all ears.
Will be posting some before/after images and or clips will be posted further on aswell.
Thanks!

[anton_foy]

Ok I found a way that could speed things up and still get great quality, to replace MCDegrain with FastDegrain.
If I use fastdegrain() alone it works super but when I use FastDegrain in the BASIQ-script I get "Unexpected error" or
SharedMemoryClient: Server got error: SharedMemoryClient: Server got error: Could not allocate video frame. Out of memory.
memory_max = 1073741824, memory_used = 500627959 Request=52439296
Is there a way to get it to work? Thanks.

[kedautinh12]

I think you lack ram memory (maybe)

[anton_foy]

I think you lack ram memory (maybe)

Yes I believed so too but when I watched the RAM usage during this it says 28% in the performance tab in the task manager although CPU went up 100% before I got the error message. Don't understand why I can use the script with McDegrain and not FastDegrain which is extremely much faster and giving almost identical results. I use MP_pipeline, could that be a problem maybe? I can post both scripts later today if needed.

[Boulder]

Running out of VRAM, not RAM on your computer.

[anton_foy]

Running out of VRAM, not RAM on your computer.

Thanks I guess then my only choice is to upgrade my comp.

EDIT: My mistake, I had forgotten to remove a test line in the Basiq-script and now it is working with fastdegrain.