MCTemporalDenoise [v1.4.20 - Update 2010/07/02]

[LaTo]

####################################################################################################################################
###                                                                                                                              ###
###                                   Motion-Compensated Temporal Denoise: MCTemporalDenoise()                                   ###
###                                                                                                                              ###
###                                                     v1.4.20 by "LaTo INV."                                                   ###
###                                                                                                                              ###
###                                                           2 July 2010                                                        ###
###                                                                                                                              ###
####################################################################################################################################
### 
### 
### 
### /!\ Needed filters: MVTools (v2.5.10), MaskTools (v2.0a43),
### ------------------- LSFmod (v1.9), FFT3Dgpu (v0.8.2.7) or FFT3Dfilter (v2.1.1),
###                     TTempsmooth (v0.9.4), RemoveGrain + Repair (v1.0PR),
###                     Deblock (v1.2), Deblock_QED (25.may.2010), DctFilter (v0.0.1.4),
###                     EEDI2 (v0.9.2), SangNom (v1.0beta),
###                     GradFun2DBmod (v1.5), GradFun2db (v1.0), AddGrainC (v1.5).
###
### 
### 
### USAGE: MCTemporalDenoise(i, radius, sigma, twopass, useTTmpSm, limit, limit2, post, chroma, interlaced,
###                          sharp, adapt, strength, SHmode, SHmethod, Slimit, Sovershoot, Tlimit, Tovershoot,
###                          protect, cutoff, threshold, maxdiff,
###                          AA, useEEDI2, reduc, maxd, AAthr, method,
###                          deblock, useQED, quant1, quant2, 
###                          edgeclean, ECrad, ECthr, ECmode,
###                          stabilize, maxr, TTstr,  
###                          enhance, GFthr, AGstr, bias, temp,
###                          bwbh, owoh, blksize, overlap,
###                          GPU, fixband, bt, ncpu, precision,
###                          thSAD, thSAD2, thSCD1, thSCD2,
###                          truemotion, MVglobal, pel, pelsearch, search, searchparam, MVsharp, DCT,
###                          p, settings,
###                          show, screenW, screenH)
###
###
###
### MCTemporalDenoise's Post Processor :
### ------------------------------------
### 
### USAGE: MCTemporalDenoisePP( denoised, source,
###                             sharp, adapt, strength, SHmode, SHmethod, Slimit, Sovershoot, Tlimit, Tovershoot,
###                             protect, cutoff, threshold, maxdiff,
###                             AA, useEEDI2, reduc, maxd, AAthr, method,
###                             deblock, useQED, quant1, quant2,
###                             edgeclean, ECrad, ECthr, ECmode,
###                             stabilize, maxr, TTstr,
###                             enhance, GFthr, AGstr, bias, temp,
###                             chroma, settings )
###
### This is the post-processor used internally in MCTemporalDenoise.
### It can be called externally if you want some specific functionality without denoising.
###
### Parameters & defaults are the same as MCTemporalDenoise. 
### By default, all filters are OFF!
###
### "Sharp" & "Protect" options need 2 inputs:
### EX: source=last
###     denoised=source.FFT3Dfilter()
###     MCTemporalDenoisePP(denoised,source,...)
### The others only need "denoised".
###
####################################################################################################################################

Update 2010.07.02 : MCTemporalDenoise v1.4.20


v1.4.20 :
tweaked some settings
updated QED_Deblock version

v1.4.18 :
fixed anti-aliasing PP (was broken since v1.4.07)
updated MVTools-MaskTools-AddgrainC version

v1.4.16 :
added switch to avoid loading of unused filters
fixed a bug in anti-aliasing part

v1.4.08 :
fixed MOD16 warning when deblock=true

v1.4.07 :
improved masking in PP-stage
cleaned code (cosmetic)

v1.4.00 :
fixed bug with maxdiff=0
fixed speed issue in MCTemporalDenoisePP (+20%)
added "deblock" to MCTemporalDenoisePP
changed "flat" to "enhance"
cleaned and optimized all the code
changed some settings
updated documentation

v1.3.09 :
added MOD4 restriction
updated MVTools version

v1.3.00 :
added a post-denoising stage (see "post")
fixed a bug with fixFFT3D and defined p clip
changed "very high" to be less insane
renamed fixFFT3D to fixband
cleaned some stuff
adapted to LSFmod v1.8

v1.2.02 :
changed default gpu=true to gpu=false (set gpu=true if you want speed!!!)
adapted to LSFmod v1.7

v1.2.00 :
adapted for LSFmod v1.6 & GradFun2DBmod v1.3
changed sharp int->bool (deleted sharp=1)

v1.1.27 :
fixed a bug in MCTemporalDenoisePP with chroma=false
adapted for LSFmod v1.5

v1.1.26:
changed "flat" post-processing to use GradFun2DBmod
changed int->float for AGstr
fixed a small bug in AA

v1.1.24:
fixed a chroma bug with PP

v1.1.23:
adapted script for MvTools v2.3.0 (deleted MT & idx parameters)
changed MCTemporalDenoisePP's defaults (now the same as MCTD)
deleted safe parameter (slow & useless)
changed some minor things

v1.0.03:
fixed a bug in MCTemporalDenoisePP

v1.0.02:
changed defaults according to new LSFmod v1.2

v1.0.01:
fixed a speed issue in MCTemporalDenoisePP (introduced in beta5)
changed documentation about MCTemporalDenoisePP

v1.0.00:
first stable version
changed sharp=2, need LSFmod v1.0 !
fix a bug introduced in beta5

v1.0beta5:
added new sharpening routine & deleted LSF (SHmode,Slimit,Sovershoot,Tlimit,Tovershoot)
changed some code (cosmetic)

v1.0beta4:
changed some descriptions & blksize default (submitted by Didée)
added auto blksize setting (16:HD & 8:SD, if you want speed use 16)

v1.0beta3:
all post-processing filters are now usable as a standalone function (MCTemporalDenoisePP)
added MC-antialiasing (see reduc parameter)

v1.0beta2:
fixed MOD16 restriction (effective only if deblock=true)
small speedup/fix with limit>0 and/or protect=true&maxdiff>0
added warning about GPU & MT in the documentation

v1.0beta1:
changed name Udegrain --> MCTemporalDenoise (to be more specific and avoid confusion with chroma)
added the choice to use MVDegrain instead of MVCompensate+TTempSmooth (since the chroma bug in MVDegrain is fixed)
added twopass option to do the denoising job in two stage (very effective but slow)
added MT parameter for MVTools (be careful +- unstable)
added fixFFT3D parameter to prevent banding/posterize with limit=-1
added stabilize options (stabilize,maxr,TTstr)
added deblock options (deblock,useQED,quant1,quant2)
added new presets "light"(=low)/"heavy"(=medium) --> 5 presets (very high is 'a bit' extrem)
added check/warning for some settings
added thSAD2 & limit2 parameters for twopass=true
added overrange parameter for controling sharpness (useful for sharpening more than the source)
changed +- much code to implement twopass/MVdegrain/MT, should be bug free (I hope)
changed sharp->MVsharp & post->sharp
changed "light Smode4" --> LimitedSharpenFaster for sharp=2 (strength,Smode,soft,overshoot)
reduce max radius to 6
optimize chroma in PP stage (+20% with all PP on)
fix a bug with external p clip
fix a bug in Star&Bright Point Protection with maxdiff>0
added YV12&MOD16 restriction
changed some settings due to new presets
restriction when using MT=true: bt=1 in FFT3Dgpu & plane=4 in MVDegrainMulti (will be removed when fixed)

v0.9.05a:
changed version number: vX.XX branch -> v0.X.XX branch
added AA options (AA,useEEDI2,maxd,AAthr,method)
changed dering --> edgeclean: you can now use your own script (see docs)
fix bug with show=true
fix bug with chroma in PP stage
fix omission with chroma=false in MVanalyse
changed some settings
added changelog in .avsi

v0.9.01a:
added adapt & SS for sharpening
added dering option (dering, DRGrad, DRGthr, RGmode)
added flat areas processing (flat, AGstr, GFthr, bias)
added modX(=blocksize) checking
changed some default settings
updated documentation in .avsi

v0.8.08c:
fix bug with chroma=false & interlaced=true

v0.8.08b:
added mocomp contra-sharpening (basic contra-sharpening always usable)

v0.7.29a:
added star & bright points protection
changed some default options
added more options for FFT3D
added YV12 checking

v0.7.25a:
set chroma=false for light settings
added temporal limiter for sharpening

v0.7.24a:
replace blankclip by nop

v0.7.23a:
first public release

[LaTo]

Speed comparison: (with blocksize=8 & overlap=4)

MCTemporalDenoise(settings="very low")  : 9.4fps
MCTemporalDenoise(settings="low")       : 4.8fps
MCTemporalDenoise(settings="medium")    : 3.4fps
MCTemporalDenoise(settings="high")      : 2.6fps
MCTemporalDenoise(settings="very high") : 0.5fps

GTDegrain()       @default              : 4.1fps
MC_Spuds()        @default              : 3.6fps
TemporalDegrain() @default              : 2.8fps

Comparison on "normal" movie : (v1.0.00)

Source

MCTemporalDenoise(settings="very low")

MCTemporalDenoise(settings="low")

MCTemporalDenoise(settings="medium")



Comparison on "300" : (v1.0.00)

Source

MCTemporalDenoise(settings="medium")

MCTemporalDenoise(settings="high")

MCTemporalDenoise(settings="very high")


Feedback please

Wow! beats the crap out of mine, and gives Didée's a run for it's money!

So let me get this straight. You are getting speed improvement by not doing a second degrain stage. So you've compensated the decreased degraining by either turning off truemotion and repairing the compensated frames (light mode) or not post sharpening (heavy mode). Am I reading the script correctly? If so, these are great ideas!

Some thoughts: might want to put some more options in to do the second degrain stage (I know it will slow it down, just don't make it the default)

Also, and this is more a question than anything, isn't "NOP()" faster than "BlankClip"? Seems like it should be, but I don't know for sure. It also seems like it would use less memory, since if a variable isn't needed, it isn't stored as a blank clip. Probably nothing, but might be worth looking into.

One more question. Is the Mask really necessary? isn't everything taken care of by the THSAD in MVCompensate already? I could see this if you weren't repairing after the degrain, but since you are (in the last part of your "Degrain" section) choosing either the degrained output or the prefilter depending on which is closest to the source, you really shouldn't need the mask.

These are just thoughts off the top of my head, and I love this contribution you have made!

-G

[LaTo]

Originally Posted by g-force

So let me get this straight. You are getting speed improvement by not doing a second degrain stage. So you've compensated the decreased degraining by either turning off truemotion and repairing the compensated frames (light mode) or not post sharpening (heavy mode). Am I reading the script correctly? If so, these are great ideas!

Yes, it's correct.

Originally Posted by g-force

Some thoughts: might want to put some more options in to do the second degrain stage (I know it will slow it down, just don't make it the default)

Maybe, if some people request this... (because it's enough for my use)

Originally Posted by g-force

Also, and this is more a question than anything, isn't "NOP()" faster than "BlankClip"? Seems like it should be, but I don't know for sure. It also seems like it would use less memory, since if a variable isn't needed, it isn't stored as a blank clip. Probably nothing, but might be worth looking into.

I don't know, some other people could help for this... (I will test tomorrow)

Originally Posted by g-force

One more question. Is the Mask really necessary? isn't everything taken care of by the THSAD in MVCompensate already? I could see this if you weren't repairing after the degrain, but since you are (in the last part of your "Degrain" section) choosing either the degrained output or the prefilter depending on which is closest to the source, you really shouldn't need the mask.

Yes but repair with prefilter is off by default because it make some artefacts on my stuff.


Let me know if you have any suggestions

[Gavino]

Originally Posted by g-force

Also, and this is more a question than anything, isn't "NOP()" faster than "BlankClip"? Seems like it should be, but I don't know for sure. It also seems like it would use less memory, since if a variable isn't needed, it isn't stored as a blank clip.

Originally Posted by LaTo

I don't know, some other people could help for this.

g-force is correct. Each use of BlankClip will use up memory.
You could assign a single instance of BlankClip to a variable and use that instead, but g-force's suggestion of NOP is the best solution. It also has the merit that if there is something wrong with your logic and the variable ends up getting used as a clip, you will get an error message.

Originally Posted by LaTo

Yes but repair with prefilter is off by default because it make some artefacts on my stuff.

Oh, just noticed that. I hate to put the burden back on you, because it really should be up to me to see if I can break your script, but I'd be curious to see an example of the artifacts that the "choose between prefilter or degrain according to whichever is closest to the source" that your approach helps. I'm not saying that you're wrong, I'm just surprised and curious, that's all.

Okay, another suggestion: I just noticed that you have "bt" set to 1 on the fft filter. Just to warn you, Didée's gonna come down hard on you for this one. Might want to put an option in to allow switching this to the other temporal settings as well. This will help with low-frequency flickering due to heavy grain. I only know this by the experience of hounding by Didée. Plus, it seems that changing "bt" to 5 doesn't slow down the filter much.

Yet another suggestion: how about adding a temporal based limiter for the sharpening? Like this:

pshrp = last
pmax  = source.MT_Logic(b1c,"max").MT_Logic(f1c,"max")
pmin  = source.MT_Logic(b1c,"min").MT_Logic(f1c,"min")
RemoveGrain(3,-1)
MT_MakeDiff(last,last.RemoveGrain(12,-1))
MT_lutxy(pshrp,last,"y 128 - 1.5 * x +",U=2,V=2)
MT_Clamp(pmax,pmin,1,1,U=2,V=2)

That's all for now... Nice work!

-G

[LaTo]

Originally Posted by g-force

Oh, just noticed that. I hate to put the burden back on you, because it really should be up to me to see if I can break your script, but I'd be curious to see an example of the artifacts that the "choose between prefilter or degrain according to whichever is closest to the source" that your approach helps. I'm not saying that you're wrong, I'm just surprised and curious, that's all.

it's appear with high sigma value for fft3d, it's like a flat solid color (because of heavy filtering)... I will make screenshot later. (the best is to use another clip for this job and not the prefilter clip)

Originally Posted by g-force

Okay, another suggestion: I just noticed that you have "bt" set to 1 on the fft filter. Just to warn you, Didée's gonna come down hard on you for this one. Might want to put an option in to allow switching this to the other temporal settings as well. This will help with low-frequency flickering due to heavy grain. I only know this by the experience of hounding by Didée. Plus, it seems that changing "bt" to 5 doesn't slow down the filter much.

So, enable it by default for you... As I said, for me it's enough...

Originally Posted by g-force

Yet another suggestion: how about adding a temporal based limiter for the sharpening? Like this:

pshrp = last
pmax  = source.MT_Logic(b1c,"max").MT_Logic(f1c,"max")
pmin  = source.MT_Logic(b1c,"min").MT_Logic(f1c,"min")
RemoveGrain(3,-1)
MT_MakeDiff(last,last.RemoveGrain(12,-1))
MT_lutxy(pshrp,last,"y 128 - 1.5 * x +",U=2,V=2)
MT_Clamp(pmax,pmin,1,1,U=2,V=2)

Why not...

[LaTo]

Originally Posted by Gavino

g-force is correct. Each use of BlankClip will use up memory.
You could assign a single instance of BlankClip to a variable and use that instead, but g-force's suggestion of NOP is the best solution. It also has the merit that if there is something wrong with your logic and the variable ends up getting used as a clip, you will get an error message.

Thanks

[Didée]

Originally Posted by g-force

Didée's gonna come down hard on you for this one.

Not at all. Right now I'm leaving for holiday -- Bye!

Originally Posted by LaTo

(the best is to use another clip for this job and not the prefilter clip)

I'd love to see that screenshot, but I agree that that heavy of a prefilter is not the best choice to use as the output sometimes.

-G

[LaTo]

Originally Posted by g-force

I'd love to see that screenshot

Here with high sigma value:

Limit = -1
Limit = 0

[Amrit]

WOW

this is amazing, I might start denoising a little bit more now

check this out

Before
http://latoninf.free.fr/udegrain/movie_SRC.png
After (Light)
http://latoninf.free.fr/udegrain/movie_UDlight.png

SOME details are lost but compared to other denoisers this has preserved more than them.

CHEERS!!

[Ranguvar]

The only complaint I have is that eyes are messed with... When there's a very small white point in someone's eye? With chroma on, it seems to turn blue. With chroma off, it swells.

Otherwise, nice work

[ficofico]

My eyes can see only lost of detail, the above source for me it's quite perfect, no need to use any filter

[LaTo]

Update 2008.07.25

7.25a:
set chroma=false for light settings
added temporal limiter for sharpening

[Ranguvar]

Some notes for anyone trying to multithread this script with SetMTMode(). This is current as of v7.25a. LaTo, feel free to add this to the OP.


Call UDegrain from your script in mode 2.

If GPU=true (which is default), you must SetMTMode(4) before the PREFILTER block. You can SetMTMode(2) after it. Note that this is after I modified bt=1 in the fft3dgpu line to bt=3; if bt=1 you may have no problems running without mode 4. I don't know.

Under the DEGRAIN block, if limit==-1 (which is default), the lines:

DD  = (limit==-1) ? mt_lutxy(pD,smD,"x 128 - abs y 128 - abs < x y ?",U=3,V=3)
\   : (limit>0)   ? mt_lut(smD,"x 128 "+string(limit)+" - < 128 "+string(limit)+" - x 128 "+string(limit)+" + > 128 "+string(limit)+" + x ? ?",U=3,V=3)
\   : NOP()

must run in mode 3. You can go back to mode 2 after.

Two further notes... Some lines might run in mode 1, which uses even less RAM but should be the same speed as mode 2. I have plenty of RAM, so I haven't tested that. Also, I did not test very many features of UDegrain, some might require additional tuning... I only used light settings, radius=1, chroma, and GPU.

[martino]

Originally Posted by Ranguvar

The only complaint I have is that eyes are messed with... When there's a very small white point in someone's eye?

I think that's just how MVDegrain works. Try to apply it on some anime with scenes from space and you'll see most of the stars gone.

[Ranguvar]

Originally Posted by martino

I think that's just how MVDegrain works. Try to apply it on some anime with scenes from space and you'll see most of the stars gone.

MC_Spuds has a parameter called starfield that tries to preserve such bright points, though... adding something similar to UDegrain would be fantastic

[martino]

It shouldn't be hard, since what it basically does is create a mask (plus that you'd have to hunt down the other functions that it uses from the script and the required variables). I don't particularly remember why, but when I was writing a function of my own it didn't work as I expected it to have, so I ended up doing a crappy one for my own need. I wish I could remember a bit more about it now...

FUNCTION Starmask(clip input, bool anime, clip nullclp, float thstar, int lumathres)
...
maskstar = ConditionalFilter(maskstarvalidate, nullclp.mt_binarize(upper=false), maskstar.mt_inflate().mt_inflate, "AverageLuma()", "<", string(thStar),false)

[LaTo]

Originally Posted by Ranguvar

MC_Spuds has a parameter called starfield that tries to preserve such bright points, though... adding something similar to UDegrain would be fantastic

Maybe in the next version