How to use a prefiltered clip for MDegrain2 motion estimation

[cy]

Hi,

Could someone please show me how to modify my script, in order to use a pre-filtered clip to feed MDegrain2?
I've read this would give more accurate motion vectors and thus more accurate de-noising/de-graining?

How "clean" must the pre-filtered clip be for improved results with MDegrain2 ?

My current script:


SetMTMode (5)

# Set DAR in encoder to 163 : 86. The following line is for automatic signalling
global MeGUI_darx = 163
global MeGUI_dary = 86

DGDecode_mpeg2source("Y:\MOVIES\84 Charing Cross Road (1987) PAL DVD\mkv\VTS_01_1.d2v", info=3)
ColorMatrix(hints=true, threads=0)

SetMTMode (2)

super = MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)

crop( 0, 14, -8, -14)

[Didée]

DGDecode_mpeg2source("... .d2v", info=3)
ColorMatrix(hints=true, threads=0)
src = last

SetMTMode (2)

preNR = src.FFT3DFilter( with_appropriate_settings ) # as an example

preNR_super = preNR.MSuper(pel=2, sharp=1)
src_super = src.MSuper(pel=2, sharp=1, levels=1)

backward_vec2 = MAnalyse(preNR_super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(preNR_super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(preNR_super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(preNR_super, isb = false, delta = 2, overlap=4)

src.MDegrain2(src_super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)
The prefiltering should be just strong enough to eliminate all noise.

[cy]

Thank you Didée !

[Blue_MiSfit]

Didee:

In your example of using fft3dfilter - I might worry a bit about banding at high sigma values. Would banding goof up the motion estimation? My guess is probably a lot less than big chunks of dirt and grain :D

~MiSfit

[Sagekilla]

It actually doesn't hurt ME. the function I made out of his degraining routine for 300 (TemporalDegrain) uses a absurdly high sigma values (sigma=16, sigma2=12, sigma3=10, sigma4=4) but it doesn't hurt ME at all. Only problem where it might hurt is in the limiting portion where banding might seep through, since the 1st degrain stage is limited against what FFT3DFilter did. And FFT3DFilter denoised -a lot-

[Nephilis]

DGDecode_mpeg2source("... .d2v", info=3)
ColorMatrix(hints=true, threads=0)
src = last

SetMTMode (2)

preNR = src.FFT3DFilter( with_appropriate_settings ) # as an example

preNR_super = preNR.MSuper(pel=2, sharp=1)
src_super = src.MSuper(pel=2, sharp=1, levels=1)

backward_vec2 = MAnalyse(preNR_super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(preNR_super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(preNR_super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(preNR_super, isb = false, delta = 2, overlap=4)

src.MDegrain2(src_super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)
The prefiltering should be just strong enough to eliminate all noise.

i think this script with fft3d is too strong and removes many details, especially for not much dirty sources..Isn't it?

[Didée]

The procedure with prefiltering-for-ME is often suited for sources with strong grain/noise. For sources with only low noise, prefiltering-for-ME is mostly not needed.

So, yes, a sledge hammer is not well suited to stick a pushpin into a cork board. However it is well suited to drive stakes into earth.

It is expected that users are aware of the difference between pushpin-into-cork and stake-into-earth. Answering each'n every question beginning with Adam & Eve is too time intensive.

[rkalwaitis]

Didee, If the source isnt that noisey could you use MVflow.

I read about this under MVTools.

It says MVFlow produces no artifacts, like other methods of denoising. Which confuses me because right after the MVFlow portion of the little script they denoise again in another way, DeGrainMedian().

super = MSuper()
backward_vectors = MAnalyse(super, isb = true)
forward_vectors = MAnalyse(super, isb = false)
forward_compensation = MFlow(super, forward_vectors, thSCD1=400) # or use MCompensate
backward_compensation = MFlow(super, backward_vectors, thSCD1=400)
# create interleaved 3 frames sequences
interleave(forward_compensation, last, backward_compensation)

DeGrainMedian() # place your preferred temporal (spatial-temporal) denoiser here

selectevery(3,1)

as always I wait for the wisdom of Didee.

[Didée]

Well, that's Off Topic. Answer-in-pictures:

Source:
http://thumbnails12.imagebam.com/3030/842cd930293722.gif (http://www.imagebam.com/image/842cd930293722)

MCompensate:
http://thumbnails16.imagebam.com/3030/3ae27430293723.gif (http://www.imagebam.com/image/3ae27430293723)

MFlow:
http://thumbnails11.imagebam.com/3030/6bb02030293726.gif (http://www.imagebam.com/image/6bb02030293726)

[rkalwaitis]

Didee,

My test of it didnt have electricity shooting out of peoples hands, Im jealous. :)

how did you use it?

MCompensate gave me lots of blocks

[Sagekilla]

rkalwaitis: There's only two denoising functions in MVTools, MDegrain and MDenoise. MCompensate is meant for creating intermediary frames to help temporal denoisers. They're not meant to be used standalone without using a denoiser.

[rkalwaitis]

Perhaps that is why Didee thought I was off base. I was thinking that Mflow was a type of denoising by itself. But now I understand that it has to be used in conjuction with a denoiser. The documentation for MVTools2 stated that the MvFlow motion compensation does not produce any artifacts. It said also it is recommended to use true motion estimation (is strongly suggested)

Of course the documentation only gave me the example below.

AVISource("c:\test.avi") # or MPEG2Source, DirectShowSource, some previous filter, etc
super = MSuper()
vectors = MAnalyse(super, isb = false)
compensation = MCompensate(super,vectors) # or use MFlow function here
# prepare blurred frame with some strong blur or deblock function:
blurred = compensation.DeBlock(quant=51) # Use DeBlock function here
badmask = MMask(vectors, kind = 2, ml=50)
overlay(compensation,blurred,mask=badmask) # or use faster MaskedMerge function of MaskTools


But this is compenstation not estimation, right? Im not even sure how I would do a similar script for estimation.

[Sagekilla]

MCompensate and MFlow both create intermediary frames suitable for denoising. Their method of doing so differs slightly: MCompensate uses block based compensation, and MFlow uses pixel based compensation. Because of how they work, you can sometimes get blocking in MCompensate (which can screw up some temporal averaging schemes) but you won't in MFlow.

And yes, compensation is a form of estimation. You're creating a frame, X-0.5, in between X-1 and X, but to do so you need to estimate how everything in the frame moved from X-1 to X.

[rkalwaitis]

So if I understand correctly it would possibly be more beneficial to use Mflow. Of course with a proper denoiser to help avoid artifacts? Im gonna see which is the best type of denoiser to use. Makes sense to start with the MDegrains.

k

[Didée]

The reason why I thought you were off base is: this thread is about how to use pre-filtering for getting a more stable vector field. Kind of an advanced question. Then you hop in with a question about how all of this mo-comp denoising stuff is functioning at all. That's much less advanced, and doesn't fit the topic.

Best way is to try all methods, and judge the results by yourself. The coarse crack-down is this: for denoising purposes, utilizing MFlow will introduce (much) more potential problems than benefit. Look again at the small pics above. That's a "naked" MCompensate vs. a "naked" MFlow. It's obvious which one has more artifacts ... and that's not only a selected case example, but the general characteristic. When building a denoiser that (internally) uses motion compensation, I'd rather chose that method out of two available that produces less artifacs.

The wording "MFlow is artifact free" is highly misleading ... probably it's a relict of the times when block overlapping wasn't yet available. Back then, the normal compensation methods were prone to produce very blocky results, which the "flow" method did not. Nowadays with block overlapping available, the raw compensation quality of the blockbased methods is superior to the flow method.


@ Sagekilla: "compensation is a form of estimation" - that's not a good statement.

a) motion estimation: the procedure of searching motion vectors.

b) motion compensation: the act of shifting a block to a new location, by 100% of the length of its attached motion vector.

c) motion interpolation: the act of shifting a block to a new location, by only a fraction of the length of its attached motion vector.

Step a) has always to be done. Step b) is for fully undoing the motion from one frame to another, so that temporal filters can act safely. Step c) is for partly undoing inter-frame motion, i.e. creating a new intermediate step of motion, mostly for framerate conversion or motion blur effects.

[Fizick]

I never said, that "MVFlow produces no artifacts".
From documentation: "any blocking artefactes"

[rkalwaitis]

Sorry Fizick, you are absolutely correct. Excuse me.

[rkalwaitis]

So if the goal is to always do motion estimation followed by motion compensation then, of course one may need prefiltering to enhance the deonising.

So a suitable script for this may be........

src=last
shrp=awarpsharp()
src_super =src.Msuper()
shrp_super=shrp.Msuper(levels=1)

forward_vectors = MAnalyse(src_super, isb = false)
backward_vectors =Manalyse(src_super,isb=true)
forward_compensation = MCompensate(src_super, forward_vectors, thSCD1=350)
backward_compensation = MCompensate(src_super, backward_vectors, thSCD1=350)
interleave(forward_Compensation, last, backward_compensation)
DeGrainMedian(limitY=2,limitUV=3,mode=2).DeGrainMedian(limitY=2,limitUV=3,mode=2)
SelectEvery(3,1)

Anyone have any tips. I tried to sharpen the image before MSuper in the hopes of keeping a sharper image. Of course Im sure that sharpening prior to does not necessarily count as prefiltering :)

[Sagekilla]

note aWarpSharp isn't a traditional sharpener! It doesn't work like LimitedSharpenFaster does, so you may get very odd results.

Also, if you read what Didee said, prefiltering is only necessary when there's excessive noise. Unless your video looks like 300, you probably don't need to use prefiltering. If you want to do that sharpening trick Didee posted earlier, you need to use MDegrain, which means you can't use the MCompensate -> Interleave -> DegrainMedian -> SelectEvery method.

[Jawed]

This is the function I developed a while back:

function Killer(clip source, int temporal, int "blksize", int "overlap", int "sharp", int "thSAD", bool "RefineMotion")
{ # Motion compensated denoiser for progressive source clip with prefiltering for strength and repair for de-artefacting
# Uses MVTools2, ReduceFlicker and RemoveGrain/Repair

blksize = default(blksize,16) # blksize value (4, 8 or 16)
overlap = default(overlap,blksize/2) # overlap value (0 to half blksize)
sharp = default(sharp,2) # 0=bilinear softest, 1=bicubic, 2=Wiener sharpest
thSAD = default(thSAD,300) # higher risks motion ghosting and swimming, lower risks blotchy denoising
RefineMotion = default(RefineMotion,true) # true means MRecalculate will be used to improve motion vectors

halfblksize = blksize/2 # MRecalculate works with half block size
halfoverlap = overlap/2 # Halve the overlap to suit the halved block size
halfthSAD = thSAD/2 # MRecalculate uses a more strict thSAD, which defaults to 150 (half of function's default of 300)

dct=5

source = source.assumeframebased() # MSuper pel=2 is faster with this

# Prefilter the clip
calm = source.reduceflicker(strength=3,aggressive=true)
calm = calm.repair(source,mode=1)
calm = calm.removegrain(mode = 17)
calm = calm.removegrain(mode = 17)
calm = calm.removegrain(mode = 17)

calm_super = calm.MSuper(pel=2, hpad=blksize, vpad=blksize, sharp=sharp)
source_super = source.MSuper(pel=2, hpad=blksize, vpad=blksize, sharp=sharp,levels=1)
recalculate = calm.MSuper(pel=2, hpad=blksize, vpad=blksize, sharp=sharp,levels=1)

backward_vec3 = MAnalyse(calm_super, blksize=blksize, isb = true, delta = 3, overlap=overlap, dct=dct)
backward_vec3 = RefineMotion ? MRecalculate(recalculate, backward_vec3, blksize=halfblksize, overlap=halfoverlap, thSAD=halfthSAD) : backward_vec3
backward_vec2 = MAnalyse(calm_super, blksize=blksize, isb = true, delta = 2, overlap=overlap, dct=dct)
backward_vec2 = RefineMotion ? MRecalculate(recalculate, backward_vec2, blksize=halfblksize, overlap=halfoverlap, thSAD=halfthSAD) : backward_vec2
backward_vec1 = MAnalyse(calm_super, blksize=blksize, isb = true, delta = 1, overlap=overlap, dct=dct)
backward_vec1 = RefineMotion ? MRecalculate(recalculate, backward_vec1, blksize=halfblksize, overlap=halfoverlap, thSAD=halfthSAD) : backward_vec1

forward_vec1 = MAnalyse(calm_super, blksize=blksize, isb = false, delta = 1, overlap=overlap, dct=dct)
forward_vec1 = RefineMotion ? MRecalculate(recalculate, forward_vec1, blksize=halfblksize, overlap=halfoverlap, thSAD=halfthSAD) : forward_vec1
forward_vec2 = MAnalyse(calm_super, blksize=blksize, isb = false, delta = 2, overlap=overlap, dct=dct)
forward_vec2 = RefineMotion ? MRecalculate(recalculate, forward_vec2, blksize=halfblksize, overlap=halfoverlap, thSAD=halfthSAD) : forward_vec2
forward_vec3 = MAnalyse(calm_super, blksize=blksize, isb = false, delta = 3, overlap=overlap, dct=dct)
forward_vec3 = RefineMotion ? MRecalculate(recalculate, forward_vec3, blksize=halfblksize, overlap=halfoverlap, thSAD=halfthSAD) : forward_vec3

temporal == 3 ? MDegrain3(source, source_super, backward_vec1, forward_vec1, backward_vec2, forward_vec2, backward_vec3, forward_vec3, thSAD=thSAD) : \
temporal == 2 ? MDegrain2(source, source_super, backward_vec1, forward_vec1, backward_vec2, forward_vec2, thSAD=thSAD) : \
MDegrain1(source, source_super, backward_vec1, forward_vec1, thSAD=thSAD)
repair(source,mode=17)
}
It's brutally strong hence thSAD defaulting to 300. You may prefer lower :)

To use it, just do:

Killer(3)

for 3 frames backward and 3 frames forward, or 2 or 1 if you prefer to use less frames.

I prefer to keep it at 3 and tune thSAD between 50 and 300, e.g.:

Killer(3,thSAD=150)

It's pretty slow, too.

Oh and it'll prolly break if you use blksize=4 and RefineMotion=true (the default). Can't remember if I tested that, since blksize=4 is ridiculously slow.

There's a FizzKiller version too. Maybe I should update my FizzKiller thread.

Jawed

[Didée]

Hey ... isn't this the first function posted to public that actually uses MRecalculate? Congrats! ;)

However on thing,
It's brutally strong hence thSAD defaulting to 300. You may prefer lower :)
with my boring little stress test, I could not find that it's "brutally" strong. Seemed more like a halfway failure...

Since the deficiency isn't in the parameters but in the basic methodology, I couldn't let my fingers off and twiddled two small tweaks into that Killer() function. Just one "#" to deactivate a line, and a little sharpen for the source superclip. Nothing spectacular.

> Result sample (http://www.mediafire.com/?hmgq1nmxoz1) < (Mediafire, 18 MB)

Is it legit to kill the Killer? :)

[Jawed]

MRecalculate's nice because with blksize=16 specified it's very similar to blksize 8 but with performance that's somewhat faster.

I suspect you deactivated the final repair. You can try a compromise with mode=1 instead of 17.

I use playback sharpening.

With clips like your test clip (which isn't as noisy as my worst test clips :eek:) I use FizzKiller, which doesn't have the final repair and has a gamma manipulation that makes your blood boil.

Jawed

[Sagekilla]

I had hoped to see some neat new function utilizing MVRecalculate that would provide a jump in quality. But, it seems like it does very little to help most of the time :(

[markanini]

DGDecode_mpeg2source("... .d2v", info=3)
ColorMatrix(hints=true, threads=0)
src = last

SetMTMode (2)

preNR = src.FFT3DFilter( with_appropriate_settings ) # as an example

preNR_super = preNR.MSuper(pel=2, sharp=1)
src_super = src.MSuper(pel=2, sharp=1, levels=1)

backward_vec2 = MAnalyse(preNR_super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(preNR_super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(preNR_super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(preNR_super, isb = false, delta = 2, overlap=4)

src.MDegrain2(src_super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)
The prefiltering should be just strong enough to eliminate all noise.

This script works great for me with FFT3DGPU as prefilter whereas TemporalDegrain can't take a change of the pel parameter without breaking. Seems faster as well but it may be my imagination....

Here's my modified version I use just for real-time viewing on sub-SD material(youtube etc). It steals many defualt setting from TemporalDegrain :devil::
src = last
preNR = src.FFT3dGPu(sigma=16,sigma2=10,sigma3=6,sigma4=4,bw=16,bh=16)
preNR_super = preNR.MSuper(pel=1, sharp=2)
src_super = src.MSuper(pel=1, sharp=2, levels=1)
backward_vec2 = MAnalyse(preNR_super, isb = true, delta = 2, overlap=4,blksize=16)
backward_vec1 = MAnalyse(preNR_super, isb = true, delta = 1, overlap=4,blksize=16)
forward_vec1 = MAnalyse(preNR_super, isb = false, delta = 1, overlap=4,blksize=16)
forward_vec2 = MAnalyse(preNR_super, isb = false, delta = 2, overlap=4,blksize=16)
src.MDegrain2(src_super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=200)


FWIW

[videoFred]

I'm using RemoveDirtMC for prefiltering.. works very fine on very grainy source.

Fred.

[Nephilis]

If anyone interesting in an advanced RemoveDirtMC, so here is mine :) Thanx to Kassandro and as always Didée

Look at post no #31 (http://forum.doom9.org/showthread.php?p=1479622#post1479622)

[Didée]

Looking at that script, I assume that the results for radius = 1 / 2 / 3 should be pixel-identical. RemoveDirt is working with a temporal radius of 1 frame, nothing else. The additional MCompensate()'s might add to the script's complexity, but they're a NOP for the actual filtering.

[Nephilis]

But "Clense" function in RemoveDirt takes previous, current and the subsequent frame into consideration, doesn't it? (ForwardClense and BackwardClense, too)
And SCSelect function in RemoveDirt takes more than a frame into account, too..
What's your opinion?

[Didée]

I tried this ...

source_clip
v1 = RemoveDirtMC_SE(radius=1)
v3 = RemoveDirtMC_SE(radius=3)
diff=mt_makediff(v1,v3,U=3,V=3)

... and analysed the "diff" clip. It contains only one value, Y=128, from 1st to last frame. There is no difference. radius=2/3 is identical to radius=1.

q.e.d. - theory proven by practice.

[Didée]

Moreover, there's a bug in the script. The function "RemoveDirt" is called two times, in two different contexts. However, within the RemoveDirt function, you're using Clense with a fixed cache slot:

Clense(input, cache=4)

Which means that the two different instances of RemoveDirt use the same recursion-clip for Clense's recursive operation.

Obviously, this is not correct.

[Nephilis]

I tried this ...

source_clip
v1 = RemoveDirtMC_SE(radius=1)
v3 = RemoveDirtMC_SE(radius=3)
diff=mt_makediff(v1,v3,U=3,V=3)

... and analysed the "diff" clip. It contains only one value, Y=128, from 1st to last frame. There is no difference. radius=2/3 is identical to radius=1.

q.e.d. - theory proven by practice.

Moreover, there's a bug in the script. The function "RemoveDirt" is called two times, in two different contexts. However, within the RemoveDirt function, you're using Clense with a fixed cache slot:

Clense(input, cache=4)

Which means that the two different instances of RemoveDirt use the same recursion-clip for Clense's recursive operation.

Obviously, this is not correct.

Here, removed fixed cache values and redundant compensate stages.

########################################################################################################################
### ###
### RemoveDirtMC ###
### ###
### **SPECIAL EDITION** ###
### By NEPHILIS ###
### @2011 ###
### ###
########################################################################################################################

##########################################################
## ##
## Requirements : - MVTools (v2.5.11.1) ##
## ----------------- - Masktools (v2.0a48) ##
## - RemoveGrainT (v1.0) package ##
## - RemoveDirt (v0.9) package ##
## - AvsRecursion (v0.2) ##
## - FluxSmooth (v1.1) ##
## ##
##########################################################

########################################################################################################################
# #
# Usage : RemoveDirtMC_SE(clip input, int"radius", int"repmode", int"clmode",\ #
# ---------- bool"TwoPassDeNoise", int"thSAD", int"thSADC") #
# #
# #
########################################################################################################################

#################
# ------------- #
# PARAMETERS #
# ------------- #
#################

#######################################################################################################################
#
# ----------
# Radius : -- Temporal Radius for MDeGrain at second stage DeNoising.
# ----------
#
# ----------
# repmode : -- RemoveDirt's "repmode" parameter.
# ----------
#
# ----------
# clmode : -- RemoveGrain's "mode" parameter in RemoveDirt function.
# ----------
#
# ----------
# Twopass
# Denoise : -- To make a Temporal Denoising (MDeGrain) after Spatial Denoising.
# ----------
#
# ----------
# thSAD,
# thSADC : -- MDeGrain's thSAD and thSADC parameters.
# ----------
#
#######################################################################################################################


Function RemoveDirtMC_SE(clip input, int"radius", int"repmode", int"clmode", bool"TwoPassDeNoise", int"thSAD", int"thSADC")

{

radius = default(radius, 2)
repmode = default(repmode, 16)
clmode = default(clmode, 5)
TwoPassDeNoise = default(TwoPassDeNoise, false)
thSAD = default(thSAD , 250)
thSADC = default(thSADC, thSAD)

ox = input.width()
oy = input.height()
HD = (ox== 1920 || oy>= 800) ? true : false
blksize = (HD==true) ? 16:8
search = (HD==true) ? 4:5
overlap = blksize/2

src = input
PreNR = input.RemoveDirt().TemporalDeGrain(5,8)
PreNR_super = MSuper(PreNR,hpad=4,vpad=4,pel=2,sharp=2)
src_super = MSuper(input,hpad=4,vpad=4,pel=2,sharp=2,levels=1)

bwv1 = MAnalyse(PreNR_super, blksize=blksize, search=search, isb=true, delta=1, overlap=overlap, truemotion=true)
bwv2 = (TwoPassDeNoise==true && radius>=2) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=true, delta=2, overlap=overlap, truemotion=true) : BlankClip()
bwv3 = (TwoPassDeNoise==true && radius==3) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=true, delta=3, overlap=overlap, truemotion=true) : BlankClip()
fwv1 = MAnalyse(PreNR_super, blksize=blksize, search=search, isb=false, delta=1, overlap=overlap, truemotion=true)
fwv2 = (TwoPassDeNoise==true && radius>=2) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=false, delta=2, overlap=overlap, truemotion=true) : BlankClip()
fwv3 = (TwoPassDeNoise==true && radius==3) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=false, delta=3, overlap=overlap, truemotion=true) : BlankClip()

b1comp = MCompensate(input,src_super,bwv1)
f1comp = MCompensate(input,src_super,fwv1)

int = interleave(f1comp, input, b1comp)

NR = int.RemoveDirt(repmode=repmode, clmode=clmode)
sum1 = SelectEvery(NR,3,1)
sum1_super = ( TwoPassDeNoise==true ) ? MSuper(sum1,hpad=4,vpad=4,pel=2,sharp=2,levels=1) : BlankClip()
sum2 = ( TwoPassDeNoise==false ) ? sum1
\ : ( radius==1 ) ? MDeGrain1(sum1, sum1_super, bwv1, fwv1, thSAD=thSAD, thSADC=thSADC)
\ : ( radius==2 ) ? MDeGrain2(sum1, sum1_super, bwv1, bwv2, fwv1, fwv2, thSAD=thSAD, thSADC=thSADC)
\ : MDeGrain3(sum1, sum1_super, bwv1, bwv2, bwv3, fwv1, fwv2, fwv3, thSAD=thSAD, thSADC=thSADC)
Return (sum2)

}


function TemporalDeGrain(clip input, int "mode", int "thr")

{
rg = RemoveGrain(input, mode=mode)
tr = TemporalRepair(rg, input)
FluxSmoothT(tr,thr)

}


function RemoveDirt(clip input, int "repmode", int "clmode")

{

repmode = default(repmode, 16)
clmode = default(clmode, 5)
clensed = Clense(input)
sbegin = ForwardClense(input)
send = BackwardClense(input)
alt = Repair(SCSelect(input, sbegin, send, clensed, debug=false), input, mode=repmode)
restore = Repair(clensed, input, mode=repmode)
corrected = RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=50, dist=1,\
dmode=2, debug=false, noise=8, noisy=12)
RemoveGrain(corrected, mode=clmode)

}

[Didée]

To bring theory to a full circle:

Actually, there is one case where the additional compensations potentially can make a difference: it's around scenechanges, because SCselect - as you noted correctly - considers more than +/-1 temporal neighbors.

(And indeed .... checking with another test sequence, I found 1 occasion where a slight difference comes up: in the frame right before an unrelated one-frame-flash.)

But then, it's not guaranteed that the difference is a "good one" - it could be even worse. SCselect is quite robust without motion comensation. When it's faced with an interleaved compensated sequence, then the problem is that MCompensate (per default) uses no scenechange detection. Therefore MCompensate will try to match as good as possible across scenechanges (i.e. make the compensation from the not-correlated neighbor scene as similar as possible to the current reference frame). And therefore, chances are that motion compensation rather hurts SCselect's scenechange detection, instead of helping.

[Nephilis]

.. And therefore, chances are that motion compensation rather hurts SCselect's scenechange detection, instead of helping.

Does help adding MSCDetection function before the compensation stage in the script to fix this problem?

[Didée]

MSCDetection is "only" producing a B/W mask for custom usage. On it's own, it does nothing.

From the guts, my trust in SCselect (without any mocomp stuff fiddled in) is bigger than in MVTools' scenechange detection. If e.g. one (very dark scene with few features) changes to another (very dark scene with few features), MVTools will fail because in last instances it uses static thresholds. SCSelect acts dynamically, adapting it's detection to what "currently is really happening."

[Didée]

Instead of this:

[...]

int = (radius==1) ? interleave( f1comp, input, b1comp )
\ : (radius==2) ? interleave( f2comp, f1comp, input, b1comp, b2comp )
\ : interleave(f1comp, f2comp, f1comp, input, b1comp, b2comp, b3comp)

NR = int.RemoveDirt(repmode=repmode, clmode=clmode)
sum1 = SelectEvery(NR,1+radius*2,radius)

[...]

function RemoveDirt(clip input, int "repmode", int "clmode")
{
repmode = default(repmode, 16)
clmode = default(clmode, 5)
clensed = Clense(input, cache=4)
sbegin = ForwardClense(input, cache=-1)
send = BackwardClense(input, cache=-1)
alt = Repair(SCSelect(input, sbegin, send, clensed, debug=false), input, mode=repmode)
restore = Repair(clensed, input, mode=repmode)
corrected = RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=50, dist=1,\
dmode=2, debug=false, noise=8, noisy=12)
RemoveGrain(corrected, mode=clmode)
}

... I'd try to make it work more or less like this:

# int = (radius==1) ? interleave( f1comp, input, b1comp )
# \ : (radius==2) ? interleave( f2comp, f1comp, input, b1comp, b2comp )
# \ : interleave(f1comp, f2comp, f1comp, input, b1comp, b2comp, b3comp)

NR = input.RemoveDirtX(f1comp,b1comp, repmode=repmode, clmode=clmode)
sum1 = NR

[...]

function RemoveDirtX(clip input, clip fcomp, clip bcomp, int "repmode", int "clmode")
{
repmode = default(repmode, 16)
clmode = default(clmode, 5)
clensed = interleave(fcomp,input,bcomp).Clense(cache=-1).selectevery(3,1)
sbegin = ForwardClense(input, cache=-1)
send = BackwardClense(input, cache=-1)
alt = Repair(SCSelect(input, sbegin, send, clensed, debug=false), input, mode=repmode)
restore = Repair(clensed, input, mode=repmode)
corrected = RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=50, dist=1,\
dmode=2, debug=false, noise=8, noisy=12)
RemoveGrain(corrected, mode=clmode)
}

I'm not completely sure ATM, because I forgot the rationale behind "ForwardClense"/"BackwardClense", and can't find it quickly now. Clense is a median filter, but you can't calculate a median from only two elements. Therefore I don't see right now what these two special-clenses are actually doing.

[Nephilis]

From RemoveGrain document:
Beginning with version 0.9 there are also the filters BackwardClense and ForwardClense. They are primarily for clensing at sharp scene changes in the RemoveDirt script function. ForwardClense is for the first frame of a new scene and BackwardClense is for the last frame of a scene. The usage is the same as Clense.

[Didée]

Yes, that I can read, too. It explains exactly nothing.

[Nephilis]

Hi, here is a new version of RemoveDirtMC..

by..

########################################################################################################################
### ###
### RemoveDirtMC ###
### v1.1 ###
### **SPECIAL EDITION** ###
### By NEPHILIS ###
### @2011 ###
### ###
########################################################################################################################

###########################################################
## ##
## Requirements : - MVTools (v2.5.11.0) ##
## ------------- - Masktools (v2.0a48) ##
## - RemoveGrainT (v1.0) package ##
## - RemoveDirt (v0.9) package ##
## - AvsRecursion (v0.2) ##
## - FluxSmooth (v1.1) ##
## - NLMeansCL package ##
## (AvsFilterNet.dll, Cloo.dll, ##
## NLMeansCL_netautoload.dll) ##
## - NVIDIA GeForce 8 series or newer VGA ##
## ##
###########################################################

########################################################################################################################
# #
# Usage : RemoveDirtMC_SE(clip input,int"radius",int"repmode",int"clmode",bool"TwoPass",int"thSAD",int"thSADC" ) #
# ------ #
# #
########################################################################################################################

#################
# ------------- #
# PARAMETERS #
# ------------- #
#################

#######################################################################################################################
#
# ----------
# Radius : -- Temporal Radius for MDeGrain at second stage DeNoising.
# ----------
#
# ----------
# repmode : -- Repair mode value in main RemoveDirtSE function.
# ----------
#
# ----------
# clmode : -- RemoveGrain mode value in main RemoveDirtSE function.
# ----------
#
# ----------
# Twopass : -- To make a Temporal Denoising (MDeGrain) after Spatial Denoising (RemoveDirtSE).
# ----------
#
# ----------
# thSAD,
# thSADC : -- MDeGrain's thSAD and thSADC parameters.
# ----------
#
#######################################################################################################################

Function RemoveDirtMC_SE(clip input, int"radius", int"repmode", int"clmode", bool"TwoPass", int"thSAD", int"thSADC")

{

version = "v1.1"

radius = default(radius, 2)
repmode = default(repmode, 16)
clmode = default(clmode, 5)
TwoPass = default(TwoPass, true)
thSAD = default(thSAD , 150)
thSADC = default(thSADC, thSAD)

ox = input.width()
oy = input.height()
HD = (ox== 1920 || oy>= 800) ? true : false
blksize = (HD==true) ? 16:8
search = (HD==true) ? 4:5
overlap = blksize/2

src = input
PreNR = src.NLMeansCL(h=radius*1.5,plane=4).TemporalDeGrain()
PreNR_super = MSuper(PreNR,hpad=4,vpad=4,pel=2,sharp=2)
src_super = MSuper(src ,hpad=4,vpad=4,pel=2,sharp=2,levels=1)

bwv1 = MAnalyse(PreNR_super, blksize=blksize, search=search, isb=true, delta=1, overlap=overlap, truemotion=true)
bwv2 = (radius>=2) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=true, delta=2, overlap=overlap, truemotion=true) : BlankClip()
bwv3 = (radius==3) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=true, delta=3, overlap=overlap, truemotion=true) : BlankClip()
fwv1 = MAnalyse(PreNR_super, blksize=blksize, search=search, isb=false, delta=1, overlap=overlap, truemotion=true)
fwv2 = (radius>=2) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=false, delta=2, overlap=overlap, truemotion=true) : BlankClip()
fwv3 = (radius==3) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=false, delta=3, overlap=overlap, truemotion=true) : BlankClip()

bcomp = MCompensate(src,src_super,bwv1)
fcomp = MCompensate(src,src_super,fwv1)

NR1 = src.RemoveDirtSE(fcomp, bcomp, repmode=repmode, clmode=clmode)
NR1_super = ( TwoPass==true ) ? MSuper(NR1,hpad=4,vpad=4,pel=2,sharp=2,levels=1) : BlankClip()
NR2 = ( TwoPass==false ) ? NR1
\ : ( radius==1 ) ? MDeGrain1(NR1, NR1_super, bwv1, fwv1, thSAD=thSAD, thSADC=thSADC)
\ : ( radius==2 ) ? MDeGrain2(NR1, NR1_super, bwv1, bwv2, fwv1, fwv2, thSAD=thSAD, thSADC=thSADC)
\ : MDeGrain3(NR1, NR1_super, bwv1, bwv2, bwv3, fwv1, fwv2, fwv3, thSAD=thSAD, thSADC=thSADC)
Return (NR2)

}

function TemporalDeGrain(clip input, int "mode", int "thr")

{
mode = default(mode, 5)
thr = default(thr, 8)
rg = RemoveGrain(input, mode=mode)
tr = TemporalRepair(rg, input)
fs = FluxSmoothT(tr,thr)
Return(fs)

}

function RemoveDirtSE(clip input, clip fcomp, clip bcomp, int "repmode", int "clmode")

{

repmode = default(repmode, 16)
clmode = default(clmode, 5)
clensed = interleave(fcomp,input,bcomp).Clense(cache=-1).selectevery(3,1)
sbegin = ForwardClense(input, cache=-1)
send = BackwardClense(input, cache=-1)
alt = Repair(SCSelect(input, sbegin, send, clensed, debug=false), input, mode=repmode)
restore = Repair(clensed, input, mode=repmode)
corrected = RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=50, dist=1,\
dmode=2, debug=false, noise=8, noisy=12)
postNR = RemoveGrain(corrected, mode=clmode)
Return(postNR)

}

[GRKNGLR]

Good work Nephilis. :thanks:

[Gerry62]

Hi, here is a new version of RemoveDirtMC..


Thanks for posting The script Nephilis, Just tested it on some shonky tatty 16mm Kine's, And they are vvveerryy Grainy, script works too well in fact and I need to do some tinkering,
This sample explains the kind of grain I mean!
http://www.megaupload.com/?d=BA654JB8

The point of my post is the subtle
NLMeansCL package
in ver1.1 of your Mighty script.

Now im only learning, but I cannot see any reference to where I would use this NLMeansCL in your new version, Id love to, as when I did some tests with NLmeansCl, I doubled my speed, 1-2fps to 5fps. So its a MASSIVE improvement in what I'm normally processing, How would I use the CL function with Ver1.1?

Excuse me if this sounds stupid, I do read all the posts and spend a couple of hours trying all the 'switches' to see what happens, I'm not one of those who wants people to do his work, Then Id learn nothing, and what's the fun in that.
although saying that, its probably staring me in the eye!

[Nephilis]

Here is a new version of RemoveDirtMC_SE for both NVIDIA and ATI GPU owners..

#########################################################################################################################
### ###
### RemoveDirtMC ###
### v1.2 ###
### **SPECIAL EDITION** ###
### By NEPHILIS ###
### @2011 ###
### ###
#########################################################################################################################

##################################################################
## ##
## -------------- ##
## Requirements : - MVTools (v2.5.11.2) ##
## -------------- - Masktools (v2.0a48) ##
## - RemoveGrainT (v1.0) package ##
## - RemoveDirt (v0.9) package ##
## - AvsRecursion (v0.2) ##
## - FluxSmooth (v1.1) ##
## ##
## As Additional ##
## -------- ##
## => To Prefiltering with FFT3DFilter ##
## - FFT3DFilter (v2.1.1) ##
## - FFTW3.dll ##
## -------- ##
## => To Prefiltering with NLMeansCL ##
## - NLMeansCL package ##
## (AvsFilterNet.dll, Cloo.dll, ##
## NLMeansCL_netautoload.dll) ##
## ##
##################################################################

##########################################################################################################################
# ------- #
# Usage : #
# ------- #
# #
# RemoveDirtMC_SE(clip input, bool"GPU", int"radius", int"repmode", int"clmode", bool"TwoPass", int"thSAD", int"thSADC") #
# #
##########################################################################################################################

##################
# ----------- #
# PARAMETERS : #
# ----------- #
##################

##########################################################################################################################
#
# ----------
# GPU : -- To choose NLMeansCL or FFT3Dfilter for PreFiltering process.
# ---------- ATI GPU owners should use "GPU=false"
#
# ----------
# Radius : -- Temporal Radius for MDeGrain at second stage DeNoising.
# ----------
#
# ----------
# repmode : -- Repair mode value in main RemoveDirtSE function.
# ----------
#
# ----------
# clmode : -- RemoveGrain mode value in main RemoveDirtSE function.
# ----------
#
# ----------
# Twopass : -- To make a Temporal Denoising (MDeGrain) after Spatial Denoising (RemoveDirtSE).
# ----------
#
# ----------
# thSAD,
# thSADC : -- MDeGrain's thSAD and thSADC parameters.
# ----------
#
##########################################################################################################################

Function RemoveDirtMC_SE(clip input, bool"GPU", int"radius", int"repmode", int"clmode", bool"TwoPass", int"thSAD", int"thSADC")

{

version = "v1.2"

GPU = default(GPU, true)
radius = default(radius, 2)
repmode = default(repmode, 16)
clmode = default(clmode, 5)
TwoPass = default(TwoPass, true)
thSAD = default(thSAD , 150)
thSADC = default(thSADC, thSAD)

ox = input.width()
oy = input.height()
HD = (ox== 1920 || oy>= 800) ? true : false
blksize = (HD==true) ? 16:8
search = (HD==true) ? 4:5
overlap = blksize/2

src = input
PreNR = (GPU==true) ? src.NLMeansCL(h=radius*1.5,plane=4).TemporalDeGrain(5,9) \
: src.FFT3DFilter(sigma=radius*2,plane=4,bw=16,bh=16,ow=8,oh=8,bt=3,wintype=1).TemporalDeGrain(5,9)
PreNR_super = MSuper(PreNR,hpad=4,vpad=4,pel=2,sharp=2)
src_super = MSuper(src ,hpad=4,vpad=4,pel=2,sharp=2,levels=1)

bwv1 = MAnalyse(PreNR_super, blksize=blksize, search=search, isb=true, delta=1, overlap=overlap, truemotion=true)
bwv2 = (radius>=2) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=true, delta=2, overlap=overlap, truemotion=true) : BlankClip()
bwv3 = (radius==3) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=true, delta=3, overlap=overlap, truemotion=true) : BlankClip()
fwv1 = MAnalyse(PreNR_super, blksize=blksize, search=search, isb=false, delta=1, overlap=overlap, truemotion=true)
fwv2 = (radius>=2) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=false, delta=2, overlap=overlap, truemotion=true) : BlankClip()
fwv3 = (radius==3) ? \
MAnalyse(PreNR_super, blksize=blksize, search=search, isb=false, delta=3, overlap=overlap, truemotion=true) : BlankClip()

bcomp = MCompensate(src,src_super,bwv1)
fcomp = MCompensate(src,src_super,fwv1)

NR1 = src.RemoveDirtSE(fcomp, bcomp, repmode=repmode, clmode=clmode)
NR1_super = ( TwoPass==true ) ? MSuper(NR1,hpad=4,vpad=4,pel=2,sharp=2,levels=1) : BlankClip()
NR2 = ( TwoPass==false ) ? NR1
\ : ( radius==1 ) ? MDeGrain1(NR1, NR1_super, bwv1, fwv1, thSAD=thSAD, thSADC=thSADC)
\ : ( radius==2 ) ? MDeGrain2(NR1, NR1_super, bwv1, bwv2, fwv1, fwv2, thSAD=thSAD, thSADC=thSADC)
\ : MDeGrain3(NR1, NR1_super, bwv1, bwv2, bwv3, fwv1, fwv2, fwv3, thSAD=thSAD, thSADC=thSADC)
Return(NR2)

}

function TemporalDeGrain(clip input, int "mode", int "thr")

{
mode = default(mode, 5)
thr = default(thr, 8)
rg = RemoveGrain(input, mode=mode)
tr = TemporalRepair(rg, input)
fs = FluxSmoothT(tr,thr)
Return(fs)

}

function RemoveDirtSE(clip input, clip fcomp, clip bcomp, int "repmode", int "clmode")

{

repmode = default(repmode, 16)
clmode = default(clmode, 5)
clensed = interleave(fcomp,input,bcomp).Clense(cache=-1).selectevery(3,1)
sbegin = ForwardClense(input, cache=-1)
send = BackwardClense(input, cache=-1)
alt = Repair(SCSelect(input, sbegin, send, clensed, debug=false), input, mode=repmode)
restore = Repair(clensed, input, mode=repmode)
corrected = RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=50, \
dist=1, dmode=2, debug=false, noise=8, noisy=12)
postNR = RemoveGrain(corrected, mode=clmode)
Return(postNR)

}

[Gerry62]

Nice!
Thanks for the GPU explanation, I did find it myself, but only after reading a 50odd page thread on one of the early GPU-CL type scripts.

Ill let you know how it works on a Gigabyte 8500
Ok finished some tests

System
Athlon 64 x2 3600+ 2Gb ram - WinXP x64 - 8500GT - Latest Nvidia x64 drivers - Last updated with MS update 5 years ago. (this system never goes online) Made sure all My Dll's matched the Version Info in 'Requirements' and thanks for that bit, makes it very easy to make sure a rouge DLL isn't to blame

With GPU=TRUE 1.03fps
With GPU=FALSE 1.36fps

Ran the last version with default settings and just GPU=False & True, and got pretty much the same 'speed', Still Its a script that shows promise with the very poor material I work with. and 1.36fps is what I get when I use another script that I like!

Just to make sure it wasn't a fft3dgpu error I ran same material that and got 26fps.

I might try replacing FFT3DFilter with the GPU version and see what happens, Will have a bash tomorrow


a quick question!
HD = (ox== 1920 || oy>= 800) ? true : false
blksize = (HD==true) ? 16:8
search = (HD==true) ? 4:5

Im confused about that little bit!
Is it for just HD material, or is it the the equivalent to
block_size= 16
block_size_v= 16
block_over= 8
Found in another script (Freds), Sorry for being a pain, Am relatively new to Synth.
Plus Im one of those 'pixel squeezers' I want the best quality for what im doing, Time is immaterial, It could run at 0.33 (one script I use does) and I coudent care, As long as it's the best one can get.

[szabi]

Hi

I did try to use "RemoveDirtMC_SE".
The latest I found is v1.56 on pastebin: click (https://pastebin.com/uNUbMQEh)
RemoveDirtMC_SE(gpu=false)
First gpu had to be disabled anyway it did give failure.
After I got the following: There is no function named 'TemporalRepair' in line 109 and 163.
Wiki does not have any info: click (http://avisynth.nl/index.php/RemoveGrain_v0.9/Repair/TemporalRepair)
Could u help me how to make RemoveDirtMC_SE to work?

Regards
szabi

[StainlessS]

http://avisynth.nl/index.php/RemoveGrain_v1.0b/Repair/TemporalRepair

EDIT:
for future reference, I googled "avisynth 'TemporalRepair"

[szabi]

I linked wiki already: RemoveGrain_v0.9/Repair/TemporalRepair (http://avisynth.nl/index.php/RemoveGrain_v0.9/Repair/TemporalRepair)
What you linked: RemoveGrain_v1.0b/Repair/TemporalRepair (http://avisynth.nl/index.php/RemoveGrain_v1.0b/Repair/TemporalRepair)
The version number is higher however both site have nothing to download.

[poisondeathray]

I linked wiki already: RemoveGrain_v0.9/Repair/TemporalRepair (http://avisynth.nl/index.php/RemoveGrain_v0.9/Repair/TemporalRepair)
What you linked: RemoveGrain_v1.0b/Repair/TemporalRepair (http://avisynth.nl/index.php/RemoveGrain_v1.0b/Repair/TemporalRepair)
The version number is higher however both site have nothing to download.


The download links are on the main page for RemoveGrain
http://avisynth.nl/index.php/RemoveGrain

Those older versions are linked from the Wayback Machine

[szabi]

I did the same mistake as few days ago (https://forum.doom9.org/showpost.php?p=1881593&postcount=743).
The following files are in the zip:
DenoiseSharpen.dll
RemoveGrain.dll
RemoveGrainS.dll
RemoveGrainSSE2.dll
RemoveGrainSSE3.dll
Repair.dll
RepairS.dll
RepairSSE2.dll
RepairSSE3.dll
RSharpen.dll
RSharpenS.dll
RSharpenSSE2.dll
RSharpenSSE3.dll
I copied only versions of SSE3. Seems it was not enough.
Nowadays I get confused several times what *.dll need to be used.
Thanks.

Regards

[pinterf]

http://avisynth.nl/index.php/RemoveGrain_v1.0b/Repair/TemporalRepair

EDIT:
for future reference, I googled "avisynth 'TemporalRepair"
Oh, I completely forgotten about that I have a pending task to release a new RgTools which includes "TemporalRepair".

EDIT: RgTools 0.98 released. Don't forget to delete old DLLs which are also containing TemporalRepair (of yourse, only when you are not using other filters from them - most of them are implemented in RgTools)

[StainlessS]

Thanks muchly P, (I did take alook at RgTools first, but saw not mention of it), then did the google search on TemporalRepair, sorry for the bum steer Szabi :)