RemoveDirt

[anton_foy]

Strange...I was just watching ""The Fearless Vampire Killers on DVD and a few minutes after I read this topic

I would really like to know what script you applied on these pics, it looks almost too good to be true. Im having the same noise/grain pattern in my footage so I would like to try that.

Thanks!

[Heini011]

Hi anton_foy,

give this hq_filter a try, it is still a bit better ;-)

function RemoveTempGrain(clip input, int _mode)
{
  rg = RemoveGrain(input, mode=_mode)
  return TemporalRepair(rg, input)
}

function RemoveDirt(clip input, int limit, bool "_grey")
{
  _grey=default(_grey, false)
  clensed=input.Clense(grey=_grey, cache=4)
  alt=input.RemoveGrain(2)
  return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=4,cthreshold=6, gmthreshold=40,dist=1,dmode=2,debug=false,noise=limit,noisy=16,grey=_grey)
}

function SRemoveDirtMC(clip,int limit, bool "_grey")
{
  _grey=default(_grey, false)
  limit = default(limit,6)
  bvec = clip.MVAnalyse(isb=false, blksize=8, delta=1, pel=2, truemotion=true, idx=1)
  fvec = clip.MVAnalyse(isb=true, blksize=8, delta=1, pel=2, truemotion=true, idx=1)
  backw = clip.MVFlow(bvec)
  forw  = clip.MVFlow(fvec)
  clp=interleave(backw,clip,forw)
  clp=clp.RemoveDirt(limit,_grey)
  dnc=clp.RemoveTempGrain(1)
  clp=clp.SelectEvery(3,1)
  dnc=dnc.SelectEvery(3,1)
  clp=clp.SeeSaw(dnc,Sstr=0.26,Szp=12,SdampHi=20,bias=40)
  return clp
}


function hq_filter(clip c)
{
  c=c.SRemoveDirtMC(6)
  c=c.YlevelsS(0,1.05,255,0,255)
  c=c.LimitedSharpenFaster(Smode=4,strength=18,overshoot=1,wide=false)
  return c
}

[SBaT]

Thankyou for the nice script example, really usefull when I'm trying to grasp the use of MvTools and RemoveDirt with avs. However I'm quite confused atm after reading the end of RemoDirt manual as far if croping should be done before or after with a script like yours. As I'm encoding my 1080i.ts to 720p I have to crop atleast 2, 0, -4, -8 to get rid of black sides.

[canuckerfan]

Heini011, will your script wortk with mvtools 1.4.11?

[superuser]

Edited

[dnaxx]

Heini011, thanks for the script. But I don't get it running because "SeeSaw" is missing. Can you help me please?

[Boulder]

Why not use the search, it would give you the answer in no time.

[rkalwaitis]

Anyone a simple question.

So this script

function RemoveTempGrain(clip input, int _mode)
{
rg = RemoveGrain(input, mode=_mode)
return TemporalRepair(rg, input)
}

function RemoveDirt(clip input, int limit, bool "_grey")
{
_grey=default(_grey, false)
clensed=input.Clense(grey=_grey, cache=4)
alt=input.RemoveGrain(2)
return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=4,cthreshold=6, gmthreshold=40,dist=1,dmode=2,debug=false,noise=limit,noisy=16,grey=_grey)
}

function SRemoveDirtMC(clip,int limit, bool "_grey")
{
_grey=default(_grey, false)
limit = default(limit,6)
bvec = clip.MVAnalyse(isb=false, blksize=8, delta=1, pel=2, truemotion=true, idx=1)
fvec = clip.MVAnalyse(isb=true, blksize=8, delta=1, pel=2, truemotion=true, idx=1)
backw = clip.MVFlow(bvec)
forw = clip.MVFlow(fvec)
clp=interleave(backw,clip,forw)
clp=clp.RemoveDirt(limit,_grey)
dnc=clp.RemoveTempGrain(1)
clp=clp.SelectEvery(3,1)
dnc=dnc.SelectEvery(3,1)
clp=clp.SeeSaw(dnc,Sstr=0.26,Szp=12,SdampHi=20,bias=40)
return clp
}

function RMBDenoise(clip input, bool "_grey", int "repmode")
{
clmode=17
clensed=Clense(input, grey=_grey)
restore=Repair(clensed, input, mode=repmode, modeU = _grey ? -1 : repmode )
corrected=RestoreMotionBlocks(clensed, restore, neighbour=input, neighbour2=clensed, gmthreshold=70, dist=0, debug=true, noise=12, noisy=12, grey=_grey)
return RemoveGrain(corrected, mode=clmode, modeU = _grey ? -1 : clmode )
}

function hq_filter(clip c)
{
c=c.SRemoveDirtMC(6)
c=c.YlevelsS(0,1.05,255,0,255)
c=c.LimitedSharpenFaster(Smode=4,strength=18,overshoot=1,wide=false)
return c
}

and the one above are simple place as they are into the avs file? Correct? Of course I have to have the appropriate plugins.

[R3Z]

Originally Posted by rkalwaitis

Anyone a simple question.

and the one above are simple place as they are into the avs file? Correct? Of course I have to have the appropriate plugins.

It will work that way and you can rename the .avs file to .avsi and it will autoload into memory without being needed to be Loaded manually.

[rkalwaitis]

R3Z

So I import the avsi file that I made. How do I call the function to do the work . RemoveDirt()? Like this?

[Adub]

So I import the avsi file that I made.

What do you mean? The whole point of the avsi file is that you just drop it into your plugins directory and you don't have to import anything.

Basically, save those functions above in a text file with ".avsi" as the extension. Drag this to your Avisynth plugins folder.

If you ever want to use those functions, just add something like "hq_filter()" or "RMBDenoise(_grey=false)" to the script that you are currently working on.

There is no need to import anything, you just call the functions like you would any normal plugin.

[rkalwaitis]

Merlin,

Thanks, that is what I meant. Now I get it. Thanks again.

[Lyris]

Heini011: thank you so much for this function. The accuracy is INCREDIBLE compared to other dirt and scratch removal plugins I've used!

The only issue I'm having is that it's difficult to keep film grain AND get rid of the dirt/scratches. I've resorted to degraining, using your function, and then re-graining the output to keep the original look.

Once again, thank you.

[turnipzoink]

Damn I'm jealous.
This is the only plugin I can't get to work? I tried email the author at gmx.de, not replying.

I tried each dll and none of them work. Avisynth is obviously loading them, but can't find the function. What am I doing wrong?

9-18-2009 8-23-52 PM.png

[videoFred]

Originally Posted by turnipzoink

I tried each dll and none of them work. Avisynth is obviously loading them, but can't find the function. What am I doing wrong?

If you are using the Heini011 function, you must use HQ_filter() instead of RemoveDirt().

Off cource all dll's must be there, and Heini's function also.

PS: you need RemoveDirt.avs too.

Fred.

[Paazabel]

To the author: Thanks for this filter. I tried a bunch of different stuff that did not work on this clip. I'm sure many people have used it, but I wanted to make sure you got a "thanks" from me.

Sampling, stacked before-and-after (Lagarith, YV12, ~100M): http://www.sgt.cc/vid/Less-spotty.avi

(side note: Software rights can be a pain in the backside; would be nice if a great script could be "packaged" with all the other plugins that make it work. I know, I'm just lazy, but I spent about 2 hours debugging differences in plugin versions. It was worth it, but for an AVIsynth newb, it would have been improbable for them to stick with it.)

Sample frames:
I was particularly impressed with the second one since there are "real" dark spots in that frame and nothing else came remotely close to finding them. The first frame shows smoke rising over the hill which is in each frame and not processed as "noise." Very cool.




Not as interesting, but ... getting my money's worth out of the ol' computer!

[Lyris]

Question: is it possible to limit RemoveDirt to be colour-specific? That is, to only have it detect black (or nearly black) and white (or nearly white) areas as spots? It would improve accuracy...

[LaTo]

Originally Posted by Lyris

Question: is it possible to limit RemoveDirt to be colour-specific? That is, to only have it detect black (or nearly black) and white (or nearly white) areas as spots? It would improve accuracy...

Yes it is possible, without testing:

source   = last
filtered = source.RemoveDirt()

dark   = 32
bright = 196
mask   = mt_lut(source,"x "+string(dark)+" < 255 x "+string(bright)+" > 255 0 ? ?",U=1,V=1).removegrain(19,-1,-1)

mt_merge(source,filtered,mask,luma=true)

Adjust dark (0 = do nothing in black areas) & bright (255 = do nothing in bright areas) to your taste...


Another way is to limit RemoveDirt where changes are really big:

source   = last
filtered = source.RemoveDirt()

min  = 16
max  = 255
mask = mt_lutxy(source,filtered,"x y - abs "+string(min)+" < 0 x y - abs "+string(max)+" > 0 255 ? ?",U=1,V=1).removegrain(19,-1,-1)

mt_merge(source,filtered,mask,luma=true)

"min" is the minimum required and "max" is the maximum allowed


If you want to merge the two method:

source   = last
filtered = source.RemoveDirt()

dark   = 32
bright = 196
mask1  = mt_lut(source,"x "+string(dark)+" < 255 x "+string(bright)+" > 255 0 ? ?",U=1,V=1).removegrain(19,-1,-1)

min   = 16
max   = 255
mask2 = mt_lutxy(source,filtered,"x y - abs "+string(min)+" < 0 x y - abs "+string(max)+" > 0 255 ? ?",U=1,V=1).removegrain(19,-1,-1)

mt_merge(source,filtered,mt_logic(mask1,mask2,"min"),luma=true)

This one should be the best after tweaking the 4 parameters

[Paazabel]

Given the temporal nature of the filter (comparing one frame vs another), I'm not sure this would "improve performance" on most dirt. The comparison is the same frame-by-frame and it even removes blue dirt, from what I can tell, as well as grain and noise that are not near those thresholds.

Which is to say - academically, yes, I think you can do that, but I'm not sure it will have any meaningful "improvement" in either accuracy or speed.

[LaTo]

Originally Posted by Paazabel

Given the temporal nature of the filter (comparing one frame vs another), I'm not sure this would "improve performance" on most dirt. The comparison is the same frame-by-frame and it even removes blue dirt, from what I can tell, as well as grain and noise that are not near those thresholds.

Which is to say - academically, yes, I think you can do that, but I'm not sure it will have any meaningful "improvement" in either accuracy or speed.

Hum, your method remove spots... but also grain & details : oversmoothed effect! (artificial look)
It isn't really good to me

Some screenshots of my purpose:

Screenshot 1:
Original frame
The filtered clip (Your)
The code above (Mine)


Screenshot 2:
Original frame
The filtered clip (Your)
The code above (Mine)


The code:

DirectShowSource("Less-spotty.avi")
ConvertToYV12()

source   = last.crop(0,0,-720,0)
filtered = last.crop(720,0,0,0)

str = 20

mt_merge(source,filtered,mt_lutxy(source,filtered,"x y - abs "+string(str)+" *",U=1,V=1).mt_expand(U=1,V=1).removegrain(19,-1,-1),luma=true,U=3,V=3)

str=20 give very good result...

No spot, but still detailled


After, you can use a more robust denoiser to remove the little noise while preserving details:

MCTemporalDenoise(chroma=true,flat=true,stabilize=true,sigma=2,strength=200,Tovershoot=2)

Screenshot 1 & Screenshot 2


So, in conclusion, compare:
Before (your) - After (mine)
Before (your) - After (mine)

The choice is easy