Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
16th April 2006, 23:05 | #1 | Link |
Registered User
Join Date: Nov 2003
Posts: 148
|
script function RemoveNoiseMC
Hi folks,
i wrote a little script function as a better alternative for the old Dust. it is optimized for removing noise with only a little lost of details. it removes larger spots and other dirt also. this script works in YV12 color mode only. the video-clip must have a mod 8 resolution. i suggest to insert in the first line of the avisynth-script: SetMemoryMax(128) # for 512MB RAM or SetMemoryMax(256) # for >=1024 MB RAM if you want to remove only a little bit of noise from a high quality movie, please look at RemoveNoiseMC_HQ Code:
function RemoveTempGrain(clip input, int _mode) { rg = RemoveGrain(input, mode=_mode) return TemporalRepair(rg, input) } function RemoveDirt(clip input, int limit, int rgrain, bool "_grey") { _grey = default(_grey, false) _dgr1 = 0.35+rgrain*0.3 _dgr2 = 0.45+rgrain*0.4 repmode = 1 clensed = Clense(input, grey=_grey, cache=4) restore = input.FluxSmoothST(3+3*rgrain,rgrain) restore = Repair(restore, input, mode=repmode, modeU= _grey ? -1 : repmode) restore = rgrain==0 ? restore.RemoveGrain(1) : \ restore.VagueDenoiser(threshold=_dgr1, chromaT=_dgr1, nsteps=7, percent=75).RemoveGrain(1) alt = input.VagueDenoiser(threshold=_dgr2, chromaT=_dgr2, nsteps=7, percent=100).RemoveGrain(5) return RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, pthreshold=4+2*rgrain, cthreshold=6+2*rgrain, gmthreshold=40, dist=1, dmode=2, debug=false, noise=limit, noisy=12, grey=_grey) } global idx_c = 25 function RemoveNoiseMC(clip,int "rdlimit", int "rgrain", int "denoise", bool "sharp", float "csharpen", bool "_grey") { rdlimit = default(rdlimit,11) rgrain = default(rgrain,2) denoise = default(denoise,8) sharp = (rgrain<1) ? default(sharp,true) : default(sharp,false) csharpen = (rgrain>2) ? default(csharpen,0.17) : (rgrain>1) ? default(csharpen,0.15) : default(csharpen,0.13) csharpen = sharp ? csharpen : csharpen+0.08 _grey = default(_grey,false) _dgr = 0.45+rgrain*0.4 dummy = clip.BlankClip(length=0) global idx_c = idx_c+1 cbs = 8 cov = (cbs>4) ? cbs/4 : 0 ccf = cbs*cbs/64 cpn = (denoise>12) ? 50*ccf : (denoise>8) ? 58*ccf : (denoise>5) ? 66*ccf : 72*ccf csh = sharp ? 1 : 0 bvec2 = clip.MVAnalyse(isb=false, blksize=cbs, delta=2, pel=2, sharp=csh, overlap=cov, truemotion=true, pnew=cpn, idx=idx_c) bvec1 = clip.MVAnalyse(isb=false, blksize=cbs, delta=1, pel=2, sharp=csh, overlap=cov, truemotion=true, pnew=cpn, idx=idx_c) fvec1 = clip.MVAnalyse(isb=true, blksize=cbs, delta=1, pel=2, sharp=csh, overlap=cov, truemotion=true, pnew=cpn, idx=idx_c) fvec2 = clip.MVAnalyse(isb=true, blksize=cbs, delta=2, pel=2, sharp=csh, overlap=cov, truemotion=true, pnew=cpn, idx=idx_c) backw1 = rdlimit>13 ? \ rdlimit>20 ? \ clip.MVFlow(bvec1, idx=idx_c).Deblock(quant=22, aOffset=6, bOffset=6) : \ clip.MVFlow(bvec1, idx=idx_c).Deblock(quant=16, aOffset=4, bOffset=4) : \ clip.MVFlow(bvec1, idx=idx_c) forw1 = rdlimit>13 ? \ rdlimit>20 ? \ clip.MVFlow(fvec1, idx=idx_c).Deblock(quant=22, aOffset=6, bOffset=6) : \ clip.MVFlow(fvec1, idx=idx_c).Deblock(quant=16, aOffset=4, bOffset=4) : \ clip.MVFlow(fvec1, idx=idx_c) clp = interleave(backw1,clip,forw1) clp = clp.RemoveDirt(rdlimit,rgrain,_grey) dnc = denoise==0 ? clp.RemoveTempGrain(rgrain).SelectEvery(3,1) : dummy clp = clp.SelectEvery(3,1) dnc = denoise==0 ? dnc : \ clp.MVDenoise(bvec2, bvec1, fvec1, fvec2, thT=denoise, thSAD=190+15*denoise, thmv=40, thSCD1=230+5*denoise) vid_mo = dnc.VagueDenoiser(threshold=_dgr, chromaT=_dgr, nsteps=7, percent=75) vid_mo = rgrain==1 ? vid_mo.RemoveGrain(1) : vid_mo.RemoveGrain(5) dnc = dnc.ConditionalFilter(dnc, vid_mo, "(YDifferenceFromPrevious()+YDifferenceToNext())/AverageLuma()", "<", "0.3") clp = clp.SeeSaw(dnc, Sstr=csharpen, Szp=12, SdampHi=20, bias=40) return clp } for slight noisy video or spot/dirt removing only: Code:
function hq_filter(clip c) { c=c.RemoveNoiseMC(rdlimit=6,rgrain=1,denoise=0) c=c.LimitedSharpenFaster(Smode=4,strength=18,overshoot=0,wide=false,ss_x=1.3,ss_y=1.3) return c } Code:
function mq_filter(clip c) { c=c.RemoveNoiseMC(rdlimit=10,rgrain=2,denoise=8) c=c.LimitedSharpenFaster(Smode=4,strength=20,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3) return c } Code:
function lq_filter(clip c) { c=c.RemoveNoiseMC(rdlimit=18,rgrain=3,denoise=14) c=c.VagueDenoiser(threshold=0.6, nsteps=6, chromaT=0.6, percent=75) c=c.deen("a3d",rad=3,thrY=3,thrUV=5,min=0.25,tthY=2,tthUV=3,scd=6) c=c.LimitedSharpenFaster(Smode=4,strength=24,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3) return c } Code:
function RemoveHighNoiseMC(clip clip,int "rdlimit", int "rgrain", int "denoise", float "csharp", bool "_grey") { rdlimit = default(rdlimit,24) rgrain = default(rgrain,3) denoise = default(denoise,15) csharp = (rgrain>2) ? default(csharp,0.32) : default(csharp,0.28) _grey = default(_grey, false) _dgr = 0.7+rgrain*0.5 global idx_c = idx_c + 1 cbs = 8 ccf = cbs*cbs/64 cpn = (denoise>12) ? 50*ccf : 57*ccf bvec3 = clip.MVAnalyse(isb=false, blksize=cbs, delta=3, pel=2, sharp=0, truemotion=true, pnew=cpn, idx=idx_c) bvec2 = clip.MVAnalyse(isb=false, blksize=cbs, delta=2, pel=2, sharp=0, truemotion=true, pnew=cpn, idx=idx_c) bvec1 = clip.MVAnalyse(isb=false, blksize=cbs, delta=1, pel=2, sharp=0, truemotion=true, pnew=cpn, idx=idx_c) fvec1 = clip.MVAnalyse(isb=true, blksize=cbs, delta=1, pel=2, sharp=0, truemotion=true, pnew=cpn, idx=idx_c) fvec2 = clip.MVAnalyse(isb=true, blksize=cbs, delta=2, pel=2, sharp=0, truemotion=true, pnew=cpn, idx=idx_c) fvec3 = clip.MVAnalyse(isb=true, blksize=cbs, delta=3, pel=2, sharp=0, truemotion=true, pnew=cpn, idx=idx_c) backw1 = rdlimit>20 ? \ clip.MVFlow(bvec1).Deblock(quant=22, aOffset=6, bOffset=6) : \ clip.MVFlow(bvec1).Deblock(quant=16, aOffset=4, bOffset=4) forw1 = rdlimit>20 ? \ clip.MVFlow(fvec1).Deblock(quant=22, aOffset=6, bOffset=6) : \ clip.MVFlow(fvec1).Deblock(quant=16, aOffset=4, bOffset=4) clp = interleave(backw1, clip, forw1) clp = clp.RemoveDirt(rdlimit,rgrain,_grey) clp = clp.SelectEvery(3,1) dnc = clp.MVDenoise(bvec3, bvec2, bvec1, fvec1, fvec2, fvec3, thT=denoise, thSAD=160+12*denoise, thmv=40, thSCD1=255+4*denoise) vid_mo = dnc.VagueDenoiser(threshold=_dgr, chromaT=_dgr, nsteps=6, percent=75) vid_mo = rgrain==1 ? vid_mo.RemoveGrain(1) : vid_mo.RemoveGrain(5) dnc = dnc.ConditionalFilter(dnc, vid_mo, "(YDifferenceFromPrevious()+YDifferenceToNext())/AverageLuma()", "<", "0.3") return csharp==0 ? dnc : clp.SeeSaw(dnc, Sstr=csharp, Szp=12, SdampHi=20, bias=40) } function vlq_filter(clip c) { c=c.RemoveHighNoiseMC(rdlimit=24,rgrain=3,denoise=16,csharp=0.3) c=c.VagueDenoiser(threshold=1.0, nsteps=6, chromaT=1.2, percent=75) c=c.deen("a3d",rad=3,thrY=4,thrUV=7,min=0.25,tthY=2,tthUV=3,scd=7) c=c.LimitedSharpenFaster(Smode=4,strength=28,overshoot=1,wide=false,ss_x=1.2,ss_y=1.2) return c } Latest MVTools 1.4, DeBlock 1.2 and VagueDenoiser 0.35.1 from Fizick's Page RemoveGrain pre 1.0 RemoveDirt 0.9 FluxSmooth 1.1a from warpenterprises SeeSaw 2006.01.02, therefor MaskTools 1.5.8 for my examples LimitedSharpenFaster, therefor MaskTools v2.0a30 deen 1.0 beta 1 (old version) suggestions for parameter selection: - set rdlimit=0, denoise=0 and rgrain according to the examples above - look for a very bad frame - increase rdlimit until the big noise is gone - increase denoise until the remaining noise is minimized - set sharp=true for a video with high resolution of fine details, which you want to encode using a high bitrate set sharp=false for older film material or tv captures revisions: april 19th: -RemoveHighNoiseMC clean-up april 20th: -deen 1.0 beta 1 attachment added -hints for parameter selection added -minor removedirt parameter change may 10th: - spatial filtering for moving block areas in RemoveDirt - minor parameter changes may 29th: deen parameter in vlq_filter example june 12th: - improved scene switch behaviour june 24th: - minor script error corrected (quality unchanged), thanks to Alain2 august 15th: - bicubic subpixel interpolation in mvanalyze. much better sharpness thanks to Fizick new MVTools! august 16th: - adjustments for sharp subpixel interpolation november 28th: - sharp option for sharp subpixel interpolation added for manual selection . Last edited by Heini011; 28th November 2006 at 22:20. |
16th April 2006, 23:51 | #3 | Link |
BDVD Team
Join Date: Dec 2004
Location: Rio de Janeiro - RJ/Brasil
Posts: 306
|
@Heini011
Hi there! It seems quite interesting, so thanks for share it with us. I just have a suggestion: posting all filters dependencies in order to get the script working, because of maybe some users can to have some difficulty to know what filters/versions are necessary to get it working properly. Cheers, |
18th April 2006, 22:45 | #8 | Link |
Registered User
Join Date: May 2005
Posts: 236
|
Thanks Heini011 for sharing your script, it proves to be rather good on some materials I have and was struggling on with other methods ; motion compensation seems to be very efficient at keeping details while doing a good job at denoising!
[edit] I just noticed you changed the vlq_filter from the initial one, introducing the RemoveHighNoiseMC, and also you changed denoise=6 to denoise=0 in the hq_filter... If you make other mods in the future, please let us know or we will miss them ^^ Last edited by Alain2; 19th April 2006 at 01:04. |
19th April 2006, 13:17 | #10 | Link |
ангел смерти
Join Date: Nov 2004
Location: Lost
Posts: 9,556
|
Another thing I can suggest is to package it into a single file, so people can just download it right to their avs folder. You're more than welcome to upload it to avisynth's wiki and link it from there (though the link will change anytime you update your script, except when linking from the wiki), or you can ask richard for a bit of space. I have a video I'd like to test this out on soon!
Last edited by foxyshadis; 20th April 2006 at 00:41. |
19th April 2006, 15:08 | #11 | Link |
Registered User
Join Date: Feb 2002
Posts: 85
|
Great functions! I tried it for my tv-capture and I really like the results!
During encoding on my AMD X2 dual core cpu, the load goes only up to 58%. Could somebody tell me, where in the script I could put the "SetMTMode" commands properly, so that it can be faster? I tried this, but got green dots over the image: Code:
SetMTMode(2) mq_filter() |
20th April 2006, 00:19 | #12 | Link |
Registered User
Join Date: May 2005
Posts: 236
|
I didn't have a problem without the deblock function from fizick, but I realize that i have also dgdecode.dll in avisynth plugins directory, which contains a deblock() function as well ; maybe it could be a good idea to rename one of these functions to avoid confusion between the 2 (as it's likely that a lot of people have dgdecode.dll in their plugins), or maybe change the script for deblock functions, to be called with the syntaxe deblock_deblock() to specifically call fizick'dll deblock function... (This is probably not a real issue as from fizick's and neuron2's changelogs it appears they didn't really touch manao's core deblock function)
|
20th April 2006, 01:12 | #13 | Link |
Registered User
Join Date: Nov 2003
Posts: 148
|
Hi,
for thresholds > 0.6 fft3dfilter makes a better job than vaguedenoiser. here is an example for an improved vlq_filter: Code:
function vlq_filter_ultimate(clip c) { c=c.RemoveHighNoiseMC(rdlimit=24,rgrain=3,denoise=16) c=c.fft3dfilter(sigma=1.2, sharpen=0.4, plane=4) c=c.deen("a3d",rad=3,thrY=4,thrUV=7,min=0.25,tthY=2,tthUV=3,scd=7) return c } fft3dfilter185.zip, therefor fftw3.dll @Morpheus_xx: SetMTMode(2) should, according to avisynth documentation, be in the first line of your avs-script to support multi-threading. @moderator: please approve my deen 1.0 beta 1 attachmend in the first post. greetings. Last edited by Heini011; 29th May 2006 at 00:21. |
20th April 2006, 07:13 | #14 | Link | |
Registered User
Join Date: Feb 2002
Posts: 85
|
Quote:
I think one of the many used filters does not work so well with MTMode(2). That why I've asked where to "tweak" these functions. |
|
21st April 2006, 16:55 | #20 | Link |
Registered User
Join Date: Aug 2004
Posts: 211
|
Well it took me some time to get it running, but now I got it and its a fantastic script!
I did a quick test with chapter 29 from Matrix and everything looked great, besides this: There is a flashlight in red for just one frame and it gets detected as dirt. I'm impressed by the quality, as even the face is cleared from the light. Really, this is no BS, no color conversion between the both and no other tuning, this is just RNMC working! I'm using the mq_filter preset... *edit* some more examples: Conclusion: This might be a pretty cool script for slow movies, but extreme action and fog is a no no, as it kills major detail And by the way: I'm using the mq preset as hq didn't remove enough noise. But I'm trying my own settings now. Last edited by Disabled; 22nd April 2006 at 08:54. |
Thread Tools | Search this Thread |
Display Modes | |
|
|