Smoothing filter that respects lines in dark areas

[Overdrive80]

Currently, I am filtering a video using mvdegrain but does not eliminate all the noise so I must use filters to soften some areas of the video. The problem is that this type of filters, as tbilateral in darker areas, blurs the lines and therefore the details of the video. The filter removes my least is MSmooth lines, but still it too.

Do you know of some other smoothing that is respectful of the lines? Could use some mascara to make this not happen, as mmask?

I tried using Repair in mode 16, but recovered after smoothing the noise from other areas ..

Thanks

NOTE: Tbilateral creates aliasing

[mastrboy]

tnlmeans / nlmeans preserves lines very well, atleast for cartoons/anime

[TheRyuu]

dfttest (relatively speaking at least).

[Overdrive80]

Ok, i try its and expected to behave properly in the dark areas. Thanks.

[Overdrive80]

Hi, I already did tests with these filters. Well, I will put a series of frames, for doing comparative.

Original Video
http://s12.postimage.org/6ye809avt/First_frame.jpg (http://postimage.org/image/6ye809avt/)http://s7.postimage.org/tktbmiymv/Second_frame.jpg (http://postimage.org/image/tktbmiymv/)http://s9.postimage.org/exfd3qxez/Third_frame.jpg (http://postimage.org/image/exfd3qxez/)

Filtered video Video with Dfttest
http://s7.postimage.org/8wz9lj2s7/First_frame.jpg (http://postimage.org/image/8wz9lj2s7/)http://s8.postimage.org/tx7prleap/Second_frame.jpg (http://postimage.org/image/tx7prleap/)http://s8.postimage.org/42xx1teap/Third_frame.jpg (http://postimage.org/image/42xx1teap/)

This is similar to use fft3dfilter and is very very slow, safe lines but thin its. Reference frame 2, in dark area(sleeve folds).

prefiltered =dfttest(last,sigma=14, sigma2=14)

ups = prefiltered.EEDI3(field=1).spline36Resize(2*width(last), 2*height(last), src_left=0.25)
super = MSuper(prefiltered, pel=2,pelclip=ups)

bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)

bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)

bv3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=4)

MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=500,thSADc=400)

LSFmod(defaults="fast", secure=true, soothe=true, preblur="ON",edgemaskHq=true,strength=120,soft=10)

TNLMeans(Ax=4, Ay=4, Az=0,sx=4,sy=4,bx=2,by=2,a=1,h=0.9,rm=5)

asharp(0.70)

GradFun2DBmod(thr=1.0)

Filtered video Video with MSmooth
http://s9.postimage.org/g2e0mv99n/First_frame.jpg (http://postimage.org/image/g2e0mv99n/)http://s10.postimage.org/4ww44z6f9/Second_frame.jpg (http://postimage.org/image/4ww44z6f9/)http://s11.postimage.org/la7dg0or3/Third_frame.jpg (http://postimage.org/image/la7dg0or3/)

Too blurred lines in frame 2, in dark area.

ups = EEDI3(field=1).spline36Resize(2*width(last), 2*height(last), src_left=0.25)
super = MSuper(pel=1,pelclip=ups)

bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)

bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)

bv3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=4)

MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=500,thSADc=400)

LSFmod(defaults="fast", secure=true, soothe=true, preblur="ON",edgemaskHq=true,strength=120,soft=10)

MSmooth(mask=false, threshold=3, strength=3, chroma=false)

asharp(0.70)

GradFun2DBmod(thr=1.0)

Filtered video Video with Tbilateral
http://s11.postimage.org/vx5rz6ma7/First_frame.jpg (http://postimage.org/image/vx5rz6ma7/)http://s7.postimage.org/b8fp8o72v/Second_frame.jpg (http://postimage.org/image/b8fp8o72v/)http://s11.postimage.org/bw619yxxr/Third_frame.jpg (http://postimage.org/image/bw619yxxr/)

Same that msmooth but more blurred

ups = EEDI3(field=1).spline36Resize(2*width(last), 2*height(last), src_left=0.25)
super = MSuper(pel=1,pelclip=ups)

bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)

bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)

bv3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=4)

MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=500,thSADc=400)

LSFmod(defaults="fast", secure=true, soothe=true, preblur="ON",edgemaskHq=true,strength=120,soft=10)

Tbilateral()

asharp(0.70)

GradFun2DBmod(thr=1.0)

Filtered video Video with TNLMeans
http://s9.postimage.org/b981n8gx7/First_frame.jpg (http://postimage.org/image/b981n8gx7/)http://s8.postimage.org/m9u00rue9/Second_frame.jpg (http://postimage.org/image/m9u00rue9/)http://s9.postimage.org/eu3x6glgr/Third_frame.jpg (http://postimage.org/image/eu3x6glgr/)

Similar that dfttest.

ups = EEDI3(field=1).spline36Resize(2*width(last), 2*height(last), src_left=0.25)
super = MSuper(pel=2,pelclip=ups)

bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)

bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)

bv3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=4)

MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=500,thSADc=400)

LSFmod(defaults="fast", secure=true, soothe=true, preblur="ON",edgemaskHq=true,strength=120,soft=10)

TNLMeans(Ax=4, Ay=4, Az=0,sx=4,sy=4,bx=2,by=2,a=1,h=0.9,rm=5)

asharp(0.70)

GradFun2DBmod(thr=1.0)

------------------------------------------------

Video for frame 1: http://www.megaupload.com/?d=RRU35L5U
Video for frame 2: http://www.megaupload.com/?d=KIHYTF5I
Video for frame 3: http://www.megaupload.com/?d=G1I88VSN

I uploaded the video if you can help me with the configuration of the filtrate. Thanks in advance

[vampiredom]

This might be a little crazy but, if the problem is that a filter processes dark and light areas inconsistently, you could try mitigating the issue:

a = Tbilateral()
b = invert().Tbilateral().invert()
Merge(a, b)

I know it is kind of a kludge - and it will double the required processing - but it may give you results that are acceptable in both light and dark areas.

[Overdrive80]

This might be a little crazy but, if the problem is that a filter processes dark and light areas inconsistently, you could try mitigating the issue:



I know it is kind of a kludge - and it will double the required processing - but it may give you results that are acceptable in both light and dark areas.

I going to try it. I did think increase level brightness and apply toon() before other filters.

Thanks

[vampiredom]

You may also try varying the weight parameter of the Merge, to bias it one way or another.

[Dogway]

The only denoiser that doesn't perform too bad in dark areas is mdegrain. But if still you have problems with it, then feed a brightened dark areas version of the clip for the motion estimation, so motion analysis can detect the lineart. With spatial denoisers I either don't denoise darks (by using a luma mask) or you can try an edge mask instead.

[naoan]

Here's my attempt :
http://i.imgur.com/9GqS7.png


strength = 4
dbw=24
dow=dbw/2
fft3dfilter(sigma=strength, sigma2=strength/2, sigma3=strength/4, sigma4=strength/8,bw=dbw,bh=dbw,ow=dow,oh=dow)
vaguedenoiser(nsteps=8, chromat=0, wiener=true)
ttempsmooth(maxr=2, strength=2)
lsfmod(defaults="slow")
gradfun3(smode=2, lsb=true).DitherPost()

[Caroliano]

One solution: you can filter less in darker areas (~zero filtering in black lines).

Here is a script that I made based on some (http://forum.doom9.org/showthread.php?t=133247) Didée posts (http://forum.doom9.org/showthread.php?t=105086):
function BrightDfttest(clip input, float "sbright", float "smedium", float "sdark",
\ int "th_low", int "th_med", int "th_high", int "tbsize")
{


sbright = default( sbright, 0 )
smedium = default( smedium, 0 )
sdark = default( sdark, 0 )

th_low = default( th_low, 20 )
th_med = default( th_med, 40 )
th_high = default( th_high, 100 )


bright = (sbright > 0) ? input.dfttest(sigma = sbright,tbsize=tbsize,sbsize=18,sosize=9): input
medium = (smedium > 0) ? input.dfttest(sigma = smedium,tbsize=tbsize,sbsize=18,sosize=9): input
dark = (sdark > 0) ? input.dfttest(sigma = sdark,tbsize=tbsize,sbsize=18,sosize=9) : input

mmask = medium.levels(th_med, 1.0, th_high, 255, 0, false)
dmask = medium.levels(th_low, 1.0, th_med, 255, 0, false)

output = bright.mt_merge(medium, mmask, U=3, V=3, luma=true)
\ .mt_merge(dark, dmask, U=3, V=3, luma=true)

return output
}
You can call like: BrightDfttest(2.0, 0.85), or BrightDfttest(5.5, 2.7, 1.0, 20, 35, 60, 1) <-- The name isn't the best... but it works...

It is pretty hardwired for my use of it. Also, it is kinda slow, because you run dfttest 2/3 times per frame, so I used faster settings. You can put any denoiser there instead dfttest.

[Overdrive80]

The only denoiser that doesn't perform too bad in dark areas is mdegrain.

Yes, was it. I adjust this filter and obtain best results, but is impossible remove all noise. I have adjust value of thSCD1 and 2, then the blurred image is reduced. In MVAnalyse, the parameter overlap is too important, respect to quality

sigma=2.6

prefiltered =FFT3Dfilter(sigma=sigma*0.8,sigma2=sigma*0.6,sigma3=sigma*0.4,sigma4=sigma*0.2,bt=5,\
bw=32, bh=32, ow=16, oh=16,sharpen=0.3, plane=4,dehalo=0.1,ncpu=6)



#ups = EEDI3(field=1).spline36Resize(2*width(last), 2*height(last), src_left=0.5,src_top=0.5)

super = MSuper(prefiltered)#,pelclip=ups)

bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)

MDegrain1(super,bv1,fv1,thSAD=550,thSADc=550,thSCD2=68)

LSFmod(defaults="fast")

TNLMeans(sx=3,sy=3,a=1,h=2.0)

maa()

GradFun2DBmod(thr=1.2)

http://s11.postimage.org/4r7onh0m7/image.jpg (http://postimage.org/image/4r7onh0m7/)http://s10.postimage.org/eida26fyt/image.jpg (http://postimage.org/image/eida26fyt/)http://s12.postimage.org/wje1h0gh5/image.jpg (http://postimage.org/image/wje1h0gh5/)

Thanks to all for answers. With this filter I conform, the only bad thing is that it is very slow. I have thought filter video in various steps. Imagine that the resulting video is the same that if coded it on one-time.

[Dogway]

If you have problems removing all noise use a temporal prefilter, you have some examples at "The Road" thread. I think 550 for thSAD might be too high. I tried to download the clip to test but the link doesn't work.
Then use this code and tweak as desired to brighten dark shades but not lineart in the prefilter clip.

str=30
mt_lut("x 255 x 35 "+string(str/5)+" - / 255 "+string(str)+" - + / ^")

tnlmeans is a killer. I barely use it, only for still images or no more than 10min sources. And of course multithreaded:

MT("tnlmeans(ax=3,ay=3,h=2.0,a=1.0,sse=false)",2,2) # sse=false for cartoons

eedi3 is also very slow.

[Overdrive80]

If you have problems removing all noise use a temporal prefilter, you have some examples at "The Road" thread. I think 550 for thSAD might be too high. I tried to download the clip to test but the link doesn't work.
Then use this code and tweak as desired to brighten dark shades but not lineart in the prefilter clip.

str=30
mt_lut("x 255 x 35 "+string(str/5)+" - / 255 "+string(str)+" - + / ^")

tnlmeans is a killer. I barely use it, only for still images or no more than 10min sources. And of course multithreaded:

MT("tnlmeans(ax=3,ay=3,h=2.0,a=1.0,sse=false)",2,2) # sse=false for cartoons

eedi3 is also very slow.

Are you referring to this post? http://forum.doom9.org/showthread.php?t=162813&highlight=road

When you say prefiltered is in various scripts or in the same. Im sorry but i dont use masktools. I included in the script "str=30
mt_lut("x 255 x 35 "+string(str/5)+" - / 255 "+string(str)+" - + / ^")" but everything became black.

Yes, if you were right with the level of thsad, with 300 the quality is the same and the image was more focused.

I reupload file http://www.mediafire.com/?b4g576dd63va1qo

Thanks

[Dogway]

With only one frame to solve your problem is very difficult to make a safe proof script. Anyways I dealed with the rest, prefilter isn't going to help much here, so I denoised spatially instead, this is no problem with animation (you can use fft3dfilter if you like), and recommended for cleaning edges a bit. For edges on darks I used a lumamask for protection, and then mdegrain over. The whole pipeline is 16 bit so it's more accurate and flat areas will look better. At last it lacked contrast so I tweaked that a bit too.

Generally it shakes too much so you might want to apply a deshaker first of all, it helps for temporal denoisers, in this example I used stabilise, but use any other you prefer.

This encoded to ffv1 at near 1 fps, it isn't bad for a 20min episode, in my core2duo 2.53gh.
You must be using masktools2, because maa() uses it and it worked for you.

50Mb
http://www.mediafire.com/?4hhwh8nfo6eoy8o
setmtmode(5,2)
trim(0,36)+\
Trim(37,131).Stabilise(capabsolutemetric(15))+\
trim(132,239).Stabilise(capabsolutemetric(15))+\
Trim(240,336).Stabilise(capabsolutemetric(15))+\
Trim(337,371)+\
Trim(372,391).Stabilise(capabsolutemetric(15))+\
Trim(392,0)


setmtmode(2,2)
Crop(6, 4, -4, -6)

####
# MAA()
input=Spline36Resize(664,480) # ITU
mask = input.mt_edge("sobel",7,7,5,5).mt_inflate()
aa_clip=input.spline36Resize(1328,960).TurnLeft().SangNom().TurnRight().SangNom().spline36Resize(664,480).MergeChroma(input)
mt_merge(input,aa_clip,mask)
####

a=last

dfttest(tbsize=1,sigma=20,lsb=true).ditherpost(mode=6)
lumamask(a,last,45,55,brightfilt=false) # tweak these 2 values for lineart on darks

smdegrain(tr=3,thSAD=300,cclip=a,refinemotion=true,chroma=false,hpad=0,vpad=0,plane=0,lsb_out=true)
SmoothContrast(0.3,sat=0.1,median=64,local="bright",lsb=true) # This can create some artefacts at around values of 64
gradfun3(0.5,13,smode=1,lsb_in=true,mode=0,ampo=1.2)

If mdegrain gives you problems like ghosting on the darks, do the prefilter workaround I explained before.

str=15
pre=mt_lut("x 255 x 35 "+string(str/5)+" - / 255 "+string(str)+" - + / ^")
smdegrain(tr=3,thSAD=300,cclip=a,refinemotion=true,pre_custom=pre,chroma=false,hpad=0,vpad=0,plane=0,lsb_out=true)

[Overdrive80]

Ok, im going to try your script, and will tell you results. Thanks

EDIT: SmoothContrast??? Isnt smoothlevels?? If you want, upload the entire fle.

[Dogway]

They are just personal custom functions, I'm showing you the idea.
lumamask has its own thread, and smoothcurve is in the smoothadjust thread, you can use sgradation which does similar type of contrast, smdegrain is just a wrapper for mdegrain with a few addons.

maa() also slows down the process a bit too much because the 3 spline36. Use splinaa or nothing instead. And I'd add the next line at the end to enhace lineart a bit and remove faint ringing:

aWarpSharp2(depth=3,type=1,blur=4,thresh=128,chroma=2)

It's a challenging source because it battles between the strong noise reduction and lineart on shades protection, it's hard to get both perfect but you can adjust as you like to one or another.

And I got it wrong, source clip is not ITU, is Generic PAR 8/9. So for square pixels after that cropping should be spline36resize(646,480)

[Overdrive80]

If i filter video adjusting it to frame 2 (sleeve), blurred frame 1 (hand). I'm beginning to think that it is impossible to filter so as to keep the details.

If increase value of gradfun3 to 1.0, frame 2 has acceptable appearance. I think there isnt residual noise but banding effect. It is as if video had problems that are antagonistic.

I thought if I used a mask could prevent the lines disappear. I've tried using mt_makediff and mt_adddiff, watching the odd script but with really bad results

I couldnt still test your script because it had no function lumamask. I already downloaded it and will use along with your new suggestion as to the lines.

@Caroliano

I used you script, have modified a bit, add lsb parameter for using dither and the result is acceptable but blurred some parts too.

[Dogway]

You can try an edgemask, but lineart is very dirty so I recommend denoising them, and resharpen/refine it afterwards.
To combat banding that's why I used 16 bit pipeline, and gradfun3. The source has a lot of noise, indeed this is a good material for tnlmeans but its speed isn't practical, so I used dfttest. Because it has lots of noise its recommended temporal denoising, but strong temporal on anime isn't recommended as well, so I use a mix of spatial, temporal and masking.

In my example I only protected dark lineart fading through masking for dfttest and through refinemotion for smdegrain. If refinemotion isn't enough you can use a gamma enhacer (as I suggested) as a prefilter clip, or for more aggressive protection apply the lumamask to smdegrain too, although separation of dark shades and lineart is very subtle and some noise could get passed through the shades.

After a while I got serious so I decided to combine edgemasking with lumamask, that is protect only dark lines of the dark areas. I think you will like this much better.
This is optimized for the sleeve scene, but change lbias, and ebias for other scenes.

Crop(6, 4, -4, -6)

a=last
dfttest(tbsize=1,sigma=20,lsb=true).ditherpost(mode=6)

Lbias=60
Ebias=5
msk=mt_logic(mt_binarize(Lbias,upper=true).mt_inpand.mt_inpand,\
mt_edge("prewitt",thY1=Ebias,thY2=Ebias).mt_expand.mt_inpand,"min")

smdegrain(tr=3,thSAD=300,cclip=a,refinemotion=true,chroma=false,hpad=0,vpad=0,plane=0,lsb_out=true)
dither_merge16_8(a.Dither_convert_8_to_16(),msk,luma=true)

SmoothContrast(0.3,sat=0.1,median=64,local="bright",lsb=true) # This can create some artefacts at around values of 64
Dither_resize16(646,480,kernel="spline36")
gradfun3(0.5,13,smode=1,lsb_in=true,mode=0,ampo=1.2)
maa()
aWarpSharp2(depth=3,type=1,blur=4,thresh=128,chroma=2)

I sent you some functions over PM.
I also think it's not a good idea to resize before dfttest, because dfttest is optimized to look for blocks.
If you still want to do antialias, do it at the end with an edge mask.
And I think you can substitute smoothcontrast with smoothlevels(lmod=2...) and smoothtweak

http://img585.imageshack.us/img585/7672/lineartei.th.png (http://img585.imageshack.us/img585/7672/lineartei.png)

[mandarinka]

Well, for my liking mdegrain also hits dark areas too much, and those lines there are already susceptible to getting damaged by the subsequent encoding.

For this reason I have an hacky masking trick that I originaly used for old anime with very strong noise in bright parts (I assume it happens when they scan from film negative).
Imho, brighter parts can be denoised a bit more than darker ones while still achieving sorta okay look. Dark parts get ugly very fast.

In any case, the point is to filter the brighter parts more and the darker parts less. I'ts crude but it works for me...


source = last
prefilt = FFT3Dgpu(sigma=2.5,bw=32,bh=32,bt=3,ow=16,oh=16,plane=4,precision=1) # prefiltering the ME source for better accuracy or something
superfilt = prefilt.MSuper(pel=2, sharp=1)
super = MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(superfilt, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(superfilt, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(superfilt, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(superfilt, isb = false, delta = 2, overlap=4)

degrain = source.MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=130)
mask = source.mt_lut("x 3.10 ^ 41000 /")
degrainless = source.MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=25)

mt_merge(degrainless,degrain,mask,luma=true) # or swap degrainless for source if you want almost no touching of dark parts


I arrived at it via experimentation, hence the weird mask formula. If you are going to tweak it, I would only touch the divisor - lower it (say to 35000 - 38000) to get a steeper ramp-up of the degraining strength.

Adjust the thsad values in degrain and degrainless calls to get the desired strenght. Degrainless is what gets applied to dark area, degrain is what gets applied to a bright area. Stuff inbetween gets a weighted mix of course.

[Overdrive80]

Dogway, i have already tested your script, and frame 1 is blurred http://s11.postimage.org/u40918w3j/image.jpg (http://postimage.org/image/u40918w3j/)

I have uploaded longer video. As I said before, think that are problems are opposite, and only with an intermediate calibration could be compensated. I hope to be wrong. ^^
http://www.megaupload.com/?d=N2NOU9ND

[mandarinka]

So, I checked out your sample and I must say I find your script totally insane. You have qutie a nice source, don't you. It has grain and it shakes but since it's old that's only natural.

What you do to the poor thing: smooth, then sharpen, then smooth, then sharpen. Then, most likely because you smoothed too bloody much, you have to actually run gradfun (debanding filter)! For your information, debandeing filters are intended to hide banding that was introduced by undithered rendering of gradients in *computer rendered* animation. people that have to use it to hide the damage they themselves caused to their sources are simply failures... (yes this criticism is for you - hand drawn animation doesn't have digitally rendered gradients, if you run a debander on such source, you it was you yourself who caused the trouble.)

In the same vein, you should really drop both your sharpeners, the look of the source's lines is perfectly good and natural in the source sample! Again, if you must sharpen because you fubared your precious source with your own damn smoothing, something is wrong on your side of things.

Why don't you try to apply *just* mild degraining (or for example the thing I posted) on the source, instead of some multilateral mutilating? Of course, if you aren't interested in how the moive/series/OVA actually looks, feel free to turn it into a damn Futurama or Powerpuff Girls or...

Forgive me for being blunt and aggressive here, but this merciless approach to "improving" video is too popular amongst people for me to stay calm...

P.S.
The thing looks pretty much like a remastered source. That's a hint that it doesn't need filtering, except whatever is completely necessary to somewhat limit the bitrate needed for encoding. As I said, just some careful touch to the grain. Best is to not remove it completely, it's best to leave some texture there, else the result looks bad.

[Dogway]

hey, that's cheating, I didn't have that shot to test! : )

As I said tweak lbias, ebias for different scenes. You don't like the remaining noise? well, then let smdegrain denoise dark lineart as well by moving the merge line above it. But by doing that then I recommend using a gamma prefilter to avoid ghosting in these parts, and possibly a denoiser prefilter as well, to even block SAD so temporal denoise is effective.

Not perfect, but if someone has a better solution Id be surprised.
Crop(6, 4, -4, -6)

a=last

#####SPATIAL DENOISE####
b=dfttest(tbsize=1,sigma=20,lsb=true)
b.ditherpost(mode=6)

#####MASK####
Lbias=80
Ebias=2
msk=mt_logic(mt_binarize(Lbias,upper=true).mt_inpand.mt_inpand,\
mt_edge("prewitt",thY1=Ebias,thY2=Ebias).mt_expand.mt_inpand,"min")

#####BACK MERGING WITH DFTTEST####
dither_merge16_8(b,a.Dither_convert_8_to_16(),msk,luma=true).ditherpost(mode=6)

#####PREFILTER#####
str=15
pre=mt_merge(medianblur(2),msk,luma=true).mt_lut("x 255 x 35 "+string(str/5)+" - / 255 "+string(str)+" - + / ^")

#####TEMPORAL DENOISE####
smdegrain(tr=3,thSAD=300,cclip=a,refinemotion=true,chroma=false,pre_custom=pre,hpad=0,vpad=0,plane=0,lsb_out=true)

#####CONTRAST####
# I strongly recommend you to change this to smoothlevels(lmode=2...) since my "local" mode is not as good as smoothlevels "lmode"
SmoothContrast(0.3,sat=0.1,median=64,local="bright",lsb=true)

Dither_resize16(646,480,kernel="spline36")
gradfun3(0.5,13,smode=1,lsb_in=true,mode=0,ampo=1.2)
maa()
aWarpSharp2(depth=3,type=1,blur=4,thresh=128,chroma=2)
http://img809.imageshack.us/img809/4558/11845.th.png (http://img809.imageshack.us/img809/4558/11845.png)


@mandarinka

hello mandarinka. I understand your point, I also see some massacres done to already good sources, and very aggresive warpsharp. And I don't like that either.

My gradfun3 line is a bit free, this is not my project so I added for the run in order to look good. In a real scenario I test to death to optimize for quality and speed.

If you analyse what I do everything is subtle and passed through masks, that's the real difference. Now, we have different thinkings about how a source should be approached. You think that the most it looks to the DVD the better (just optimizing for bitrate at encodings), my thinking is totally opposite, the most it looks or resembles to original cels, the better. Shaking, strong noise/blocking, off AR, and wrong luma range for me are distracting, but maybe not for you...

You asked me about the lineart, but remember that the OP was using AA to begin with, and I told him to don't use it if not necessary, although lineart has blocking, some aliasing and I also hinted some rainbowing.

if he doesn't want that look then it is as easier as:
a=last
str=15
pre=dfttest(tbsize=1,sigma=30,lsb=true).ditherpost(mode=6).mt_lut("x 255 x 35 "+string(str/5)+" - / 255 "+string(str)+" - + / ^")
smdegrain(tr=3,thSAD=300,contrasharp=true,refinemotion=true,pre_custom=pre)
lumamask(a,40,75)

or use your script instead.

[mandarinka]

I was targetting the original poster you with my rant, not you :) Allthough I also considered the processing you posted unnecessary. However TBH, I also didn't look into your snippet in detail, obviously.

That being said, I'm not sure about you getting back to "the original cels"... That's a problematic concept, but in any case my stance is that at least some of the texture (aka grain) needs to be retained, even if it comes from the film material and not from the cel - because the image simply doesn't look good when all the texture is smoothed out. The preference for dirty/noisy images probably has some psychologic roots but that's irelevant...

[Dogway]

It's difficult to guess since you didn't name anyone and I was aiming for cel resemblance.
It's about likes and dislikes. Nowadays you get tons of anime with no grain, just plain flat, and they are all success. In the other hand I also know of some anime which used grain as an artistic resource although this is not the case. The deshaking is not necessarily a preference, image stabilisation helps the temporal denoiser and later encoding as well, but I'm yet to know a good deshaker out there for this kind of sources.

[Overdrive80]

So, I checked out your sample and I must say I find your script totally insane. You have qutie a nice source, don't you. It has grain and it shakes but since it's old that's only natural.

What you do to the poor thing: smooth, then sharpen, then smooth, then sharpen. Then, most likely because you smoothed too bloody much, you have to actually run gradfun (debanding filter)! For your information, debandeing filters are intended to hide banding that was introduced by undithered rendering of gradients in *computer rendered* animation. people that have to use it to hide the damage they themselves caused to their sources are simply failures... (yes this criticism is for you - hand drawn animation doesn't have digitally rendered gradients, if you run a debander on such source, you it was you yourself who caused the trouble.)

In the same vein, you should really drop both your sharpeners, the look of the source's lines is perfectly good and natural in the source sample! Again, if you must sharpen because you fubared your precious source with your own damn smoothing, something is wrong on your side of things.

Why don't you try to apply *just* mild degraining (or for example the thing I posted) on the source, instead of some multilateral mutilating? Of course, if you aren't interested in how the moive/series/OVA actually looks, feel free to turn it into a damn Futurama or Powerpuff Girls or...

Forgive me for being blunt and aggressive here, but this merciless approach to "improving" video is too popular amongst people for me to stay calm...

P.S.
The thing looks pretty much like a remastered source. That's a hint that it doesn't need filtering, except whatever is completely necessary to somewhat limit the bitrate needed for encoding. As I said, just some careful touch to the grain. Best is to not remove it completely, it's best to leave some texture there, else the result looks bad.


Ohhh, yes, you are really agressive but i understand your way of thinking. Then, do you think that Blurays DBZ´s remastered of Funimation is masterwork? Because is pure grain.

I will not lie, and tell him I did not know the anime has no color gradient, and thank you to tell me. I'm just trying to make it look more attractive, which dont means correct to their knowledge or other general concepts.

@Dogway

Thanks for your script, i will test it. And forgive me if i causes some inconvenience you. ^^

[Overdrive80]

For somebody needs it (its trivial), mod of BrightDfftest for using dither:

function BrightDfttestMod(clip input, float "sbright", float "smedium", float "sdark",
\ int "th_low", int "th_med", int "th_high", int "tbsize", bool "lsb", int "mode")
{


sbright = default( sbright, 0 )
smedium = default( smedium, 0 )
sdark = default( sdark, 0 )

th_low = default( th_low, 20 )
th_med = default( th_med, 40 )
th_high = default( th_high, 100 )

lsb = default( lsb, false )
mode = default( mode, 0 )


bright = (sbright > 0 && lsb==false) ? input.dfttest(sigma = sbright,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb):\
(sbright > 0 && lsb==true) ? input.dfttest(sigma = sbright,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb).ditherpost(mode=mode) : input
medium = (smedium > 0 && lsb==false) ? input.dfttest(sigma = smedium,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb):\
(smedium > 0 && lsb==true) ? input.dfttest(sigma = smedium,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb).ditherpost(mode=mode) : input
dark = (sdark > 0 && lsb==false) ? input.dfttest(sigma = sdark,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb) :\
(sdark > 0 && lsb==true) ? input.dfttest(sigma = sdark,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb).ditherpost(mode=mode) : input

mmask = medium.levels(th_med, 1.0, th_high, 255, 0, false)
dmask = medium.levels(th_low, 1.0, th_med, 255, 0, false)

output = bright.mt_merge(medium, mmask, U=3, V=3, luma=true)
\ .mt_merge(dark, dmask, U=3, V=3, luma=true)

return output
}

Example of using: BrightDfttestMod(sbright=6.5, smedium=1, sdark=0,tbsize=1,lsb=true)

Return video without LSB.

[Mystery Keeper]

In most cases I use pure temporal denoising with motion compensation. Whether it blurs anything or not depends purely on motion estimation precision. And that is hindered by the noise itself.

[Overdrive80]

In this case, would be it right?

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-
#FILTRADO
#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-

original=last

#/*Prefiltrado*/#

pre=original.BrightDfttestMod(sbright=6.5, smedium=0.25, sdark=0,th_low=20, th_med=40, th_high=100, tbsize=1,lsb=true)

mascara=original.mt_edge("hprewitt",thY1=8, thY2=255,thc1=0,thc2=255)

prefiltrado=mt_merge(pre, original, mascara,u=2,v=2)

#Motion Compensation

super = MSuper(original, levels=1,mt=true) #Un nivel para mccompensate
super_filt= MSuper(prefiltrado, mt=true) #Todos los niveles para los vectores

fwd_vectors = MAnalyse(super_filt, isb = true, overlap=4,search=4)
bwd_vectors = MAnalyse(super_filt, isb = false, overlap=4,search=4)

fwd_compensation = MCompensate(original, super, fwd_vectors)
bwd_compensation = MCompensate(original, super, bwd_vectors)

interleave(fwd_compensation, prefiltrado, bwd_compensation)

prefiltrado = Selectevery(3,1)

#Temporal degrain

tr = 6

super2= MSuper(prefiltrado,mt=true)

multi_vec = MAnalyse (super2, multi=true, delta=tr, overlap=4,search=4)

last=prefiltrado.MDegrainN (super2, multi_vec, tr, thSAD=470, thSAD2=210,plane=4,lsb=true).ditherpost()

#Motion Compensation 2

super_filt2= MSuper(last, mt=true) #Todos los niveles para los vectores

fwd_vectors2 = MAnalyse(super_filt2, isb = true, overlap=4,search=4)
bwd_vectors2 = MAnalyse(super_filt2, isb = false, overlap=4,search=4)

fwd_compensation2 = MCompensate(original, super, fwd_vectors2)
bwd_compensation2 = MCompensate(original, super, bwd_vectors2)

interleave(fwd_compensation2, prefiltrado2, bwd_compensation2)

last = Selectevery(3,1)

My doubt is in red, original or other clip like prefitrado,prefiltrado2.

[StainlessS]

Very definitely no expert here but, would it be based off this (from mvtools docs, those marked in blue use implicit Last as first arg):


To use prefiltered clip for more reliable motion estimation, but compensate motion of not-prefiltered clip (denoising example)

AVISource("c:\test.avi") # or MPEG2Source, DirectShowSource, some previous filter, etc
# Use some denoiser (blur) or deflicker for prefiltering
prefiltered = blur(1.0)
super = MSuper(levels=1) # one level is enough for MCompensate
superfilt = MSuper(prefiltered) # all levels for MAnalyse
backward_vectors = MAnalyse(superfilt, isb = true)
forward_vectors = MAnalyse(superfilt, isb = false)
# use not-prefiltered (super) clip for motion compensation
forward_compensation = MCompensate(super, forward_vectors)
backward_compensation = MCompensate(super, backward_vectors)
# create interleaved 3 frames sequences
interleave(forward_compensation, last, backward_compensation)

DeGrainMedian() # place your preferred temporal (spatial-temporal) denoiser here

selectevery(3,1) # return filtered central (not-compensated) frames only

[Overdrive80]

Ok, if last should be first argumento then conde is fine forma me. Thanks stainless.

[GMJCZP]

Overdrive80, do you have not considered using ContraSharpening?