Anti-aliasing script

[luke83]

Originally Posted by WooWoo

Hi, I need some help. I am trying to encode to x264 using aaa() loading the latest versions of SangNom and UnFilter and importing AAA.avs (I got all the files today from the avisynth wiki). Everything looks right in the preview window for programs such as VirtualDub and MeGUI. However, when my x264 encode finishes, the height is double the size (width is the same), and the fps is halved. So instead of it being 640x480 @ 23.976 FPS, it is 640x960 @ 11.988 FPS. When I encode to XviD, everything comes out at the correct resolution and FPS using the same script. All I'm doing is loading the source file and plugins/scripts needed for AAA(), and then running AAA() in my avs script.

I then tried out each version of SangNom that Didée posted and bcn_246 tested out. They all resulted in the the same problem except for sangnom_old.dll (CRC32: 60BE3D92). This particular version worked perfectly without changing the resolution or FPS, however it reduced my FPS to a laughable ~2 FPS on my i5 quad core machine. Anybody else having this issue, or am I not loading the correct versions of everything, or what's going on exactly? Why is it changing my resolution and fps?

post your script please. I'm using AAA from a lot of time and i haven't any problem yet. The only problem is that AAA is quite slow. However is the only anti-aliasing script that works good for me.

[byme]

how do to put the nnedi3 instead of BilinearResize in the SAA?
is it possible?
I'm not able...

Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().BilinearResize(OX,OY)

Now, for the use that I do, I'm putting spline36resize

Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().spline36resize(OX,OY)

[Keiyakusha]

Not possible. nnedi3 doesn't do downscaling.

[jofori89]

Originally Posted by list

can't see any artifact there
BTW...

#FASTER ANTI ALIASING FILTER
#REQUIERED:
#    SangNom
#    mt_masktools

function FAA(clip input,int "str") {
	str = Default(str, 50)
	vid = input.IsYV12() ? input : input.ConvertToYV12()
	w=width(input)
	h=height(input)

	aa=vid.pointResize(w*2,h) . TurnLeft() . SangNom(AA=str) . TurnRight(). lanczosResize(w,h)

	return mt_merge(vid , aa , vid.mt_edge().mt_inflate() )
}

Maybe it's not the best anti aliasing filter, but 3x faster, and the first i can use in real time with some sources.

this one is good enough.

[spx]

Hi, i'm using the aaa() from wiki, and i got a question: What adds suchs a heavy noise to picture?
I've got a pretty clean animation frame, then i add aaa(Xshrp=55,Yshrp=55,chroma=false), and i get a tremendous amout of temporal noise. it changes with every frame. I'm afraid to clean it up because of detail loss.
You guys have the same results?

here's an example: http://screenshotcomparison.com/comparison/33785

at first i thought it's because of unsharp values of 55, but it adds noise on default, too...

[Seedmanc]

In regards to ediaa(), wouldn't replacing

return a.EEDI2(field=1).TurnRight().EEDI2(field=1).TurnLeft().spline36resize(a.width,a.height,-0.5,-0.5,2*a.width+.001,2*a.height+.001)

with

return a.EEDI2(field=1).spline36resize(a.width,a.height,0,-0.5,0,2*a.height+.001).TurnRight().EEDI2(field=1).TurnLeft().spline36resize(a.width,a.height,-0.5,0,2*a.width+.001,0)

be better speed- and memory-wise? So that second EEDI2 doesn't have to deal with unnecessarily 2x wider clip when it only works with the height, not width.
For me it gives a speed increase from 1 fps to 1.6 on 1080p.

[wOxxOm]

AA scripts that use turnleft and/or sangnom may be significantly sped-up with sangnom2+fturn - for better results drop the chroma plane by using Y8 colorspace during processing and copy original chroma in the end (example). Realtime AA of 1080p @ 24fps on my pc...

[may24]

Hi all,
sorry for capturing this thread , but I've a similar issue.
I got a poorly deinterlaced clip here (simple bob().SelectEven() ) with quite a lot of jaggy edges I'd like to fight.
This is not Anime material but captured from a VHS tape.
Your anti-alias-script uses SangNom, but that's for interlaced material - not ?
But my vid is already progressive encoded and now I'd like to reencode plus a bit upsizing.
So what would be the best "way to do it ?"

The original is lost only reencoded vid is here.
TIA

[real.finder]

my first function, MC DAA3

#Motion-Compensated Anti-aliasing with contra-sharpening by A.SONY
#some ideas was taken from Didée scripts daa and this http://forum.doom9.org/showthread.php?t=153219
#created because when applied daa3 to fixed scenes, it could damage some details and other issues.
# v3.55

function mcdaa3(clip input) {
sup    = input.HQdn3d().FFT3DFilter().MSuper(pel=2,sharp=1)
fv1    = sup.MAnalyse(isb=false,delta=1,DCT=2,Truemotion=false)
fv2    = sup.MAnalyse(isb=true,delta=1,DCT=2,Truemotion=true)
csaa   = input.daa3mod()
momask1 = input.MMask(fv1, kind = 1, ml=2)
momask2 = input.MMask(fv2, kind = 1, ml=3)
momask = mt_average(momask1,momask2,u=3,v=3)
mt_merge(input,csaa,momask,u=3,v=3)
}


#############################

#Anti-aliasing with contra-sharpening by Didée with some modifications, v3.55

    FUNCTION daa3mod(clip c1, val "slow", int "threads") {
avs25 = VersionNumber() < 2.60
isslod = defined(slow)

slow    = default (slow      , true)
isslob = isbool(slow)
issloi = Isint(slow)
Assert(isslob || issloi, "'slow' only accepts bool or int")
slowb  = isslob ? slow : isslod
threads = default (threads   ,    0)

rescale = slowb && issloi ? slow > -1 : slowb
upsizer = rescale ? issloi ? slow!=0 ? "BilinearResize" : "Spline36Resize" : "Spline36Resize" : nop()
downsizer = slowb ? issloi ? slow!=0 ? "BicubicResize" : "Spline36Resize" : "Spline36Resize" : nop()

c = rescale ? threads!=1 ? Eval("""try { Eval("c1."+upsizer+"mt(c1.width, c1.height*3/2,threads=threads)") } catch(error_msg) { Eval("c1."+upsizer+"(c1.width, c1.height*3/2)") }""") : Eval("c1."+upsizer+"(c1.width, c1.height*3/2)") : c1

    nn      = issloi ? slow > 1 ? c.Padding(slow==2 ? 0 : 4,4,slow==2 ? 0 : 4,4).nnedi3(field=-2,threads=threads).crop(slow==2 ? 0 : 4,4,slow==2 ? 0 : -4,-4) : c.nnedi3(field=-2,threads=threads) : c.nnedi3(field=-2,threads=threads)
 c =isyuy2(c) && avs25 ? c.Interleaved2Planar() : isyuy2(c) ? c.converttoyv16() : c
 nn=isyuy2(nn) && avs25 ? nn.Interleaved2Planar() : isyuy2(nn) ? nn.converttoyv16() : nn
    isslow4 = slowb && issloi ? slow > 3 || slow < -1 : false
    istof   = isslow4 ? c1.getparity : nop()
    seto    = isslow4 ? istof ? selecteven(nn) : selectodd(nn) : selecteven(nn)
    seto    = isslow4 ? threads!=1 ? Eval("""try { Eval("seto."+downsizer+"mt(c.width, c.height,src_top=-0.5,threads=threads)") } catch(error_msg) { Eval("seto."+downsizer+"(c.width, c.height,src_top=-0.5)") }""") : Eval("seto."+downsizer+"(c.width, c.height,src_top=-0.5)") : seto
    sebo    = isslow4 ? istof ? selectodd(nn) : selecteven(nn) : selectodd(nn)
    sebo    = isslow4 ? threads!=1 ? Eval("""try { Eval("sebo."+downsizer+"mt(c.width, c.height,src_top=0.5,threads=threads)") } catch(error_msg) { Eval("sebo."+downsizer+"(c.width, c.height,src_top=0.5)") }""") : Eval("sebo."+downsizer+"(c.width, c.height,src_top=0.5)") : sebo

    dbl     = mt_average(seto,sebo,U=3,V=3)
    dblD    = mt_makediff(c,dbl,U=3,V=3)
    shrpD   = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 11,Planar=true),U=3,V=3)
    DD      = shrpD.repair(dblD,13,Planar=true)
 fd1 = dbl.mt_adddiff(DD,U=3,V=3)

 fd2 = rescale ? threads!=1 ? Eval("""try { Eval("fd1."+downsizer+"mt(c1.width, c1.height,threads=threads"+((downsizer!="Spline36Resize") ? ",b=0.3, c=0.2" : "" )+")") } catch(error_msg) { Eval("fd1."+downsizer+"(c1.width, c1.height"+((downsizer!="Spline36Resize") ? ",b=0.3, c=0.2" : "" )+")") }""") : Eval("fd1."+downsizer+"(c1.width, c1.height"+((downsizer!="Spline36Resize") ? ",b=0.3, c=0.2" : "" )+")") : fd1
 fd3 = isyuy2(c1) && avs25 ? fd2.Planar2Interleaved() : isyuy2(c1) ? fd2.converttoyuy2() : fd2
return fd3
}

I created this script because daa3 when applied to fixed scenes, it could damage some details such as horizontal lines on the screen

like this example:- original - daa3 - mcdaa3

And here apply of daa3 and mcdaa3 on a moving scene:- original -daa3 -mcdaa3

download link (it's old now)

edit1: mcdaa3 and normal daa can deal with ifade too, In fact, the daa function is the remnants Interlaced cleaner after IVTC

edit2: now you can use it like this smam(filter=daa3mod()) , this support YUY2 and more up to date

edit3: since smam now will add daa3mod() is some cases, if you use it like this smam(filter="daa3mod()") it will be 2 daa3mod(), so you must change smam settings like this smam(filter="daa3mod()",usedaa3mod=0)
or use it as I write in "edit2" above, or smam(filter=last)

edit4: if the clip has black border on top or bottom it beter to crop it before using any daa/daa3 method since it will add some Artifacts unless you set slow to -2 or more than 0 (better but slower as you go on with the number)

[mandarinka]

Isn't it just motion adaptive, instead of motion-compensated? /Or am I reading it wrong, I'm no good at any sort of programming/.

[Pier]

Originally Posted by real.finder

my first function, MC DAA3

#Motion-Compensated Anti-aliasing with contra-sharpening by A.SONY
#some ideas was taken from Didée scripts daa and this http://forum.doom9.org/showthread.php?t=153219
#created because when applied daa3 to fixed scenes, it could damage some details and other issues.

function mcdaa3(clip input) {
sup    = input.HQdn3d.FFT3DFilter.MSuper(pel=2,sharp=1)
fv1    = sup.MAnalyse(isb=false,delta=1,DCT=2,Truemotion=false)
fv2    = sup.MAnalyse(isb=true,delta=1,DCT=2,Truemotion=true)
csaa   = input.daa3mod
momask1 = input.MMask(fv1, kind = 1, ml=2)
momask2 = input.MMask(fv2, kind = 1, ml=3)
momask = mt_average(momask1,momask2)
mt_merge(input,csaa,momask,u=3,v=3)
}


#############################

#Anti-aliasing with contra-sharpening by Didée with some modifications

    FUNCTION daa3mod(clip c1) {
c = c1.Spline36Resize(c1.width, c1.height*3/2)
    nn      = c.nnedi3(field=-2)
    dbl     = mt_average(selecteven(nn),selectodd(nn),U=3,V=3)
    dblD    = mt_makediff(c,dbl,U=3,V=3)
    shrpD   = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 11),U=3,V=3)
    DD      = shrpD.repair(dblD,13)
return dbl.mt_adddiff(DD,U=3,V=3).Spline36Resize(c1.width, c1.height)
}

I created this script because daa3 when applied to fixed scenes, it could damage some details such as horizontal lines on the screen

like this example:- original - daa3 - mcdaa3

And here apply of daa3 and mcdaa3 on a moving scene:- original -daa3 -mcdaa3

download link

Not sure, but if i remember correctly "HQdn3d()" is faster than "HQdn3d", same for every other filter.

[Kein]

Originally Posted by real.finder

my first function, MC DAA3

#Motion-Compensated Anti-aliasing with contra-sharpening by A.SONY
#some ideas was taken from Didée scripts daa and this http://forum.doom9.org/showthread.php?t=153219
#created because when applied daa3 to fixed scenes, it could damage some details and other issues.

function mcdaa3(clip input) {
sup    = input.HQdn3d.FFT3DFilter.MSuper(pel=2,sharp=1)
fv1    = sup.MAnalyse(isb=false,delta=1,DCT=2,Truemotion=false)
fv2    = sup.MAnalyse(isb=true,delta=1,DCT=2,Truemotion=true)
csaa   = input.daa3mod
momask1 = input.MMask(fv1, kind = 1, ml=2)
momask2 = input.MMask(fv2, kind = 1, ml=3)
momask = mt_average(momask1,momask2,u=3,v=3)
mt_merge(input,csaa,momask,u=3,v=3)
}


#############################

#Anti-aliasing with contra-sharpening by Didée with some modifications

    FUNCTION daa3mod(clip c1) {
c = c1.Spline36Resize(c1.width, c1.height*3/2)
    nn      = c.nnedi3(field=-2)
    dbl     = mt_average(selecteven(nn),selectodd(nn),U=3,V=3)
    dblD    = mt_makediff(c,dbl,U=3,V=3)
    shrpD   = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 11),U=3,V=3)
    DD      = shrpD.repair(dblD,13)
return dbl.mt_adddiff(DD,U=3,V=3).Spline36Resize(c1.width, c1.height)
}

I created this script because daa3 when applied to fixed scenes, it could damage some details such as horizontal lines on the screen

like this example:- original - daa3 - mcdaa3

And here apply of daa3 and mcdaa3 on a moving scene:- original -daa3 -mcdaa3

download link

edit: mcdaa3 and normal daa can deal with ifade too, In fact, the daa function is the remnants Interlaced cleaner after IVTC

Which plugins it requires?
Or, rather, do I need to just invoke the functions with empty parameters or I need to specify clip?

[StainlessS]

Which plugins it requires?
Or, rather, do I need to just invoke the functions with empty parameters or I need to specify clip?

HQDn3D, FFT3DFilter, MvTools v2, Masktools v2, nnedi3. Repair from RemoveGrain.

function mcdaa3(clip input)

Above tells you everything you need to know (call with named clip or it will use implicit Last clip).

http://avisynth.nl/index.php/External_filters

[Kein]

Originally Posted by StainlessS

HQDn3D, FFT3DFilter, MvTools v2, Masktools v2, nnedi3. Repair from RemoveGrain.

function mcdaa3(clip input)

Above tells you everything you need to know (call with named clip or it will use implicit Last clip).

http://avisynth.nl/index.php/External_filters

Thanks!

[tormento]

Originally Posted by real.finder

my first function, MC DAA3

Does it support AVS+ color formats?

If not, could you give it a version bump?

Thank you

[real.finder]

Originally Posted by tormento

Does it support AVS+ color formats?

If not, could you give it a version bump?

Thank you

it should https://github.com/realfinder/AVS-St...p/daa3MOD.avsi but as mandarinka https://forum.doom9.org/showpost.php...3&postcount=90 said it's motion adaptive (I was kinda noob back then)

but anyway, now it better to use smam(filter=daa3mod(slow=2)) or with Vinverse instead of daa3mod/daa3

[tormento]

Originally Posted by real.finder

but anyway, now it better to use smam(filter=daa3mod(slow=2)) or with Vinverse instead of daa3mod/daa3

As far as I can see they are both for combing restoration not for antialiasing. Am I wrong?

Where can I find smam?

I am just looking for a good AA for anime contents, now that they use 3D a lot and it's a mess as far as aliasing is concerned.

[real.finder]

Originally Posted by tormento

As far as I can see they are both for combing restoration not for antialiasing. Am I wrong?

yes they are for combing, sometimes in old anime the fields not matched ideally for some reason like they use an analog master or 60i Effects

like this https://i.imgur.com/f9tW8bl.png it's aliasing because of fields not matched perfectly

Where can I find smam?

as always http://forum.doom9.org/showthread.php?t=174121 or https://github.com/realfinder/AVS-Stuff

I am just looking for a good AA for anime contents, now that they use 3D a lot and it's a mess as far as aliasing is concerned

then you must look for something else like maa2 or santiag

[tormento]

Originally Posted by real.finder

then you must look for something else like maa2 or santiag

Thanks for your explanation.

At the end, there is no "modernized" AA filter?

[orchid]

Originally Posted by tormento

Thanks for your explanation.

At the end, there is no "modernized" AA filter?

There is, but as the anime community has entirely moved to vapoursynth sometime in 2017, such modernized filters remain exclusive to vapoursynth.