LSharpAAF - AntiAliasing Combo

[Leinad4Mind]

Another AntiAliasing Script. :devil: It's more a combo... a combination of all antialiasing I know. :)

And it has some Extras like FastLineDarkenMOD, Presharpening, Postsharpening and Postsmoothing. :rolleyes:

It's very alike SharpAAMCmod... :search:

It contains: Sangnom, EEDI2, maa, daa, ediaa, aaa, ssa, splinaa, supaa, naa, shaarp, aaf, nnedi, nnedi2 and antialiasing :eek:

:script: Okay... I will post it... Here is the Code:

# LSharpAAF() by Leinad4Mind
# v1.50 updated in 2011-12-06
# Requirements: awarpsharp2; EEDI2.dll; FastLineDarkenMOD.avsi; LSFmod.v1.9.avsi; MaskTools.dll; mt_masktools-26.dll; mvtools.dll;
# nnedi2.dll; RemoveGrain*.dll; Repair*.dll; SangNom.dll; UnFilter.dll
# * HD; HDS; S; SSE2; SSE3; T; TSSE2; TSSE3
#
# All AA Scripts are in this script, so there are no need to download them separately; Except some of them that you can find in AnimeIVTC, as maa, daa.
#
# This is an Anti-Aliasing Combo with some Extras like FastLineDarkenMOD, Presharpening, Postsharpening and Postsmoothing
#
# Thanks to @ Didée, thetoof, Mystery Keeper, Soulhunter, MisterHatt, martino, mf, Akirasuto, SpikeSpiegel, ScharfisBrain, LaTo & Sagekilla
#
# Example: LSharpAAF(StrDrk=18, ShPre=100, ShPost=280, SmPost=80, aatype="ediaa")
# OR: LSharpAAF(18,100,280,80,"ediaa")

FUNCTION LSharpAAF(clip a, int "StrDrk", int "ShPre", int "ShPost", int "SmPost", bool "stabilize", int "tradius", bool "MT", int "aapel", int "aaov", int "aablk", string "aatype"){
StrDrk = default(StrDrk,38) #FastLineDarkenMOD
ShPre = default(ShPre,18) #Presharpening
ShPost = default(ShPost,300) #Postsharpening
SmPost = default(SmPost,100) #Postsmoothing
stabilize= default(stabilize,false ) # Use post stabilization with Motion Compensation
tradius = default(tradius,2) # 2 = MDegrain2 / 3 = MDegrain3
MT=default(MT,false) # Use josey_wells' branch of MVTools
aapel = default(aapel,1) # accuracy of the motion estimation
# \ (Value can only be 1, 2 or 4. 1 means a precision to the pixel. 2 means a precision to half a pixel, 4 means a precision to quarter a pixel, produced by spatial interpolation (better but slower).)
aaov = default ( aaov,(a.width>1100) ? 8 : 4 ) # block overlap value (horizontal). Must be even and less than block size. (Higher = more precise & slower)
aablk = default ( aablk,(a.width>1100) ? 16 : 8 ) # Size of a block (horizontal). It's either 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.
aatype = default(aatype,"EEDI2") # Use EEDI2() ; EEDI3() ; Sangnom() ; maa() ; daa() ; ediaa() ; aaa() ; saa() ; splinaa() ; supaa() ; naa() ; shaarp() ; aaf() ; laa() ; nnedi() ; nnedi2() ; nnedi3() ; antialiasing() ; nediaa() ; naa3mod(); for anti-aliasing.

a=a.AssumeTFF()
w=width(a)
h=height(a)
m=mt_logic(a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").mt_lut("x 128 / 0.86 ^ 255 *")

preaa=(ShPre==0 && StrDrk==0) ? a : (ShPre==0) ? a.fastlinedarkenmod(thinning=0, strength=StrDrk) : (StrDrk==0) ? a.lsfmod(defaults="slow",preblur="ON",strength=ShPre)
\ : a.fastlinedarkenmod(thinning=0, strength=StrDrk).lsfmod(defaults="slow",preblur="ON",strength=ShPre)

aa=(aatype=="EEDI2") ? preaa.TurnLeft().EEDI2().TurnRight().EEDI2().spline36resize(w,h,0.5,-0.5,2*w+.001,2*h+.001)
\ : (aatype=="EEDI3") ? preaa.TurnLeft().EEDI3().TurnRight().EEDI3().spline36resize(w,h)
\ : (aatype=="Sangnom") ? preaa.spline64resize(w*2,h*2).TurnLeft().SangNom(aa=255).TurnRight().SangNom(aa=255).spline36resize(w,h)
\ : (aatype=="maa") ? preaa.maa() : (aatype=="daa") ? preaa.daa() : (aatype=="ediaa") ? preaa.ediaa() : (aatype=="aaa") ? preaa.aaa()
\ : (aatype=="saa") ? preaa.SAA() : (aatype=="splinaa") ? preaa.splinaa() : (aatype=="supaa") ? preaa.supaa() : (aatype=="naa") ? preaa.naa()
\ : (aatype=="shaarp") ? preaa.shaarp() : (aatype=="aaf") ? preaa.aaf(aam=-0.7,rep=true) : (aatype=="laa") ? preaa.laa(96, 10, true, 2.0)
\ : (aatype=="nnedi") ? preaa.NNEDI(dh=true,field=1).TurnRight().NNEDI(dh=true,field=1).TurnLeft().Spline36Resize(w,h)
\ : (aatype=="nnedi2") ? preaa.NNEDI2(dh=true,field=1,nsize=2,qual=2).TurnRight().NNEDI2(dh=true,field=1,nsize=2,qual=2).TurnLeft().Spline36Resize(w,h,0.5,-0.5,2*w+.001,2*h+.001)
\ : (aatype=="nnedi3") ? preaa.NNEDI3(dh=true,field=1,nsize=2,qual=2).TurnRight().NNEDI3(dh=true,field=1,nsize=2,qual=2).TurnLeft().Spline36Resize(w, h)
\ : (aatype=="antialiasing") ? preaa.antialiasing()
\ : blankclip(pixel_type="YV12").subtitle("Please use Sangnom, EEDI2, EEDI3, maa, daa, ediaa, aaa, ssa, splinaa, supaa, naa, shaarp, aaf, laa,
nnedi, nnedi2, nnedi3, or antialiasing for aatype")

postsh=(ShPost==0 && SmPost==0) ? aa : aa.LimitedSharpenFaster(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost)

merged=mt_merge(a,postsh,m,Y=3,U=3,V=3)

sD=mt_makediff(a,merged)

asuper= a.MSuper(pel=aapel)
sDsuper = sD.MSuper(pel=aapel, levels=1)

fv1 = tradius>=1 ? asuper.MAnalyse(isb=false,delta=1,overlap=aaov,blksize=aablk) : nop()
bv1 = tradius>=1 ? asuper.MAnalyse(isb=true, delta=1,overlap=aaov,blksize=aablk) : nop()
fv2 = tradius>=2 ? asuper.MAnalyse(isb=false,delta=2,overlap=aaov,blksize=aablk) : nop()
bv2 = tradius>=2 ? asuper.MAnalyse(isb=true, delta=2,overlap=aaov,blksize=aablk) : nop()
fv3 = tradius==3 ? asuper.MAnalyse(isb=false,delta=3,overlap=aaov,blksize=aablk) : nop()
bv3 = tradius==3 ? asuper.MAnalyse(isb=true, delta=3,overlap=aaov,blksize=aablk) : nop()

_thSAD = 600
_idx = 13

mbv1 = a.MVAnalyse(isb=true, delta=1,idx=_idx)
mbv2 = a.MVAnalyse(isb=true, delta=2,idx=_idx)
mfv2 = a.MVAnalyse(isb=false,delta=2,idx=_idx)
mfv1 = a.MVAnalyse(isb=false,delta=1,idx=_idx)
allv = (MT==false) ? a : a.MVAnalyseMulti(refframes=tradius,idx=_idx)

sDD = (MT) ? sD.MVDegrainMulti(allv,idx=14) : sD.MVDegrain2(mbv1,mfv1,mbv2,mfv2,idx=14)
sDD2 = tradius==1 ? sDD.MDegrain1(sDsuper,bv1,fv1,thSAD=_thSAD) : tradius==2 ? sDD.MDegrain2(sDsuper,bv1,fv1,bv2,fv2,thSAD=_thSAD)
\ : sDD.MDegrain3(sDsuper,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=_thSAD)

reduc = 0.4
sDD2 = mt_lutxy(sD,sDD2,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD2,1.0-reduc)

return stabilize ? a.mt_makediff(sDD2,U=2,V=2) : merged
}

#Simple Anti-aliasing by Soulhunter
FUNCTION saa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().BilinearResize(OX,OY)
CP ? Last : MergeChroma(Clp)
Return(Last) }

#Normal Anti-aliasing by Didée
FUNCTION antialiasing(clip orig,int "th_luma",int "th_croma",string "type",int "aath") {

# "th_luma" & "th_croma" are the edge detection thres.: lower values=more edges filtered
# "type" is the matrix used for edge detection: with "sobel" (default) only the
# hi-contrast edges, where artefacts are more noticeable, are filtered. If you want
# to test other matrices, read the MaskTools Guide for more info.
# "aath" = anti-aliasing strenght (default should be fine)

th_luma = Default(th_luma, 20)
th_croma = Default(th_croma, 20)
type = Default(type, "sobel")
aath = Default(aath, 48)
ox = orig.width
oy = orig.height
dx = orig.width * 2
dy = orig.height * 2
clp = orig.IsYV12() ? orig : orig.ConvertToYV12()

a=clp
b=clp.Lanczos4Resize(dx,dy).TurnLeft().SangNom(aa=aath).TurnRight().SangNom(aa=aath) \
.LanczosResize(ox,oy)
c=clp.EdgeMask(th_luma,th_luma,th_croma,th_croma,type)
MaskedMerge(a,b,c) }

# Written by MisterHatt, based off of Soulhunter's SAA() and martino's excellent MAA(), and is slightly
# faster at times for some reason or another. These functions work by generally supersampling, usually with
# nnedi2, and then running a deinterlacer (sangnom and nnedi2 respectively) to get rid of most jaggies.
# I have no idea how destructive these are on whatever random thing people care to throw them at.
# For large supersamples, avs2yuv bitches for no real reason and requires assumeframebased() in your script.
# Requires nnedi2, sangnom, o9k hours of your life.
FUNCTION splinaa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().Spline36Resize(OX,OY)
CP ? Last : MergeChroma(Clp)

Return(Last) }

FUNCTION supaa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.nnedi2_rpow2(rfactor=SS,cshift="spline36resize",qual=3).SangNom().TurnRight()
\ .SangNom().TurnLeft().Spline36Resize(OX,OY)
CP ? Last : MergeChroma(Clp)

Return(Last) }

FUNCTION naa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.nnedi3_rpow2(rfactor=SS,cshift="spline36resize",qual=2).nnedi3(nns=2).TurnRight().nnedi3(nns=2).TurnLeft().Spline36Resize(OX,OY)
CP ? Last : MergeChroma(Clp)

Return(Last) }

FUNCTION shaarp(clip input, int "mask", int "type") {
mask = Default(mask,1)
type = Default(type,1)
Assert(mask == 1 || mask == 2, "Please use mask = 1 or 2")
Assert(type == 1 || type == 2, "Please use type = 1 or 2")
aa_clip = (type == 1) ? input.spline36Resize(width(input)*2,height(input)*2) : input.nnedi2_rpow2(rfactor=2,qual=3)
aa_clip = aa_clip.TurnLeft().nnedi2().TurnRight().nnedi2().spline36Resize(width(input),height(input)).MergeChroma(input)
mask = (mask==1) ? input.mt_edge("sobel",7,7,5,5).mt_inflate() : input.mt_edge("roberts",0,4,0,4).mt_inflate()

return mt_merge(input,aa_clip,mask) }

function aaf(clip input, float "aam", int "aay", "aax", bool "rep")
{
input = input.isYV12() ? input : input.converttoyv12()

aam = default(aam, -0.6)
aar = aam<0 ? (aam-1)*0.25 : (aam+1)*0.25
aay = default(aay, 28)
aax = default(aax, aay)
rep = default(rep, true)

sx = width(input)
sy = height(input)

aa = aar<0 ? input.LanczosResize(sx,int(sy*abs(aar))*4) :
\ aar==0.5 ? input.Pointresize(sx*2, sy*2) :
\ input.LanczosResize(int(sx*abs(aar)*4),int(sy*abs(aar))*4)
aa = aay>0 ? aa.SangNom(aa=aax) : input
aa = aar<0 && aax>0 ? aa.LanczosResize(int(sx*abs(aar))*4,sy) : aa
aa = aax>0 ? aa.turnright().SangNom(aa=aax).Turnleft() : aa
aa = aa.LanczosResize(sx,sy)

return rep==true ? aa.repair(input,18) : aa
}

### LaTo Antialiasing Script v2
### Need: asharp.dll & sangnom.dll
function LAA(clip input, int "strength", int "threshold", bool "sharp", float "ss") {

strength = default(strength, 96)
threshold = default(threshold, 10)
sharp = default(sharp, true)
ss = default(ss, 2.0)
aastr = int(strength) /2
asthr = float(strength) /100
ox = input.width
oy = input.height
ox2 = round(ox*ss/8)*8
oy2 = round(oy*ss/8)*8

process = input.spline36resize(ox2,oy2).turnleft().sangnom(aastr).turnright().sangnom(aastr)
process = ( sharp == true ) ? process.asharp(asthr,0,0).spline36resize(ox,oy) : process.spline36resize(ox,oy)
mask = mt_average(input,process,u=1,v=1).mt_edge(thy1=threshold,thy2=threshold,u=1,v=1)
output = mt_merge(input,process,mask,u=2,v=2)

return ( output )
}

function NediAA(clip c)
{ c.nnedi3(field=-2)
merge(selecteven(),selectodd()) }

FUNCTION naa3mod(Clip Clp,Bool "CP")
{
OX = Clp.Width()
OY = Clp.Height()
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clpnn1 = Clp.nnedi3(1, dh=true,nsize=2,qual=2,U=CP,V=CP).nnedi3(0,nsize=2,qual=2,U=CP,V=CP)
Clpy1 = Clpnn1.Spline36Resize(OX,OY,0,-0.5,OX,OY*2)
Clps1 = CP ? Clpy1.MergeChroma(Clpnn1.Spline36Resize(OX,OY,0,-1,OX,OY*2)) : Clpy1.MergeChroma(Clp)
Clpnn2 = Clps1.turnright().nnedi3(1, dh=true,nsize=2,qual=2,U=CP,V=CP).nnedi3(0,nsize=2,qual=2,U=CP,V=CP)
Clpy2 = Clpnn2.Spline36Resize(OY,OX,0,-0.5,OY,OX*2)
CP ? Clpy2.MergeChroma(Clpnn2.Spline36Resize(OY,OX,0,-1,OY,OX*2)).turnleft() : Clpy2.turnleft().MergeChroma(Clp)
Return(Last)
}


Here is all the require dll or extra functions: http://www.megaupload.com/?d=HQ7NLMW0

If u want to test it. Or improve the code, you're welcome :thanks:

[Frank K Abbott]

So this is an all-in-one anti-aliasing solution script? And how does it compare to other anti-aliasing methods that have been out?

[Leinad4Mind]

Well, it has 5 new methods: splinaa, supaa, naa, shaarp and aaf. And it's too an denoiser. You can apply sharpening and it has the LSFMod included to sharp and a line darkening to darken the lines as u wish.
And this AIO includes SAA by soulhunter and others antialiasing by didée that can be lost in some pages of doom9. So, this is good for everyone to not forget about the good AA we already have.

And I think it can be helpful. Because an AIO is always good. xD

Best Regards.

[markanini]

Cool, all AA functions/filters in place. :)

[osgZach]

Damn, if only we could get x64 version of all the required plugins

[thanhtu5013]

@Leinad4Mind.

I made my script and it works on VirtualDub and MeGui but unfortunately there has no effects in my video... :confused:
There has still a lot of aliasing... :eek:

Watch my screens.

- Original AVI :
http://img697.imageshack.us/img697/1253/screenoriginal.png

- My script :
http://img21.imageshack.us/img21/636/screenretouched.png


My code:

AVISOURCE("C:\GTA IV_ENB Series 0.079 SORA - 02 - It'S Your Call_1080p.avi")

FUNCTION LSharpAAF(clip orig, int "StrDrk", int "ShPre", int "ShPost", int "SmPost", string "aatype"){
StrDrk = default(StrDrk,38) #FastLineDarkenMOD
ShPre = default(ShPre,80) #Presharpening
ShPost = default(ShPost,300) #Postsharpening
SmPost = default(SmPost,100) #Postsmoothing
aatype = default(aatype,"EEDI2") # Use EEDI2() or Sangnom() or maa() or daa() or ediaa() or aaa() or saa() or splinaa() or supaa() or naa() or shaarp() or nnedi() or nnedi2() or antialiasing() for anti-aliasing.
a=orig.AssumeTFF()
w=width(a)
h=height(a)
m=logic(a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale().levels(0,0.8,128,0,255,false)
b=a.fastlinedarkenmod(thinning=0, strength=StrDrk)
b=(aatype=="EEDI2") ? b.TurnLeft().EEDI2().TurnRight().EEDI2().spline64resize(w,h,0.5,-0.5,2*w+.001,2*h+.001) : (aatype=="Sangnom") ? b.spline36resize(w*2,h*2).TurnLeft().SangNom(aa=255).TurnRight().SangNom(aa=255).spline36resize(w,h) : (aatype=="maa") ? b.maa() : (aatype=="daa") ? b.daa() : (aatype=="ediaa") ? b.ediaa() : (aatype=="aaa") ? b.aaa() : (aatype=="saa") ? b.SAA() : (aatype=="splinaa") ? b.splinaa() : (aatype=="supaa") ? b.supaa() : (aatype=="naa") ? b.naa() : (aatype=="shaarp") ? b.shaarp() : (aatype=="nnedi") ? b.NNEDI(dh=true,field=1).TurnRight().NNEDI(dh=true,field=1).TurnLeft().Spline36Resize(w,h) : (aatype=="nnedi2") ? b.nnedi2(dh=true,field=1) : (aatype=="antialiasing") ? b.antialiasing() : blankclip(pixel_type="YV12").subtitle("Please use Sangnom, EEDI2, maa, daa, ediaa, aaa, ssa, splinaa, supaa, naa, shaarp, nnedi, nnedi2 or antialiasing for aatype")
b=b.LSFMod(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost)
mt_merge(a,b,m,Y=3,U=3,V=3)
sD=mt_makediff(a,last)
bv1 = a.MVAnalyse(isb=true, delta=1,idx=13)
bv2 = a.MVAnalyse(isb=true, delta=2,idx=13)
fv2 = a.MVAnalyse(isb=false,delta=2,idx=13)
fv1 = a.MVAnalyse(isb=false,delta=1,idx=13)
sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14)
reduc = 0.4
sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc)
a.mt_makediff(sDD,U=2,V=2)
return last
}

FUNCTION ediaa(clip a) {
return a.EEDI2(field=1).TurnRight().EEDI2(field=1).TurnLeft().spline36resize(a.width,a.height,-0.5,-0.5) }

FUNCTION daa(clip c) {
nn = c.nnedi2(field=-2)
dbl = mt_average(selecteven(nn),selectodd(nn),U=3,V=3)
dblD = mt_makediff(c,dbl,U=3,V=3)
shrpD = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 11),U=3,V=3)
DD = shrpD.repair(dblD,13)
return dbl.mt_adddiff(DD,U=3,V=3) }

FUNCTION saa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().BilinearResize(OX,OY)
CP ? Last : MergeChroma(Clp)
Return(Last) }

FUNCTION maa(clip input, int "mask") {
mask = input.mt_edge("sobel",7,7,5,5).mt_inflate()
aa_clip=input.spline36Resize(width(input)*2,height(input)*2).TurnLeft().SangNom().TurnRight().SangNom().spline36Resize(width(input),height(input)).MergeChroma(input)
return mt_merge(input,aa_clip,mask) }

FUNCTION AAA(clip clp, int "Xres", int "Yres", int "Xshrp", int "Yshrp",
\ int "US", int "DS", bool "chroma")
{
clp = clp.isYV12() ? clp : clp.ConvertToYV12()
ox = clp.width
oy = clp.height
Xres = default(Xres, ox)
Yres = default(Yres, oy)
us = default(us, 1)
ds = default(ds, 2)
Xshrp = default(Xshrp, 15)
Yshrp = default(Yshrp, 15)
chroma = default(chroma, false)

us==0 ? clp.PointResize(ox*2,oy*2) : clp.LanczosResize(ox*2,oy*2)

TurnLeft()
SangNom()

TurnRight()
SangNom()

ds==0 ? BilinearResize(Xres,Yres) :
\ ds==1 ? BicubicResize(Xres,Yres) :
\ LanczosResize(Xres,Yres)

Unfilter(Xshrp,Yshrp)

chroma ? MergeChroma(clp.Lanczosresize(Xres,Yres)) : last }

FUNCTION antialiasing(clip orig,int "th_luma",int "th_croma",string "type",int "aath") {

th_luma = Default(th_luma, 20)
th_croma = Default(th_croma, 20)
type = Default(type, "sobel")
aath = Default(aath, 48)
ox = orig.width
oy = orig.height
dx = orig.width * 2
dy = orig.height * 2
clp = orig.IsYV12() ? orig : orig.ConvertToYV12()

a=clp
b=clp.Lanczos4Resize(dx,dy).TurnLeft().SangNom(aa=aath).TurnRight().SangNom(aa=aath) \
.LanczosResize(ox,oy)
c=clp.EdgeMask(th_luma,th_luma,th_croma,th_croma,type)
MaskedMerge(a,b,c) }

FUNCTION splinaa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().Spline36Resize(OX,OY)
CP ? Last : MergeChroma(Clp)

Return(Last) }

FUNCTION supaa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.nnedi2_rpow2(rfactor=SS,cshift="spline36resize",qual=3).SangNom().TurnRight()
\ .SangNom().TurnLeft().Spline36Resize(OX,OY)
CP ? Last : MergeChroma(Clp)

Return(Last) }

FUNCTION naa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.nnedi2_rpow2(rfactor=SS,cshift="spline36resize",qual=3).nnedi2(qual=3).TurnRight()
\ .nnedi2(qual=3).TurnLeft().Spline36Resize(OX,OY)
CP ? Last : MergeChroma(Clp)

Return(Last) }

FUNCTION shaarp(clip input, int "mask", int "type") {
mask = Default(mask,1)
type = Default(type,1)
Assert(mask == 1 || mask == 2, "Please use mask = 1 or 2")
Assert(type == 1 || type == 2, "Please use type = 1 or 2")
aa_clip = (type == 1) ? input.spline36Resize(width(input)*2,height(input)*2) : input.nnedi2_rpow2(rfactor=2,qual=3)
aa_clip = aa_clip.TurnLeft().nnedi2().TurnRight().nnedi2().spline36Resize(width(input),height(input)).MergeChroma(input)
mask = (mask==1) ? input.mt_edge("sobel",7,7,5,5).mt_inflate() : input.mt_edge("roberts",0,4,0,4).mt_inflate()

return mt_merge(input,aa_clip,mask) }

[Didée]

unfortunately there has no effects in my video
Medicine doesn't help when it's just lying in the shelf. You need to ingest it.

Wrong:
1.) Have a clip
2.) Define a function
some_clip

function foo(clip c, param p) { blablabla }

This does nothing. The "function" statement does only create a function. After a function has been defined, it still has to be used.


Correct:
1.) Define a function
2.) Have a clip
3.) Use function on the clip

function foo(clip c, param p) { blablabla }

some_clip

foo()

[thanhtu5013]

OK.
I understand what you say Didée.

So I have to import the script of Leinad4Mind at first.
then adding my video source (AVISOURCE("C:\...")).

But the last step I don't understand...
I have really to write "foo()" at last ?

[Didée]

No. "Foo" is frequently used as a placeholder for "[use whatever name does fit here]".

So, instead of "foo()", you would call

either

LSharpAAF()

or

ediaa()

or

daa()

or

saa()

or

AAA()

or

antialias()

or

splinaa()

or

supaa()

or

naa()

or

shaarp()

[thanhtu5013]

Thanks a lot Didée. :)

I tried all functions. Every of them works except "antialias()".
"ediaa()" & "saa()" give me the best results.

Unfortunately, I still have a bit of aliasing.

Does it possible to make antialiasing filter works stronger ?
( Maybe it could make my video blurrer but anyway I want to try and I'll take a decision later. )

Sorry if I ask stupid questions. :p
I'm a noobs with Avisynth. :helpful:

[Great Dragon]

thanhtu5013, why don't you try an in-game AA option? I'm sure it's more powerful that AA-scripts for Avisynth.
Leinad4Mind, is it any benefits to SharpAAMCmod?

[cretindesalpes]

Does it possible to make antialiasing filter works stronger ?

You can run it multiple times. For example, in the following script you can select the strength for horizontal and vertical antialiasing. Set strh and strv between 0 and 3, over this value there is not much improvement and it generates artifacts.

# santiag v1.6
# Simple antialiasing

# This program is free software. It comes without any warranty, to
# the extent permitted by applicable law. You can redistribute it
# and/or modify it under the terms of the Do What The Fuck You Want
# To Public License, Version 2, as published by Sam Hocevar. See
# http://sam.zoy.org/wtfpl/COPYING for more details.

# type = "nnedi3", "eedi2", "eedi3" or "sangnom"
# "eedi3" requires v0.9.2 or later
# "sangnom" requires sangnom2
Function santiag (clip c, int "strh", int "strv", string "type", int "nns", int "aa", int "aac", int "threads", int "nsize", int "vcheck", int "fw", int "fh", bool "halfres", string "scaler_post", int "maskt", string "typeh", string "typev")
{
strh = Default (strh, 1)
strv = Default (strv, 1)
type = Default (type, "nnedi3")
typeh = Default (typeh, type)
typev = Default (typev, type)
halfres = Default (halfres, false)
scaler_post = Default (scaler_post, "Spline36Resize")

c

w = Width ()
h = Height ()
fwh = (strv < 0) ? fw : w
fhh = (strv < 0) ? fh : h

(strh >= 0) ? santiag_dir (strh, typeh, halfres, scaler_post, nns=nns, aa=aa, aac=aac, threads=threads, nsize=nsize, vcheck=vcheck, maskt=maskt, fw=fwh, fh=fhh) : last
(strv >= 0) ? TurnLeft ().santiag_dir (strv, typev, halfres, scaler_post, nns=nns, aa=aa, aac=aac, threads=threads, nsize=nsize, vcheck=vcheck, maskt=maskt, fw=fh, fh=fw ).TurnRight () : last

fwx = Default (fw, w)
fhx = Default (fh, h)
(strh < 0 && strv < 0) ? Eval (scaler_post + " (fwx, fhx)") : last
}

Function santiag_dir (clip c, int strength, string type, bool halfres, string scaler_post, int "nns", int "aa", int "aac", int "threads", int "nsize", int "vcheck", int "maskt", int "fw", int "fh")
{
c
w = Width ()
h = Height ()
fw = Default (fw, w)
fh = Default (fh, h)

multi = (IsYUV () && ! IsYV12 ())
c_y = (multi ) ? ConvertToYV12 () : last
c_u = (multi && ! IsY8 ()) ? UToY ().ConvertToYV12 () : last
c_v = (multi && ! IsY8 ()) ? VToY ().ConvertToYV12 () : last
c_y = c_y.santiag_stronger (strength, type, halfres, multi, nns=nns, aa=aa, aac=aac, threads=threads, nsize=nsize, vcheck=vcheck, maskt=maskt)
c_u = (multi && ! IsY8 ()) ? c_u.santiag_stronger (strength, type, halfres, true , nns=nns, aa=aa, aac=aac, threads=threads, nsize=nsize, vcheck=vcheck, maskt=maskt) : c_u
c_v = (multi && ! IsY8 ()) ? c_v.santiag_stronger (strength, type, halfres, true , nns=nns, aa=aa, aac=aac, threads=threads, nsize=nsize, vcheck=vcheck, maskt=maskt) : c_v
(multi && ! IsY8 ()) ? YToUV (c_u, c_v, c_y)
\ : IsY8 () ? c_y.ConvertToY8 ()
\ : c_y

Eval (scaler_post + " (fw, fh, src_left=0, src_top=(halfres)?0:0.5, src_width=Width (), src_height=Height ())")
}

Function santiag_stronger (clip c, int strength, string type, bool halfres, bool lumaonly, int "nns", int "aa", int "aac", int "threads", int "nsize", int "vcheck", int "maskt")
{
strength = (strength < 0) ? 0 : strength
field = strength % 2
dh = (strength <= 0 && ! halfres)

(strength > 0) ? c.santiag_stronger (strength - 1, type, halfres, lumaonly, nns=nns, aa=aa, aac=aac, threads=threads, nsize=nsize, vcheck=vcheck, maskt=maskt) : c

(type == "nnedi3" ) ? santiag_nnedi3 (dh, field, lumaonly, nns=nns, threads=threads, nsize=nsize)
\ : (type == "eedi2" ) ? santiag_eedi2 (dh, field)
\ : (type == "eedi3" ) ? santiag_eedi3 (dh, field, lumaonly, nns=nns, threads=threads, nsize=nsize, vcheck=vcheck, maskt=maskt)
\ : (type == "sangnom") ? santiag_sangnom (dh, field, aa=aa, aac=aac)
\ : Assert (false, "Santiag: unexpected value for type.")
}

Function santiag_nnedi3 (clip c, bool dh, int field, bool lumaonly, int "nns", int "threads", int "nsize")
{
c.nnedi3 (dh=dh, field=field, nns=nns, threads=threads, nsize=nsize, u=!lumaonly, v=!lumaonly)
}

Function santiag_eedi2 (clip c, bool dh, int field)
{
c
w = Width ()
h = Height ()
(dh) ? last : PointResize (w, h / 2, 0, 1-field, w, h)
eedi2 (field=field)
}

# maskt = 24
Function santiag_eedi3 (clip c, bool dh, int field, bool lumaonly, int "nns", int "threads", int "nsize", int "vcheck", int "maskt")
{
chroma = (lumaonly) ? 1 : 3
sclip = c.santiag_nnedi3 (dh, field, lumaonly, nns=nns, threads=threads, nsize=nsize)
mclip_flag = (Defined (maskt) && maskt > 0)

mclip = (mclip_flag && dh) ? sclip.BicubicResize (c.Width (), c.Height (), src_top=0.5-field) : sclip
mclip = (mclip_flag)
\ ? mclip.mt_edge (mode="1 2 1 0 0 0 -1 -2 -1 1", thY1=0, thY2=255, u=chroma, v=chroma)
\ .mt_binarize (threshold=maskt, y=3, u=chroma, v=chroma)
\ .mt_expand (y=3, u=chroma, v=chroma)
\ : santiag_undef ()

c.eedi3 (dh=dh, field=field, threads=threads, vcheck=vcheck, sclip=sclip, mclip=mclip, u=!lumaonly, v=!lumaonly)
}

Function santiag_sangnom (clip c, bool dh, int field, int "aa", int "aac")
{
c
w = Width ()
h = Height ()
(dh) ? Spline36Resize (w, h * 2, 0, -0.25, w, h) : last
SangNom2 (order=field, aa=aa, aac=aac)
}

Function santiag_undef () { }

NNEDI3 is a CPU hog but gives excellent results. Replace it with NNEDI2 if it takes too long. Here (http://img688.imageshack.us/img688/5742/90871764.png) is an example of santiag (3, 3) with the picture (http://img697.imageshack.us/img697/1253/screenoriginal.png) you linked on a previous message.

[2Bdecided]

NNEDI3 is a CPU hog but gives excellent results. Replace it with NNEDI2 if it takes too long. Here (http://img688.imageshack.us/img688/5742/90871764.png) is an example of santiag (3, 3) with the picture (http://img697.imageshack.us/img697/1253/screenoriginal.png) you linked on a previous message.Very impressive.

Though the tree looks rather strange now.

Cheers,
David.

[markanini]

Is it wise to believe that needi2 and eedi2 are the most effective for AA or could SangNom and unfilter serve as acomplement?

Here's an image I'm trying to improve:
http://homepages.ihug.co.nz/~online/nocaster/blpic-s.jpg

I want to resize x2 and AA.

With:
nnedi3_rpow2(rfactor=2,cshift="Spline36Resize",nsize=1,qual=1).nnedi3(nsize=1,qual=1).turnright().needi3(nsize=1,qual=1).turnleft()

I get:
http://img695.yfrog.com/img695/7528/blpics0422x0562.jpg

[thanhtu5013]

thanhtu5013, why don't you try an in-game AA option? I'm sure it's more powerful that AA-scripts for Avisynth.
Because if I activate AA when I'm capturing during my plays, the framerate fall a lot.
( Under 9-10 fps so it's not possible to make a video )

I prefer playing without and apply a one through AviSynth.

You can run it multiple times. For example, in the following script you can select the strength for horizontal and vertical antialiasing. Set strh and strv between 0 and 3, over this value there is not much improvement and it generates artifacts.

NNEDI3 is a CPU hog but gives excellent results. Replace it with NNEDI2 if it takes too long. Here (http://img688.imageshack.us/img688/5742/90871764.png) is an example of santiag (3, 3) with the picture (http://img697.imageshack.us/img697/1253/screenoriginal.png) you linked on a previous message.
It's exactly what I need.
Congratulations ! You're my man ! :)

Unfortunately, it doesn't work with me... :confused:
Maybe I made a mistake on my script or I don't have the required plugin.

Function santiag (clip c, int "strh", int "strv")
{
strh = Default (strh, 1)
strv = Default (strv, 1)

c
santiag_dir (strh)
TurnLeft ()
santiag_dir (strv)
TurnRight ()
}

Function santiag_dir (clip c, int strength)
{
c
w = Width ()
h = Height ()
santiag_stronger (strength)
Spline36Resize (w, h, 0, -0.5, w, h * 2)
}

Function santiag_stronger (clip c, int strength)
{
c
(strength <= 0)
\ ? NNEDI3 (dh=true, field=0)
\ : santiag_stronger (strength - 1)
\ . NNEDI3 (dh=false, field=(strength % 2))
}

AVISOURCE("C:\GTA IV_ENB Series 0.079 SORA - 02 - It'S Your Call_1080p.avi")
ConvertToYV12

santiag()

[thanhtu5013]

I've just watched on my AviSynth plugins folder. I have nnedi2.dll but I don't have nnedi3.dll...

Unfortuantely, I can't find it on the web.
Someone could send me it ? :)

[thanhtu5013]

I've just tried with nnedi2.dll. It works nice.
I'm still seeking for nnedi3.dll

[cretindesalpes]

NNEDI3 is here (http://forum.doom9.org/showthread.php?p=1388290#post1388290).

[markanini]

I upped NNEDI3 qual and nszie parameters to 2 and prescrn=false and didnt get much better results:
http://img526.yfrog.com/img526/5739/blpics0422x0562.png
Note the artifacts in the neck area of the guitar.

[thanhtu5013]

I've just tried with Nnedi2... 0,74FPS.
With Nnedi3... Forget it ! :p
Maybe in 2013, it will finish... :D

Anyway, I prefer personally Nnedi2 that blurring less the video than Nnedi3 and keeps mode details than Nnedi3.

Thanks to everyone. :)
Specially to cretindesalpes.

P.S: Your pseudo sounds french, you're french like me, right ? :p

[cretindesalpes]

I upped NNEDI3 qual and nszie parameters to 2 and prescrn=false and didnt get much better results
These antialiasing techniques (using edge-directed interpolation) focus on smoothing lines and connecting them. They cannot improve Moiré patterns, they make them even worse because the ghost lines become real.

Your pseudo sounds french, you're french like me, right ? :p
Saperlipopette, je suis démasqué !

[thanhtu5013]

I have a last question.

Does it possible to make nnedi2 filter more efficient except changing strength values ?

With strh=2 and strv=2, the filter works fine but there has still a bit of aliasing.
It works like a 2x Antialiasing. I'm looking for something like 4x or 8x antialiasing.
Changing strh and strv values doesn't modify anything about that.

@cretindesalpes

Ton filtre marche très bien mais il y'a encore un peu de crénelage.
Changer les valeurs de strh et strv jouent sur le floutage ou le détail mais ne résout pas mon problème.
Peux tu me dire comment je dois modifier mon script pour qu'il applique un Antialiasing 4x ou 8x (si tu joues aux jeux, tu comprends ce que je veux dire) ?

[2Bdecided]

Here's an image I'm trying to improve:
http://homepages.ihug.co.nz/~online/nocaster/blpic-s.jpgCan't you ask for a bigger copy?

Use TinEye if you can't remember where you got it from.

(sorry if that's a stupid suggestion, but you're never going to get the guitar strings on the neck to look right from that source - the aliasing is too bad).

Cheers,
David.

[markanini]

Youre right about tineye 2Bd, its a great tool. I my case it didnt help but that's the nature of some things...
Do you know any tricks that might help in simmilar situations beside lowpass via spline36resize?

[Shon]

Not downloaded http://www.megaupload.com/?d=LCFXGZL0

[Leinad4Mind]

The link works dude! xD

[mastrboy]

Nice idea, and a great resource.

I kind of miss some documentation of the different AA types, with slowness and quality comparison.

I ran some tests myself on a avs script im working on for a anime dvd rip.

There is alot of others filters also in this script, but here is some fps and cpu data on a 720x480 source. Encode was directly with x264.exe on a i7 860 with disabled HT:

maa fps: 4,9 | cpu: 30%
ediaa fps: 4.6 | cpu: 99%
aaa fps: 5,7 | cpu: 40%
antialiasing fps: 5,6 | cpu: 40%
splinaa fps: 6,0 | cpu: 35%
supaa fps: 2,2 | cpu: 60%
naa fps: 1,9 | cpu: 75%
shaarp fps: 4,2 | cpu: 55%


(the FPS and CPU load is average over 5min)

[Leinad4Mind]

Thanks mastrboy. Hope you guys like it. ^^

I have update today the script with 1 more GOOD antialiasing: "aaf". The new avsi is attach.

Best Regards

[Leinad4Mind]

New version of it. I've found a bug in Presharpening, I've forget to finish it xD. And I've improve the code a lot ^^.

I've added (bool "stabilize", int "tradius", bool "MT", int "aapel", int "aaov", int "aablk"). tradius can be 1, 2 or 3 for choosing MDegrain1,2 or 3. MT option can be turned ON.
And added some code for motion estimation.
Added NEEDI3 too as AA.

And I have remove some antialiasers from the avs, like maa and daa, 'cause they are in AnimeIVTC... so... But I have added in the Requirements Pack the AnimeIVTC.avsi.

Best Regards and check the 1st page ;)

[supernater]

I get the following error when trying to run LSharpAAF()...

Avisynth script error.
Script Error, there is no function named "MSuper"
(LSharpAAF.v1.2.avsi, line 59)

I copied the required plugins into my plugins folder. Is there something else I'm doing wrong?

[poisondeathray]

NNEDI3 is a CPU hog but gives excellent results. Replace it with NNEDI2 if it takes too long. Here (http://img688.imageshack.us/img688/5742/90871764.png) is an example of santiag (3, 3) with the picture (http://img697.imageshack.us/img697/1253/screenoriginal.png) you linked on a previous message.

Is there a way to write the santiag function so there isn't a frame shift? (like the way nnedi2_rpow2() has the cshift arugment?)

[Gavino]

Is there a way to write the santiag function so there isn't a frame shift? (like the way nnedi2_rpow2() has the cshift arugment?)
As far as I can see, this is already built-in, via the line
Spline36Resize (w, h, 0, -0.5, w, h * 2)
in function santiag_dir.

[poisondeathray]

As far as I can see, this is already built-in, via the line
Spline36Resize (w, h, 0, -0.5, w, h * 2)
in function santiag_dir.

Weird, did you try it out on any sample?

I double checked that I copied the function correctly, and used different sources, they all seem to exhibit the shift (like the old NNEDI before the cshift was implemented)

[Gavino]

Weird, did you try it out on any sample?
No, I didn't - I just accepted it at face value.
Looking at it again, I think it should be
Spline36Resize (w, h, 0, +0.5, w, h * 2)

Also it looks to me that strh and strv are the wrong way round - strh affects vertical and strv horizontal.

[poisondeathray]

Yes the "+" fixed it , thanks Gavino . I haven't tested the strh or strv

[poisondeathray]

To cretindesalpes or Gavino or anyone else:

Regarding the santiag() function I have some observations & questions:

When it's a YV12 / YUY2 source, the Spline36Resize (w, h, 0, +0.5, w, h * 2) seems to fix the shift

When it's a RGB24 source, the Spline36Resize (w, h, 0, -0.5, w, h * 2) seems to fix the shift

1) Why would this be?

2) Is it possible to modify the function to automatically adjust for this (maybe if/then statements or conditional ) ? My programming skills are NIL :)

[elguaxo]

Also it looks to me that strh and strv are the wrong way round - strh affects vertical and strv horizontal.

Yes, indeed.

Source:
http://i.imgur.com/7qlox.png

santiag(strh=2,strv=10):
http://i.imgur.com/JPr5w.png

santiag(strh=10,strv=2):
http://i.imgur.com/t5rCj.png

[Gavino]

Regarding the santiag() function I have some observations & questions:
When it's a YV12 / YUY2 source, the Spline36Resize (w, h, 0, +0.5, w, h * 2) seems to fix the shift
When it's a RGB24 source, the Spline36Resize (w, h, 0, -0.5, w, h * 2) seems to fix the shift

1) Why would this be?
2) Is it possible to modify the function to automatically adjust for this (maybe if/then statements or conditional ) ?
The usual reason for such differences is that Avisynth processes RGB 'upside-down', so the shift is in the opposite direction. However, here I would expect it to 'come out in the wash', since the direction of the shift is a consequence of the use of field=0 in the call to nnedi3(dh=true). Perhaps it is actually a bug in nnedi3's RGB handling. (I didn't even realise till now that nnedi3 accepts RGB, its nnedi predecessors didn't.)

Whether the fault lies with santiag or with nnedi3, a fix/workaround would be
shift = IsRGB() ? -0.5 : 0.5
Spline36Resize (w, h, 0, shift, w, h * 2)

EDIT: Tests suggest this is a fault in nnedi3 - see here.

[poisondeathray]

Thanks Gavino

The cshift parameter for nnedi2_rpow2/nnedi3_rpow2 isn't affected by YV12 /YUY2 vs. RGB ; is the same "workaround" being used there ?

[Gavino]

The cshift parameter for nnedi2_rpow2/nnedi3_rpow2 isn't affected by YV12 /YUY2 vs. RGB ; is the same "workaround" being used there ?
I guess that's really a question for tritical - I can only speculate.

What I do see is that if nnedi2/3_rpow2 is called without the cshift parameter, it produces different results for YUY2 and RGB.
That's not necessarily wrong - since that function has no 'field' parameter to say where the original pixels should end up, the function is free to do what it likes - but it seems anomalous.
When cshift is specified, I guess the implementatation takes that difference into account, so the final result is correct.

[Archimedes]

I've changed the posted function santiag (http://forum.doom9.org/showthread.php?p=1393006#post1393006) a little bit.

Santiag has now three parameters: type, strh and strv. Type means the anti aliasing type. Possible values are "EEDI3", "NNEDI2" and "NNEDI3". strh is the strength for the horizontal anti aliasing and strv is the strength for the vertical anti aliasing. Default values are: type="nnedi3", strh=1 and strv=1.

Contrary to the first posted function, santiag(strh=0, strv=0) now means no anti aliasing. I've also corected the center shift. With NNEDI3 as type, the center shift now works correct for all possible input clips (YV12, YUY2 and RGB24).

Requirements:
LoadPlugin("plugins\EEDI3\eedi3.dll")
LoadPlugin("plugins\GScript\GScript.dll")
LoadPlugin("plugins\NNEDI2\nnedi2.dll")
LoadPlugin("plugins\NNEDI3\nnedi3.dll")

function SantiagMod(clip input, string "type", int "strh", int "strv") {
type = Default(type, "NNEDI3")
strh = Default(strh, 1)
strv = Default(strv, 1)

input

strh > 0 ? AntiAliasing(type=type, strength=strh) : NOP()
TurnLeft()
strv > 0 ? AntiAliasing(type=type, strength=strv) : NOP()
TurnRight()

function AntiAliasing(clip input, string "type", int "strength") {
input

GScript("""
if (type == "EEDI3") {
EEDI3(dh=True, field=0)
for (i = 2, strength) {
EEDI3(dh=False, field=(i + 1) % 2)
}
}
else if (type == "NNEDI2") {
NNEDI2(dh=True, field=0)
for (i = 2, strength) {
NNEDI2(dh=False, field=(i + 1) % 2)
}
}
else {
NNEDI3(dh=True, field=0)
for (i = 2, strength) {
NNEDI3(dh=False, field=(i + 1) % 2)
}
}
""")

Spline36Resize(input.Width(), input.Height(), 0, 0.5, input.Width(), input.Height() * 2)
}
}

Also it looks to me that strh and strv are the wrong way round - strh affects vertical and strv horizontal.
For me, it looks correct.

Source:
http://img822.imageshack.us/img822/1018/sn850643pointresize.jpg

santiag(strh=2, strv=0):
http://img830.imageshack.us/img830/6213/sn850643strh2strv0.jpg

santiag(strh=0, strv=2):
http://img231.imageshack.us/img231/8074/sn850643strh0strv2.jpg

[Gavino]

For me, it [ie strh, strv] looks correct.
OK, I understand your terminology now.
I was confused because I saw that strh affected vertical resampling and strv horizontal resampling. However, by horizontal anti-aliasing, I think you mean fix (near-)horizontal lines, which requires vertical resampling, and vice versa. Correct?

The code can actually be simplified a bit. The Evals are redundant and, since string comparison is case insensitive, there is no need to convert the type string to upper case. In santiag_dir and santiag_stronger, 'type' and 'strength' should not really be in quotes, since they are not optional parameters there.

For example, santiag_stronger could be rewritten (according to taste) as
function santiag_stronger(clip c, string type, int strength) {
c

type == "EEDI3" ?
\ (strength <= 1) ? EEDI3(dh=true, field=0) : santiag_stronger(type=type, strength=strength - 1).EEDI3(dh=false, field=(strength + 1) % 2)
\ :
\ type == "NNEDI2" ?
\ (strength <= 1) ? NNEDI2(dh=true, field=0) : santiag_stronger(type=type, strength=strength - 1).NNEDI2(dh=false, field=(strength + 1) % 2)
\ :
\ type == "NNEDI3" ?
\ (strength <= 1) ? NNEDI3(dh=true, field=0) : santiag_stronger(type=type, strength=strength - 1).NNEDI3(dh=false, field=(strength + 1) % 2)
\ : NOP()

last
}

[Archimedes]

I was confused because I saw that strh affected vertical resampling and strv horizontal resampling. However, by horizontal anti-aliasing, I think you mean fix (near-)horizontal lines, which requires vertical resampling, and vice versa. Correct?
Yes.

In the meanwhile, I have changed the function again. Now it is an iterative (non recursive) function, called "SantiagMod". It is easier to unterstand than the recursive function. This version looks also more pleasant to me. :)

[Heaud]

Is there any documentation on aaf()? I want to know what each value does before I start tweaking some settings.

[markanini]

^^+1!

BTW Results with aaf() look good and it runs in realtime on my machine, so please tell us more about it.

[Mystery Keeper]

I'm glad that script we assembled is still being improved. But let me remind you the original idea behind it.

I needed a script that would:

Anitalias anime. Resulting script did marverls on anime, but looked weird on real video. See why below.
Only work on edges and lines. For that I spent some time searching for a good mask. Don't remember what script I took it out from, but it was quite unique and much better than usual Sobel. Now anime contains a lot of contrasting edges and lines. Real video doesn't contain as much. It is unpredictable, which part is affected by script in each frame.
Make lines thiner and finer. For that warpsharp is used. I see no reason to use it on real video.
Darken the lines, so they are still visible after everything done on them. Again, why would someone do that on real video?
Do some postsharpening. That just needed to be done. Unrelated to video type.
Soothe lines shaking. In some anime horisontal scene moving introduces vertical lines shaking. The script we had so far made that shaking much more noticable. That's where motion compensation is used for soothing (great thanks, Didee!) Never seen vertical lines shaking in real video.

So you guys testing it on real video is confusing to me. Is new script tuned for it? Got to test it for anime and see if settings are good for it.

Now what the hell with all the interpolators? I used to use EEDI2 for antialiasing real video. Really smoothes some lines, but also blures the picture. In this script Sangnom did just as good as EEDI2, worked twice as fast and didn't introduce a shift like EEDI2 does. So I just threw EEDI2 out of it, it really wasn't needed. But in your script I see all kind of edge-directed interpolators. Apart from EEDI2 they are all quite sharp. They don't do any antialiasing at all. Look at the small AA functions you use. They all use Sangnom. I suggest you take out EDIs from your script at least for the sake of stopping confusing newcomers, making them think that EDIs are good for antialiasing.

[Leinad4Mind]

I've share the last version of it. Topic updated! I'll in the future try to update more, and do some new stuff.

Cheers

[Edge343]

I've been trying to add AA to screenshots/videos from an old XBox game, and though LSharpAAF certainly makes things easier I could still use some help in getting good results. The XBox game in question is called Panzer Dragoon Orta, and I'm currently running it from an Xbox360 (the Xbox360 is capable of emulating certain original Xbox games, this is one of them). I'm capturing the game over HDMI in 720P, and I assume the XBox360 is simply upscaling the image since the original game didn't run in HD resolutions. So basically this is what I'm getting:

Screenshot 1 (http://i361.photobucket.com/albums/oo53/Radik2/PanzerDragoon/PanzerDragoonOrta_1.png)
Screenshot 2 (http://i361.photobucket.com/albums/oo53/Radik2/PanzerDragoon/PanzerDragoonOrta_4.png)

From what I've tried on these images LSharpAAF(aatype="eedi3") seems to deliver the best results:

Screenshot 1 - EEDI3 (http://i361.photobucket.com/albums/oo53/Radik2/PanzerDragoon/PanzerDragoonOrta_1_EEDI3.png)
Screenshot 2 - EEDI3 (http://i361.photobucket.com/albums/oo53/Radik2/PanzerDragoon/PanzerDragoonOrta_4_EEDI3.png)

But given how little I know about all these AA methods being used I though someone here might be able to help me get better results. Any help would be appreciated.

[Bloax]

Does the whole game look like that?
If so, let me cook something alternate up.

Meanwhile:
TurnRight()
Merge(SangNom(0,255).SangNom(1,255),Merge(SangNom(1,255),SangNom(0,255),0.5),0.25)
TurnLeft()
Seems to look "alright".

Edit: And if that's how it looks, here's another:
PointResize(Width/2,Height/2)
TurnRight()
Merge(SangNom(0,255),SangNom(1,255),0.5)
TurnLeft()
NNEDI3_rpow2(2,0,0,1).SangNom(0,255)
Which looks less messy.

[Edge343]

Thanks Bloax. This is basically how the game looks yes, although I uploaded some actual gameplay footage here (http://www.megaupload.com/?d=42YMLISZ).

The second script does seem to deal with the aliasing nicely, although it also softens the image more, which I understand is somewhat inevitable when adding anti-aliasing. Here's a comparison (http://i361.photobucket.com/albums/oo53/Radik2/PanzerDragoonOrta_comparison.png), from left to right is the original, LSharpAAF(aatype=eedi3), your first and then second script. I didn't notice it at first but eedi3 does something weird to the background as well. The last script seems to lose a bit more detail than I'd like. An unfortunate side-effect is also that the game has subtitles at some points and your two scripts seem to add artifacts to those...

[Bloax]

The big blurring I'd guess comes from NNEDI3, since it's rather soft.
As for subtitles, it'd be nice with a picture to mess around with. ;)

[Edge343]

The big blurring I'd guess comes from NNEDI3, since it's rather soft.
As for subtitles, it'd be nice with a picture to mess around with. ;)
Here's an example (http://i361.photobucket.com/albums/oo53/Radik2/PanzerDragoonOrta_2.png). There's also one in the video I linked to.

[Bloax]

This script really is only "usable" for this. Which is aliased and upscaled stuff.
Anything else, and it wrecks it. No exaggeration.


I think it's a bit overkill by now. (Way, way overkill - to be exact.) ;P
Edit: Updated to look cool now.
orig = last
msk=RAverageW(last,32,last.RemoveGrain(19,-1),-32,u=0,v=0,sse=8,bias=-255).RemoveGrain(4,-1).Mt_Deflate()
PointResize(Width/2,Height/2)
NNEDI3(0,true,true,true,true,0,0,1).PointResize(Width,Height)
TurnLeft()
Merge(SangNom(1,30),SangNom(0,30),0.5)
TurnRight()
NNEDI3_rpow2(2,0,0,1)
TurnRight().SangNom(0,64).TurnLeft()
RMerge(last,orig,msk)
RemoveGrain(2) # Fixing up stuff

In my little opinion, I think it'd be best just to do this:
PointResize(Width/2,Height/2)
TurnLeft()
Merge(SangNom(1,30),SangNom(0,30),0.5)
TurnRight()
And call it a day.
Hm, actually. Let me see if I can fix it up. (Seems like parameter overkill.)

Edit: Woop! Be careful with them SangNom settings kids, they're daaangerous.
Anyways, fixed to look like a cool inflation-antialias instead of the uncool holey mess it was.

[Edge343]

Thanks Bloax, I have no idea what the first script does but the results seem to look good :) I'll try and compare it to the second one to see if the difference is worth it. I'd probably be better off capturing in 480P to begin with which I think would avoid the upscaling, but unfortunately my capture card can't capture the 480P output from the XBox360 (which I believe is 640x480).

[kentafilo]

Here is all the require dll or extra functions: http://www.megaupload.com/?d=HQ7NLMW0



Help me:magaupload is down :(

[librarian]

See here (http://www.2shared.com/file/hWF929wD/LSharpAAF_v12Requirements.html). Archive downloaded on December 23rd 2011.

[x265]

Which aatype should i use for anime?

[GodRealm]

See here (http://www.2shared.com/file/hWF929wD/LSharpAAF_v12Requirements.html). Archive downloaded on December 23rd 2011.

Link is off.

[Overdrive80]

Hi, using mod by Dogway I have updated this filter: https://dl.dropboxusercontent.com/u/19135067/LSharpAAF2.00.zip

Updates:


Remove LimitedSharpenFaster dependency
Remove Sangnom and Turn dependency
Added Sangnom2 and FTurn
Added maa2, Santiag and Sangnom2AA


Dependencies:


awarpsharp2 - http://avisynth.nl/index.php/AWarpSharp2 (Modded by Set)
FastLineDarkenMOD.avsi - http://www.mediafire.com/?gpil7zkz24lq4 ( Modded by Dogway (http://forum.videohelp.com/threads/369142-Simple-MDegrain-Mod-v3-0d-A-Quality-Denoising-Solution?p=2364030&viewfull=1#post2364030))
LSFmod.v1.9.avsi - http://avisynth.nl/index.php/LSFmod (Didée/LaTo)
MaskTools.dll - http://avisynth.nl/index.php/MaskTools (Manao)
mt_masktools-2x.dll - http://forum.doom9.org/showthread.php?t=169832 (TP7)
EEDI2.dll - http://avisynth.nl/index.php/EEDI2 (Tritical)
nnedi2.dll, nnedi3.dll - http://forum.doom9.org/showthread.php?p=1343668 - http://avisynth.nl/index.php/Eedi3 (Tritical)
RemoveGrain.dll, Repair.dll from RGTools - http://forum.doom9.org/showthread.php?t=169832 (TP7)
Sangnom2.dll - http://forum.doom9.org/showthread.php?t=169832 (TP7)
UnFilter.dll - http://avisynth.nl/index.php/UnFilter (Tom Barry)

[Great Dragon]

@Overdrive80
LSharpAAF(aatype="maa2") - it doesn't work.

I get "Only YV12 colorspace is supported" and errors in 490, 439, 132 lines
Sure my source is in YV12 colorspace.

I'm using my own modification of LSharpAAF to use maa2 and it work as it should.
I've used official 1.50 version to modify.


# LSharpAAF() by Leinad4Mind
# v1.60 updated in 2015-03-23 by Great Dragon
# Requirements: awarpsharp2; EEDI2.dll; FastLineDarkenMOD.avsi; LSFmod.v1.9.avsi; MaskTools.dll; mt_masktools-26.dll; mvtools.dll;
# nnedi2.dll; RemoveGrain*.dll; Repair*.dll; SangNom.dll; UnFilter.dll
# * HD; HDS; S; SSE2; SSE3; T; TSSE2; TSSE3
#
# All AA Scripts are in this script, so there are no need to download them separately; Except some of them that you can find in AnimeIVTC, as maa, daa.
#
# This is an Anti-Aliasing Combo with some Extras like FastLineDarkenMOD, Presharpening, Postsharpening and Postsmoothing
#
# Thanks to @ Didйe, thetoof, Mystery Keeper, Soulhunter, MisterHatt, martino, mf, Akirasuto, SpikeSpiegel, ScharfisBrain, LaTo & Sagekilla
#
# Example: LSharpAAF(StrDrk=18, ShPre=100, ShPost=280, SmPost=80, aatype="ediaa")
# OR: LSharpAAF(18,100,280,80,"ediaa")

FUNCTION LSharpAAF(clip a, int "StrDrk", int "ShPre", int "ShPost", int "SmPost", bool "stabilize", int "tradius", bool "MT", int "aapel", int "aaov", int "aablk", string "aatype"){
StrDrk = default(StrDrk,38) #FastLineDarkenMOD
ShPre = default(ShPre,18) #Presharpening
ShPost = default(ShPost,300) #Postsharpening
SmPost = default(SmPost,100) #Postsmoothing
stabilize= default(stabilize,false ) # Use post stabilization with Motion Compensation
tradius = default(tradius,2) # 2 = MDegrain2 / 3 = MDegrain3
MT=default(MT,false) # Use josey_wells' branch of MVTools
aapel = default(aapel,1) # accuracy of the motion estimation
# \ (Value can only be 1, 2 or 4. 1 means a precision to the pixel. 2 means a precision to half a pixel, 4 means a precision to quarter a pixel, produced by spatial interpolation (better but slower).)
aaov = default ( aaov,(a.width>1100) ? 8 : 4 ) # block overlap value (horizontal). Must be even and less than block size. (Higher = more precise & slower)
aablk = default ( aablk,(a.width>1100) ? 16 : 8 ) # Size of a block (horizontal). It's either 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.
aatype = default(aatype,"EEDI2") # Use EEDI2() ; EEDI3() ; Sangnom() ; maa() ; daa() ; ediaa() ; aaa() ; saa() ; splinaa() ; supaa() ; naa() ; shaarp() ; aaf() ; laa() ; nnedi() ; nnedi2() ; nnedi3() ; antialiasing() ; nediaa() ; naa3mod(); for anti-aliasing.

a=a.AssumeTFF()
w=width(a)
h=height(a)
m=mt_logic(a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").mt_lut("x 128 / 0.86 ^ 255 *")

preaa=(ShPre==0 && StrDrk==0) ? a : (ShPre==0) ? a.fastlinedarkenmod(thinning=0, strength=StrDrk) : (StrDrk==0) ? a.lsfmod(defaults="slow",preblur="ON",strength=ShPre)
\ : a.fastlinedarkenmod(thinning=0, strength=StrDrk).lsfmod(defaults="slow",preblur="ON",strength=ShPre)

aa=(aatype=="EEDI2") ? preaa.TurnLeft().EEDI2().TurnRight().EEDI2().spline36resize(w,h,0.5,-0.5,2*w+.001,2*h+.001)
\ : (aatype=="EEDI3") ? preaa.TurnLeft().EEDI3().TurnRight().EEDI3().spline36resize(w,h)
\ : (aatype=="Sangnom") ? preaa.spline64resize(w*2,h*2).TurnLeft().SangNom(aa=255).TurnRight().SangNom(aa=255).spline36resize(w,h)
\ : (aatype=="maa") ? preaa.maa() : (aatype=="daa") ? preaa.daa() : (aatype=="ediaa") ? preaa.ediaa() : (aatype=="aaa") ? preaa.aaa()
\ : (aatype=="saa") ? preaa.SAA() : (aatype=="splinaa") ? preaa.splinaa() : (aatype=="supaa") ? preaa.supaa() : (aatype=="naa") ? preaa.naa()
\ : (aatype=="shaarp") ? preaa.shaarp() : (aatype=="aaf") ? preaa.aaf(aam=-0.7,rep=true) : (aatype=="laa") ? preaa.laa(96, 10, true, 2.0)
\ : (aatype=="nnedi") ? preaa.NNEDI(dh=true,field=1).TurnRight().NNEDI(dh=true,field=1).TurnLeft().Spline36Resize(w,h)
\ : (aatype=="nnedi2") ? preaa.NNEDI2(dh=true,field=1,nsize=2,qual=2).TurnRight().NNEDI2(dh=true,field=1,nsize=2,qual=2).TurnLeft().Spline36Resize(w,h,0.5,-0.5,2*w+.001,2*h+.001)
\ : (aatype=="nnedi3") ? preaa.NNEDI3(dh=true,field=1,nsize=2,qual=2).TurnRight().NNEDI3(dh=true,field=1,nsize=2,qual=2).TurnLeft().Spline36Resize(w, h)
\ : (aatype=="antialiasing") ? preaa.antialiasing()
\ : (aatype=="maa2") ? preaa.maa2()
\ : blankclip(pixel_type="YV12").subtitle("Please use Sangnom, EEDI2, EEDI3, maa, daa, ediaa, aaa, ssa, splinaa, supaa, naa, shaarp, aaf, laa,
nnedi, nnedi2, nnedi3, or antialiasing for aatype")

postsh=(ShPost==0 && SmPost==0) ? aa : aa.LimitedSharpenFaster(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost)

merged=mt_merge(a,postsh,m,Y=3,U=3,V=3)

sD=mt_makediff(a,merged)

asuper= a.MSuper(pel=aapel)
sDsuper = sD.MSuper(pel=aapel, levels=1)

fv1 = tradius>=1 ? asuper.MAnalyse(isb=false,delta=1,overlap=aaov,blksize=aablk) : nop()
bv1 = tradius>=1 ? asuper.MAnalyse(isb=true, delta=1,overlap=aaov,blksize=aablk) : nop()
fv2 = tradius>=2 ? asuper.MAnalyse(isb=false,delta=2,overlap=aaov,blksize=aablk) : nop()
bv2 = tradius>=2 ? asuper.MAnalyse(isb=true, delta=2,overlap=aaov,blksize=aablk) : nop()
fv3 = tradius==3 ? asuper.MAnalyse(isb=false,delta=3,overlap=aaov,blksize=aablk) : nop()
bv3 = tradius==3 ? asuper.MAnalyse(isb=true, delta=3,overlap=aaov,blksize=aablk) : nop()

_thSAD = 600
_idx = 13

mbv1 = a.MVAnalyse(isb=true, delta=1,idx=_idx)
mbv2 = a.MVAnalyse(isb=true, delta=2,idx=_idx)
mfv2 = a.MVAnalyse(isb=false,delta=2,idx=_idx)
mfv1 = a.MVAnalyse(isb=false,delta=1,idx=_idx)
allv = (MT==false) ? a : a.MVAnalyseMulti(refframes=tradius,idx=_idx)

sDD = (MT) ? sD.MVDegrainMulti(allv,idx=14) : sD.MVDegrain2(mbv1,mfv1,mbv2,mfv2,idx=14)
sDD2 = tradius==1 ? sDD.MDegrain1(sDsuper,bv1,fv1,thSAD=_thSAD) : tradius==2 ? sDD.MDegrain2(sDsuper,bv1,fv1,bv2,fv2,thSAD=_thSAD)
\ : sDD.MDegrain3(sDsuper,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=_thSAD)

reduc = 0.4
sDD2 = mt_lutxy(sD,sDD2,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD2,1.0-reduc)

return stabilize ? a.mt_makediff(sDD2,U=2,V=2) : merged
}

#Simple Anti-aliasing by Soulhunter
FUNCTION saa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().BilinearResize(OX,OY)
CP ? Last : MergeChroma(Clp)
Return(Last) }

#Normal Anti-aliasing by Didйe
FUNCTION antialiasing(clip orig,int "th_luma",int "th_croma",string "type",int "aath") {

# "th_luma" & "th_croma" are the edge detection thres.: lower values=more edges filtered
# "type" is the matrix used for edge detection: with "sobel" (default) only the
# hi-contrast edges, where artefacts are more noticeable, are filtered. If you want
# to test other matrices, read the MaskTools Guide for more info.
# "aath" = anti-aliasing strenght (default should be fine)

th_luma = Default(th_luma, 20)
th_croma = Default(th_croma, 20)
type = Default(type, "sobel")
aath = Default(aath, 48)
ox = orig.width
oy = orig.height
dx = orig.width * 2
dy = orig.height * 2
clp = orig.IsYV12() ? orig : orig.ConvertToYV12()

a=clp
b=clp.Lanczos4Resize(dx,dy).TurnLeft().SangNom(aa=aath).TurnRight().SangNom(aa=aath) \
.LanczosResize(ox,oy)
c=clp.EdgeMask(th_luma,th_luma,th_croma,th_croma,type)
MaskedMerge(a,b,c) }

# Written by MisterHatt, based off of Soulhunter's SAA() and martino's excellent MAA(), and is slightly
# faster at times for some reason or another. These functions work by generally supersampling, usually with
# nnedi2, and then running a deinterlacer (sangnom and nnedi2 respectively) to get rid of most jaggies.
# I have no idea how destructive these are on whatever random thing people care to throw them at.
# For large supersamples, avs2yuv bitches for no real reason and requires assumeframebased() in your script.
# Requires nnedi2, sangnom, o9k hours of your life.
FUNCTION splinaa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().Spline36Resize(OX,OY)
CP ? Last : MergeChroma(Clp)

Return(Last) }

FUNCTION supaa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.nnedi2_rpow2(rfactor=SS,cshift="spline36resize",qual=3).SangNom().TurnRight()
\ .SangNom().TurnLeft().Spline36Resize(OX,OY)
CP ? Last : MergeChroma(Clp)

Return(Last) }

FUNCTION naa(Clip Clp,Int"SS",Bool"CP") {
OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clp.nnedi3_rpow2(rfactor=SS,cshift="spline36resize",qual=2).nnedi3(nns=2).TurnRight().nnedi3(nns=2).TurnLeft().Spline36Resize(OX,OY)
CP ? Last : MergeChroma(Clp)

Return(Last) }

FUNCTION shaarp(clip input, int "mask", int "type") {
mask = Default(mask,1)
type = Default(type,1)
Assert(mask == 1 || mask == 2, "Please use mask = 1 or 2")
Assert(type == 1 || type == 2, "Please use type = 1 or 2")
aa_clip = (type == 1) ? input.spline36Resize(width(input)*2,height(input)*2) : input.nnedi2_rpow2(rfactor=2,qual=3)
aa_clip = aa_clip.TurnLeft().nnedi2().TurnRight().nnedi2().spline36Resize(width(input),height(input)).MergeChroma(input)
mask = (mask==1) ? input.mt_edge("sobel",7,7,5,5).mt_inflate() : input.mt_edge("roberts",0,4,0,4).mt_inflate()

return mt_merge(input,aa_clip,mask) }

function aaf(clip input, float "aam", int "aay", "aax", bool "rep")
{
input = input.isYV12() ? input : input.converttoyv12()

aam = default(aam, -0.6)
aar = aam<0 ? (aam-1)*0.25 : (aam+1)*0.25
aay = default(aay, 28)
aax = default(aax, aay)
rep = default(rep, true)

sx = width(input)
sy = height(input)

aa = aar<0 ? input.LanczosResize(sx,int(sy*abs(aar))*4) :
\ aar==0.5 ? input.Pointresize(sx*2, sy*2) :
\ input.LanczosResize(int(sx*abs(aar)*4),int(sy*abs(aar))*4)
aa = aay>0 ? aa.SangNom(aa=aax) : input
aa = aar<0 && aax>0 ? aa.LanczosResize(int(sx*abs(aar))*4,sy) : aa
aa = aax>0 ? aa.turnright().SangNom(aa=aax).Turnleft() : aa
aa = aa.LanczosResize(sx,sy)

return rep==true ? aa.repair(input,18) : aa
}

### LaTo Antialiasing Script v2
### Need: asharp.dll & sangnom.dll
function LAA(clip input, int "strength", int "threshold", bool "sharp", float "ss") {

strength = default(strength, 96)
threshold = default(threshold, 10)
sharp = default(sharp, true)
ss = default(ss, 2.0)
aastr = int(strength) /2
asthr = float(strength) /100
ox = input.width
oy = input.height
ox2 = round(ox*ss/8)*8
oy2 = round(oy*ss/8)*8

process = input.spline36resize(ox2,oy2).turnleft().sangnom(aastr).turnright().sangnom(aastr)
process = ( sharp == true ) ? process.asharp(asthr,0,0).spline36resize(ox,oy) : process.spline36resize(ox,oy)
mask = mt_average(input,process,u=1,v=1).mt_edge(thy1=threshold,thy2=threshold,u=1,v=1)
output = mt_merge(input,process,mask,u=2,v=2)

return ( output )
}

function NediAA(clip c)
{ c.nnedi3(field=-2)
merge(selecteven(),selectodd()) }

FUNCTION naa3mod(Clip Clp,Bool "CP")
{
OX = Clp.Width()
OY = Clp.Height()
CP = Default(CP,True)
Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()
Clpnn1 = Clp.nnedi3(1, dh=true,nsize=2,qual=2,U=CP,V=CP).nnedi3(0,nsize=2,qual=2,U=CP,V=CP)
Clpy1 = Clpnn1.Spline36Resize(OX,OY,0,-0.5,OX,OY*2)
Clps1 = CP ? Clpy1.MergeChroma(Clpnn1.Spline36Resize(OX,OY,0,-1,OX,OY*2)) : Clpy1.MergeChroma(Clp)
Clpnn2 = Clps1.turnright().nnedi3(1, dh=true,nsize=2,qual=2,U=CP,V=CP).nnedi3(0,nsize=2,qual=2,U=CP,V=CP)
Clpy2 = Clpnn2.Spline36Resize(OY,OX,0,-0.5,OY,OX*2)
CP ? Clpy2.MergeChroma(Clpnn2.Spline36Resize(OY,OX,0,-1,OY,OX*2)).turnleft() : Clpy2.turnleft().MergeChroma(Clp)
Return(Last)
}

function maa2(clip c, int "mask", bool "chroma", float "ss", int "aa", int "aac", int "threads", int "show", int "maskt")
{
chroma = Default(chroma, false)
mask = Default(mask, 1)
maskt = Default(maskt, 1)
mtresh = (mask < 0) ? -mask : 7
show = Default(show, 0)
uv = (chroma) ? 3 : 1

Assert(c.IsY8 || c.IsYV12 || c.IsYV24 || c.IsYV16, "MAA2: Input must be Y8, YV12, YV16 or YV24")
Assert(0 <= show <= 2, "MAA2: Parameter 'show' must be between 0 and 2")

# create mask
if (mask != 0) {
m = (maskt != 1) ? c.mt_edge("min/max", 0, mtresh, 0, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv) : c.mt_edge("sobel", mtresh, mtresh, mtresh-6, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv)
}

# run sangnom2-based aa
if (!chroma || show > 0) {
c_aa = c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads)
}
else if (c.IsYV16) {
c_aa_u = c.UtoY8().Sangnom2AA(ss, aac, threads=threads)
c_aa_v = c.VtoY8().Sangnom2AA(ss, aac, threads=threads)
c_aa = YToUV(c_aa_u, c_aa_v, c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads))
}
else { c_aa = c.Sangnom2AA(ss, aa, aac, threads) }

# prepare chroma planes for mask overlay
if (show == 1) {
c_aa = (c.IsY8) ? c.ConvertToYV12().mt_lut(y=2, u=0, v=0)
\ : c.mt_lut("x 2 /", y=2, u=3, v=3)
}
else if (show == 2) {
c_aa = (c.IsY8) ? c_aa.ConvertToYV12().mt_lut(y=2, u=0, v=0)
\ : YtoUV(c.UtoY8(), c.VtoY8(), c_aa).mt_lut("x 2 /", y=2, u=3, v=3)
}

# merge aa'ed lines into source
if (mask == 0) {
return mt_logic(c_aa, "and", y=4, u=2, v=2)
}
else if (show > 0) {
if (c.IsYV16) {
m_uv = BilinearResize(m, m.width/2, m.height)
return mt_merge(c, c_aa, YtoUV(m_uv, m_uv, m), u=3, v=3)
}
else {
return (c.IsYV24) ? mt_merge(c, c_aa, m.YtoUV(m,m), u=3, v=3)
\ : mt_merge(c.ConvertToYV12(), c_aa, m, u=3, v=3, luma=true)
}
}
else {
return (chroma) ? mt_merge(c, c_aa, m, u=3, v=3)
\ : mt_merge(c, c_aa, m, u=2, v=2)
}
}

function Sangnom2AA(clip c, float "ss", int "aa", int "aac", int "threads")
{
threads = Default(threads, 4)
aa = Default(aa, 48)
aac = Default(aac, aa-8)
aac = (aac < 0) ? 0 : aac
ss = Default(ss, 2.0)
ss_w = int(round(c.width*ss/4.0)*4)
ss_h = int(round(c.height*ss/4.0)*4)

Assert(ss > 0, "MAA2: Supersampling factor must be > 0")

return c.Spline36Resize(ss_w, ss_h).TurnLeft() \
.SangNom2(threads=threads, aa=aa, aac=aac).TurnRight().SangNom2(threads=threads, aa=aa, aac=aac).Spline36Resize(c.width, c.height)
}

[Great Dragon]

v2.00 even doesn't work as it should with default settings

http://t1.someimage.com/r17lo5G.jpg (http://someimage.com/r17lo5G) http://t1.someimage.com/3PSwtZd.jpg (http://someimage.com/3PSwtZd)

[Overdrive80]

I can not reproduce it.


function maa2(clip c, int "mask", bool "chroma", float "ss", int "aa", int "aac", int "threads", int "show", int "maskt")
{
chroma = Default(chroma, false)
mask = Default(mask, 1)
maskt = Default(maskt, 1)
mtresh = (mask < 0) ? -mask : 7
show = Default(show, 0)
uv = (chroma) ? 3 : 1

Assert(c.IsY8 || c.IsYV12 || c.IsYV24 || c.IsYV16, "MAA2: Input must be Y8, YV12, YV16 or YV24")
Assert(0 <= show <= 2, "MAA2: Parameter 'show' must be between 0 and 2")

# create mask
if (mask != 0) {
m = (maskt != 1) ? c.mt_edge("min/max", 0, mtresh, 0, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv) : c.mt_edge("sobel", mtresh, mtresh, mtresh-6, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv)
}

# run sangnom2-based aa
if (!chroma || show > 0) {
c_aa = c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads)
}
else if (c.IsYV16) {
c_aa_u = c.UtoY8().Sangnom2AA(ss, aac, threads=threads)
c_aa_v = c.VtoY8().Sangnom2AA(ss, aac, threads=threads)
c_aa = YToUV(c_aa_u, c_aa_v, c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads))
}
else { c_aa = c.Sangnom2AA(ss, aa, aac, threads) }

# prepare chroma planes for mask overlay
if (show == 1) {
c_aa = (c.IsY8) ? c.ConvertToYV12().mt_lut(y=2, u=0, v=0)
\ : c.mt_lut("x 2 /", y=2, u=3, v=3)
}
else if (show == 2) {
c_aa = (c.IsY8) ? c_aa.ConvertToYV12().mt_lut(y=2, u=0, v=0)
\ : YtoUV(c.UtoY8(), c.VtoY8(), c_aa).mt_lut("x 2 /", y=2, u=3, v=3)
}

# merge aa'ed lines into source
if (mask == 0) {
return mt_logic(c_aa, "and", y=4, u=2, v=2)
}
else if (show > 0) {
if (c.IsYV16) {
m_uv = BilinearResize(m, m.width/2, m.height)
return mt_merge(c, c_aa, YtoUV(m_uv, m_uv, m), u=3, v=3)
}
else {
return (c.IsYV24) ? mt_merge(c, c_aa, m.YtoUV(m,m), u=3, v=3)
\ : mt_merge(c.ConvertToYV12(), c_aa, m, u=3, v=3, luma=true)
}
}
else {
return (chroma) ? mt_merge(c, c_aa, m, u=3, v=3)
\ : mt_merge(c, c_aa, m, u=2, v=2)
}
}

This function not use avisynth language, I think.

EDIT: I am using avisynth 2.6 and I dont get error with my source that is yv12 (if use converttoyv24 or converttoy8 get error because lsfmod is limited yv12 colourspaces but not for maa2).

Is possible that you didn't remove dll of removegrain, repair, etc... from pack of kassandro?? Please, use plugins listed in dependencies http://forum.doom9.org/showthread.php?p=1716077#post1716077 and remove old plugins.

I use maa2 http://avisynth.nl/index.php/MAA2

[Great Dragon]

I'm using Avisynth+ but I'll check plugins version. However I'm pretty sure they're up to date.

UPDATE:
I'm using 0.41mod version of maa2 http://pastebin.com/fqR4CMbV
There is also 0.42+ mod http://pastebin.com/F0GLtrx5 I guess it's made specially for Avisynth+ however 0.41mod work fine for me too.

UPDATE2:
@Overdrive80 can you please check if those mods works for you with Avisynth 2.6 ?

[Overdrive80]

@Overdrive80 can you please check if those mods works for you with Avisynth 2.6 ?

Ok, but I have convert sintaxis for maa2 on avisynth from your avisynth+ mod.

[Overdrive80]

Ok, with code below of maa2 0.42+ runs fine on avisynth 2.6:

/*
MAA2MOD v0.42
=========

Updated version of the MAA antialising script from AnimeIVTC.
MAA2 uses tp7's SangNom2, which provide a nice speedup for SangNom-based antialiasing,
especially when only processing the luma plane.
The defaults of MAA2 match up with MAA, so you'll get identical output (save for the more accurate border region processing of SangNom2)
when using this script as a drop-in replacement.

MAA2 supports Y8, YV12, YV16 and YV24 input.

Requirements:

* AviSynth
* SangNom2 0.3+
* Masktools 2.0b1

Parameters:

+ [int] mask (1)
* 0: Disable masking
* 1: Enable masking
* -i: Enable masking with custom treshold
+ [bool] chroma (false)
* false: Don't process chroma channels (copy UV from the source clip if present)
* true: Process chroma channels
+ [float] ss (2.0)
* Supersampling factor (sensible values are between 2.0 and 4.0 for HD content)
+ [int] aa (48)
* Sangnom2 luma antialiasing strength
+ [int] aac (aa-8)
* Sangnom2 chroma antialiasing strength
+ [int] threads (4)
* Number of threads used by every Sangnom2 instance
+ [int] show (0)
* 0: Don't overlay mask
* 1: Overlay mask only
* 2: Overlay mask and run antialiasing on the luma plane
+ [int] maskt (1)
* 1: sobel
* 2: min/max

*/

function maa2mod(clip c, int "mask", bool "chroma", float "ss", int "aa", int "aac", int "threads", int "show", int "maskt")
{
chroma = Default(chroma, false)
mask = Default(mask, 1)
maskt = Default(maskt, 1)
mtresh = (mask < 0) ? -mask : 7
show = Default(show, 0)
uv = (chroma) ? 3 : 1

Assert(c.IsY8 || c.IsYV12 || c.IsYV24 || c.IsYV16, "MAA2: Input must be Y8, YV12, YV16 or YV24")
Assert(0 <= show <= 2, "MAA2: Parameter 'show' must be between 0 and 2")

# create mask
m = mask != 0 ? c.mt_edge("sobel", mtresh, mtresh, mtresh-6, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv) : \
maskt != 1 ? c.mt_edge("min/max", 0, mtresh, 0, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv) : nop

# run sangnom2-based aa
c_aa_u = (c.IsYV16) ? c.UtoY8().Sangnom2AA(ss, aac, threads=threads) : nop
c_aa_v = (c.IsYV16) ? c.VtoY8().Sangnom2AA(ss, aac, threads=threads) : nop

c_aa = (!chroma || show > 0) ? c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads) : \
(c.IsYV16) ? YToUV(c_aa_u, c_aa_v, c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads)) : c.Sangnom2AA(ss, aa, aac, threads)

# prepare chroma planes for mask overlay
c_aa = (Show == 1 && !c.IsY8) ? c.mt_lut("x 2 /", y=2, u=3, v=3) : \
(Show == 1 && c.IsY8) ? c.ConvertToYV12().mt_lut(y=2, u=0, v=0) : \
(show == 2 && !c.IsY8) ? YtoUV(c.UtoY8(), c.VtoY8(), c_aa).mt_lut("x 2 /", y=2, u=3, v=3) : \
(show == 2 && c.IsY8) ? c.ConvertToYV12().mt_lut(y=2, u=0, v=0) : c_aa

# merge aa'ed lines into source
m_uv = (c.IsYV16) ? BilinearResize(m, m.width/2, m.height) : nop

last = (mask == 0) ? mt_logic(c_aa, "and", y=4, u=2, v=2) : \
(mask != 0 && show > 0 && c.isYV16) ? mt_merge(c, c_aa, YtoUV(m_uv, m_uv, m), u=3, v=3) : \
(mask != 0 && show > 0 && c.IsYV24) ? mt_merge(c, c_aa, m.YtoUV(m,m), u=3, v=3) : \
(mask != 0 && show > 0) ? mt_merge(c.ConvertToYV12(), c_aa, m, u=3, v=3, luma=true) : \
(mask != 0 && show == 0 && chroma) ? mt_merge(c, c_aa, m, u=3, v=3) : mt_merge(c, c_aa, m, u=2, v=2)

return last
}




function Sangnom2AA(clip c, float "ss", int "aa", int "aac", int "threads")
{
threads = Default(threads, 4)
aa = Default(aa, 48)
aac = Default(aac, aa-8)
aac = (aac < 0) ? 0 : aac
ss = Default(ss, 2.0)
ss_w = int(round(c.width*ss/4.0)*4)
ss_h = int(round(c.height*ss/4.0)*4)

Assert(ss > 0, "MAA2: Supersampling factor must be > 0")

return c.Spline36Resize(ss_w, ss_h).TurnLeft() \
.SangNom2(threads=threads, aa=aa, aac=aac).TurnRight().SangNom2(threads=threads, aa=aa, aac=aac).Spline36Resize(c.width, c.height)
}

Please, check if all code is fine for you, I am not tested too much.

Note: I rename function to maa2mod

[Great Dragon]

It works for me and produces identical result compare to original maa2+ 0.42 mod

[Overdrive80]

Ok, I have incluided in LSharpAAF and runs fine on avisynth 2.6 (maybe problem with avisynth+??), I dont get error for yv12 input.

https://dl.dropboxusercontent.com/u/19135067/LSharpAAF2.01.zip

[Great Dragon]

No errors this time but the script still does nothing. Picture remains the same no matter what aatype I choose.
So maybe it has some Avisynth+ compatibility issue.

[real.finder]

No errors this time but the script still does nothing. Picture remains the same no matter what aatype I choose.
So maybe it has some Avisynth+ compatibility issue.

this will work in both avs and avs+

MAA2 v0.432 mod https://pastebin.com/8qz1kUbM

try it with maa2(mask=-100, maskt=2, ss=3)

ss=3 is good for noisy source

[kuchikirukia]

If I rewrite my Santiag v1.6 to use Resize8 for the frame shift instead of the internal Spline36Resize, will it avoid the chroma shift bug in the internal resizers? Or is that already taken care of in the frame shift?

[cretindesalpes]

If I rewrite my Santiag v1.6 to use Resize8 for the frame shift instead of the internal Spline36Resize, will it avoid the chroma shift bug in the internal resizers? Or is that already taken care of in the frame shift?

Santiag doesn’t take any chroma shift bug into account. But you don’t need to rewrite it to use your own scaler. Just provide it as scaler_post argument in the call (it should have the same prototype as the standard avisynth scalers), or change the default value.

I’ll check how chroma location issues affect the final resizing and will publish a fix if required.

[killerteengohan]

Hey cretindesalpes, why is it when I try to use the Santiag v1.6 you posted "aa=24" parameter I get this error every time.

(The named argument "aa" was passed more than once to santiag_dir)
santiagv1.6.avsi, line 30

[cretindesalpes]

Thanks, typo fixed.