Anti-aliasing script

[SpikeSpiegel]

function antialiasing(clip orig,int "th_luma",int "th_croma",string "type",int "aath") {

# "th_luma" & "th_croma" are the edge detection thres.: lower values=more edges filtered
# "type" is the matrix used for edge detection: with "sobel" (default) only the
# hi-contrast edges, where artefacts are more noticeable, are filtered. If you want
# to test other matrices, read the MaskTools Guide for more info.
# "aath" = anti-aliasing strenght (default should be fine)

th_luma = Default(th_luma, 20)
th_croma = Default(th_croma, 20)
type = Default(type, "sobel")
aath = Default(aath, 48)
ox = orig.width
oy = orig.height
dx = orig.width * 2
dy = orig.height * 2
clp = orig.IsYV12() ? orig : orig.ConvertToYV12()

a=clp
b=clp.Lanczos4Resize(dx,dy).TurnLeft().SangNom(aa=aath).TurnRight().SangNom(aa=aath) \
.LanczosResize(ox,oy)
c=clp.EdgeMask(th_luma,th_luma,th_croma,th_croma,type)
MaskedMerge(a,b,c)

}

Hi!

This filter is meant for good quality clips with soft antialiasing (like DVs);
thanks to MaskTools' functions, a strong AA-filter (Sangnom) is applied only where it's needed without producing artefacts typical of this filter.

Requires:
-MaskTools
-SangNom

Let me know what do you think about it!

P.S. Thanks to Didée, mf, scharfis_brain and Soulhunter (in alphabetical order) for their help!

EDIT1: Ooops! Now, with Lanczos4Resize instead of PointResize, there's no definition loss nor artefacts.
...and now SangNom's config. is aviable
EDIT2: Credits
EDIT3: Fixed a typo, erased a useless option, more info added.

[Didée]

Did a quick comparison of some methods. I post the script as well, in case I've made an error in the hurry ...

http://img15.exs.cx/img15/3225/some_anti_aliasing_methods2.jpg

script:
mpeg2source()
crop(much)
orig=last
ox=last.width
oy=last.height
jaggy=orig.separatefields().selecteven()
SangN=jaggy.antialiasing().lanczosresize(ox,oy)
SangNSS=jaggy.lanczosresize(ox,oy).antialiasing()
tom=jaggy.tomsmocomp(1,-1,0)
tomSS=jaggy.lanczosresize(ox,oy).tomsmocomp(1,-1,1).lanczosresize(ox,oy)
trit=orig.tdeint(mode=0,order=1,field=1,mthreshL=-1,mthreshC=-1,type=1)
tritSS=jaggy.pointresize(ox,oy).tdeint(mode=0,order=1,field=1,mthreshL=-1,mthreshC=-1,type=1)
dble=jaggy.pointresize(ox,oy).tdeint(mode=0,order=1,field=1,mthreshL=-1,mthreshC=-1,type=1)
\ .tomsmocomp(1,-1,1).lanczosresize(ox,oy)
dbleS=dble.LimitedSharpen(ss_x=2.0,ss_y=1.0,Smode=3,strength=200,Lmode=2,overshoot=1)

# without the dreadful subtitle commands:
stackvertical(
\ stackhorizontal(jaggy.lanczosresize(ox,oy).subtitle(jaggystr,size=12),orig),
\ stackhorizontal(SangN,SangNSS),
\ stackhorizontal(tom,tomSS),
\ stackhorizontal(trit,tritSS),
\ stackhorizontal(dble,dbleS) )

return last

edit: better placing of cropped area.

[SpikeSpiegel]

Originally posted by Didée
Did a quick comparison of some methods. I post the script as well, in case I've made an error in the hurry ...

script:
mpeg2source()
crop(much)
orig=last
ox=last.width
oy=last.height
jaggy=orig.separatefields().selecteven()
SangN=jaggy.antialiasing().lanczosresize(ox,oy)
SangNSS=jaggy.lanczosresize(ox,oy).antialiasing()
tom=jaggy.tomsmocomp(1,-1,0)
tomSS=jaggy.lanczosresize(ox,oy).tomsmocomp(1,-1,1).lanczosresize(ox,oy)
trit=orig.tdeint(mode=0,order=1,field=1,mthreshL=-1,mthreshC=-1,type=1)
tritSS=jaggy.pointresize(ox,oy).tdeint(mode=0,order=1,field=1,mthreshL=-1,mthreshC=-1,type=1)
dble=jaggy.pointresize(ox,oy).tdeint(mode=0,order=1,field=1,mthreshL=-1,mthreshC=-1,type=1)
\ .tomsmocomp(1,-1,1).lanczosresize(ox,oy)
dbleS=dble.LimitedSharpen(ss_x=2.0,ss_y=1.0,Smode=3,strength=200,Lmode=2,overshoot=1)

# without the dreadful subtitle commands:
stackvertical(
\ stackhorizontal(jaggy.lanczosresize(ox,oy).subtitle(jaggystr,size=12),orig),
\ stackhorizontal(SangN,SangNSS),
\ stackhorizontal(tom,tomSS),
\ stackhorizontal(trit,tritSS),
\ stackhorizontal(dble,dbleS) )

return last

edit: better placing of cropped area.

Hi!

I've already done a few tests (always with a deinterlaced DV) with TomsMoComp(x,1,1), but results weren't really encouraging:
with original resolution it removes almost all artefacts, but details are blurred; vice-versa with double height (good definition, but too much alias).

On the contrary, TDeint+doubleheight works quite well: results are similiar to "antialiasing", but there's a little definition loss.

Anyway, give a try to antialias (the last "version") with a hi-medium resolution source: this script is meant for good quality movies, that why it didn't perform well with the WB sample.

[Bulletproof]

Looks promising but it would be really great if someone could figure out how faroujda's DCDI method works to create an avisynth plugin. I personally own 3 devices with the DCDI chip for deinterlacing and it is pretty amazing when it comes to deinterlacing, it really eliminates jaggies good and keeps alot of resolution, here is an example:

http://www.av-outlet.com/images/help/dcdi-flag.jpg

It's not that great of a sample but if you've seen DCDI in action you will see that there is nothing that can beat it right now (I think so anyway). More detailed explination here: http://www.av-outlet.com/en-us/dept_335.html

[Didée]

Originally posted by SpikeSpiegel
Anyway, give a try to antialias (the last "version") with a hi-medium resolution source: this script is meant for good quality movies, that why it didn't perform well with the WB sample.
Erh, sorry. If the quality of that sample is not sufficient for your script, then your script is useless. The given sample is perfectly valid (It's the original PAL trailer of HPII, in full resolution. I just cropped down to an 'interesting' area).
In fact, most sources that do need antialiasing will be of not-so-impressing quality to start with - most probably they'll be of lower quality than the sample I used. And just discarding one field from a progressive sequence, as I did here, is a good simulation of common aliasing problems (capturing only one field, dumb bobbing, one field useless due to fieldblending, etc. pp.)


However, your script has a couple of issues that could be improved:


1.
You're offering the aa threshold for sangnom, but it is not used in the script. Your call of SangNom should read

SangNom(order,aath)


2.
Chroma probably is just passed through (it IS on that P-III I'm sitting at right now).
Better would be:

c=edgemask(...).FitY2UV()

I've found that in many cases, it's better to use the luma edges for chroma also, insted of creating edges on chroma directly.

If you prefer to use "native" chroma edges, then you must still write

c=edgemask(... , Y=3,U=3,V=3)

Else, you won't get any edges on the chroma planes at all (as in your posted script).

Finally, you should write

MaskedMerge(a,b,c, Y=3,U=3,V=3)

AFAIK, MaskedMerge's default is to leave chroma alone, unless one specifies U|V=3.


3.
As it is typical for EDI resamplers, SangNom too delivers an output that is spatially shifted from the input. This would not be a big issue ... if you would create the edgemask on the shifted frame. But you are creating it on the not-shifted input, and apply it later on the shifted output. That's not ideal. So, the declaration of "c" better should read

c=b.edgemask( .. ) ..


Below script addresses abovementioned issues, and includes these additional changes:

- added "HQedge" [bool]. It's slower, reckognizes edges better, is more robust against noise ... and perhaps is not suited at all for this application. So, it's just for testing & playing.
- normal edgemask is created on the original sized frame. HQedgemask is created on the (for sangnom) double-sized frame.

Without much testing:
function antialiasing( clip orig, int "th_luma", int "th_croma", string "type", int "order", int "aath", bool "HQedge")
{

# "th_luma" and "th_croma" affect directly the edge detection: higher values = more edges filtered
# set "order = 0" for Top Field First; order = 1 --> Bottom Field First
# aath = anti-aliasing strenght (default should be fine)


th_luma = Default(th_luma, 20)
th_croma = Default(th_croma, 20)
type = Default(type, "sobel")
order = Default(order, 1)
aath = Default(aath, 48)
HQedge = Default(HQedge, false)
ox = orig.width
oy = orig.height
dx = orig.width * 2
dy = orig.height * 2

orig.convertToYV12()
a=last
b=lanczos4Resize(dx, dy).TurnLeft().SangNom(order,aath).TurnRight().SangNom(order,aath)

# native chroma edges:
#c=b.lanczosresize(ox,oy).EdgeMask(0, th_luma, 0, th_croma, type,Y=3,U=3,V=3)

# use luma edges for chroma:
c=b.lanczosresize(ox,oy).EdgeMask(0, th_luma, 0, th_croma, type,Y=3,U=1,V=1).FitY2UV()

d= logic( b.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=1,V=1)
\ ,b.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=1,V=1)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false).FitY2UV().lanczosresize(ox,oy)
b=b.lanczosresize(ox,oy)

HQedge ? MaskedMerge(a,b,d,Y=3,U=3,V=3) : MaskedMerge(a,b,c,Y=3,U=3,V=3)
}
Just some suggestions.

[SpikeSpiegel]

@ Didée:

Thanks for your help, as you may have noticed (the 1st error was quite exemplifying :o ) I didn't pay too much attention on the creation of the script itself (that needs MUCH improving), rather I wanted to give an idea, i.e.:

For aliasing is present only on edges, why we should filter the entire frame?

And the main reason of filtering only what needs it (edges), in this case, is not speed gain (...and you should know why ;) ), but to keep max definition.
Obviously, using a bad-quality source (where there's something more than simple alias), there's no need of this, that's why I wrote that it's more suiteg for medim-good quality stuff (even if this reduces the possible applications).

Anyway the EdgeDetection is tuned for sharp vids, then, if you are testing with mpeg2 samples, you should increase the threshold.

Now I don't have much free time, but asap I'll test HQedge and the other settings!


P.S. I don't see any difference between:

c=edgemask( .. )

and

c=b.edgemask( .. ).FitY2UV()

:confused:

[Soulhunter]

Hmm...

Anti-aliasing via SangNom seems to work !!!

Source

http://img83.exs.cx/img83/6343/Source.png

SangNom

http://img83.exs.cx/img83/3789/Sagnom.png

Think the edge-detection is the culprit ???

-> Range is not wide enough...


Btw, I thought the purpose is simple aliasing reduction !?!

But it seems you wanna use it to reduce interlace-leavings... :rolleyes:


Bye

[Soulhunter]

And it works very good @ anime/toon sources !!!

Sample with bad aliasing...

Source

http://img83.exs.cx/img83/5109/Source_DBGT_01.png

SangNom

http://img83.exs.cx/img83/7342/SangNom_DBGT_01.png


Bye

[SpikeSpiegel]

Originally posted by Soulhunter
Anti-aliasing via SangNom seems to work !!!

Hi!

Did you use only SangNom or the anti-alias script?

Originally posted by Soulhunter
Btw, I thought the purpose is "simple" anti-aliasing !?!

But it seems you wanna use it for interlace-leavings...

No, absolutely not, as you can see in the Didée's test, it doesn't work properly with interlace-leavings; anyway, some adaptive deinterlace filters may produce aliasing, that can be easily erased with a script like this, but only if there aren't other kinds of imperfections.

[Soulhunter]

Originally posted by SpikeSpiegel

Did you use only SangNom or the anti-alias script?

For the samples above Ive simply used the stuff mf told me... ;)

LanczosResize(X*2,Y*2).TurnLeft().SangNom().TurnRight().SangNom().LanczosResize(X,Y)


Bye

[scharfis_brain]

try

pointResize(X*2,Y*2).TurnLeft().SangNom().TurnRight().SangNom().LanczosResize(X,Y)

[Soulhunter]

Originally posted by scharfis_brain
try

pointResize(X*2,Y*2).TurnLeft().SangNom().TurnRight().SangNom().LanczosResize(X,Y)

Uhm, actually this is what mf told me... :D

before = last
PointResize(width*2, height*2)
ConvertToYUY2()
TurnLeft()
ConvertToYV12()
SangNom()
ConvertToYUY2()
TurnRight()
ConvertToYV12()
SangNom()
LanczosResize(width/2, height/2)
MergeChroma(before)


Sorry n' Bye

[Soulhunter]

Ok, a small update...

Source

http://img83.exs.cx/img83/5109/Source_DBGT_01.png

Lanczos->SangNom->Lanczos

http://img83.exs.cx/img83/7342/SangNom_DBGT_01.png

PointResize->SangNom->Lanczos

http://img83.exs.cx/img83/9470/SangNom_PointResizeLanczos.png

Lanczos->SangNom->Bilinear

http://img83.exs.cx/img83/5355/SangNom_LanczosBilinear.png

Lanczos->SangNom->Bilinear->Unfilter(50,50)

http://img83.exs.cx/img83/8154/SangNom_LanczosBilinearUnfilter50.png


Imo the PointResize sample is sharp but shows still some aliasing as the Lanczos sample... :(

The Bilinear/Unfilter sample looks same sharp but has near like no aliasing left !!!

#################################################################

X = 320 # Resolution width
Y = 240 # Resolution height

A = 50 # Unfilter value A
B = 50 # Unfilter value B

#################################################################

AviSource("C:\MovieBackups\Source.avi").ConverttoYV12()

#################################################################

Before = Last

LanczosResize(X*2,Y*2)

ConvertToYUY2()

TurnLeft()

ConvertToYV12()

SangNom()

ConvertToYUY2()

TurnRight()

ConvertToYV12()

SangNom()

BilinearResize(X,Y)

Unfilter(A,B)

MergeChroma(Before)

#################################################################


But I dunno how to make a function out of this... :o

AnimeAntiAliasing: AAA(X,Y,A,B)


Bye

[Didée]

What's that, Soulhunter?

A brigade of inbetween "ConvertTo...ConvertTo...ConvertTo...", and in the end, the unprocessed original chroma is copied back. Perhaps one should throw some lsresize()'s in the mix, to make those NOPs even more valuable :D

However, making a function out of that is pretty simple:

You just have to /*whisper...whisper...whisper*/, that's all :p

[Soulhunter]

# _______________________________________________________
#
# SAA | Simple Antialiasing
# _______________________________________________________
#
# Simplicity by Soulhunter... \m\ ^_^ /m/
# _______________________________________________________
#
#
# Requirements:
#
# SangNom.dll
# 100GHz CPU
#
#

Function SAA(Clip Clp,Int"SS",Bool"CP")
{

OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)


Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()

Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().BilinearResize(OX,OY)

CP ? Last : MergeChroma(Clp)

Return(Last)

}


Bye

[SpikeSpiegel]

Originally posted by Didée
As it is typical for EDI resamplers, SangNom too delivers an output that is spatially shifted from the input. This would not be a big issue ... if you would create the edgemask on the shifted frame. But you are creating it on the not-shifted input, and apply it later on the shifted output. That's not ideal. So, the declaration of "c" better should read

c=b.edgemask( .. ) ..

There's still a great problem:
this way, the base clip is still in the original position, then, overlaying base & overlay clip (shifted), I get artefacts as well (=alias on one side of edges).

If the SangNom's "shift vector" is (1,1) (as it seems to be), I should use something like:


orig = clip.ConvertToRGB()
clp = orig.crop(0,1,-1,0).addborders(0,1,1,0).ConvertToYV12()
shift = orig.crop(1,0,0,-1).addborders(0,1,1,0).ConvertToYV12()

...

a= clp
b= shift.Lanczos4Resize(dx, dy).TurnLeft()...
c= clp.EdgeMask(0, th_luma, 0, th_croma, ....)


But, if I wrote "There's still a great problem", there must be a reason ;) :
actually the "shift vector" shouldn't have integer components (it's more like (1/2,1/2) ), then...I don't see any solution! :scared:

EDIT1: fixed crop&addborders

[scharfis_brain]

I don't see any solution!

upsize using with something better than pointresize to 2x & 2y.

then shift by the desired vector and to a point-downsize to the original resolution.

[SpikeSpiegel]

Originally posted by scharfis_brain
I don't see any solution!

upsize using with something better than pointresize to 2x & 2y.

then shift by the desired vector and to a point-downsize to the original resolution.

I've already tried something like this with (1/2,1/2--> too much) and then with (1/4,1/4--> closer):

ox = clip.width
oy = clip.height
qx = clip.width * 4
qy = clip.height * 4
orig = clip.ConvertToRGB()
clp = orig.Lanczos4Resize(qx, qy).crop(0,1,-1,0).addborders(0,1,1,0).ConvertToYV12().LanczosResize(ox, oy)
shift = orig.Lanczos4Resize(qx, qy).crop(1,0,0,-1).addborders(0,1,1,0).ConvertToYV12().LanczosResize(ox, oy)

...but still I can't find the right vector (that maybe has different components, like (x,y) ) and the speed loss is already excessive.
I suppose that a fast "shiftVector()" plugin doesn't already exist, isn't it?

[Didée]

Originally posted by SpikeSpiegel
I suppose that a fast "shiftVector()" plugin doesn't already exist, isn't it?
No, alas not.

But you can take either GeneralConvolution() or MaskTool's DEdgeMask(), together with pen & paper, and produce the frame shifts yourself through an operation on a 3x3 kernel:
0 0 1
0 2 0
0 0 0
with normalization 3 gives a shift of 1/3 to the upper right.
Since this is simple bilinear sampling, you'll loose sharpness as well. Perhaps upsample first to 2x with lanczos, shift by 2x of the initial target vector, and downsize back.

[SpikeSpiegel]

Originally posted by Didée
No, alas not.

But you can take either GeneralConvolution() or MaskTool's DEdgeMask(), together with pen & paper, and produce the frame shifts yourself through an operation on a 3x3 kernel:
0 0 1
0 2 0
0 0 0
with normalization 3 gives a shift of 1/3 to the upper right.
Since this is simple bilinear sampling, you'll loose sharpness as well. Perhaps upsample first to 2x with lanczos, shift by 2x of the initial target vector, and downsize back.

Good idea, but there is another solution:
throw away PointResize.

The "culprit" wasn't SangNom: using lanczos4resize to upsize, there's no shift at all.

Thanks anyway, this method is really faster and I'm sure it will be useful in other situations.

[joshbm]

I tried to use your script... It's waaaay slow under 1 fps (lucky if it hits 1) with my Deinterlacer before that (usually get around 8-9fps with my entire script and 29-33 fps with just the deinterlacer).

It's definately *not* my computer specs:

3.2 Ghz P4
1 GB ram
300+ gb storage
etc.

Regards,
Josh

[ChronoCross]

not suprising.....it is designed for mfspeed. Make sure that you have the latest versions of the dll's. Try a simple script like

Source
Deinterlace
AAA(640,480) ///anti-aliasing script

and see what speeds you get. if it's still 0-1fps with your pc specs then you should try a different version of the dll's...sometimes older dll's run faster(don't ask me why). if it's not that then there might be another culprit behind the slow speed.

Q: do you have HT technology?


EDIT: I almost forgot to add that I can't even encode with that script. my weak A** machine can't handle it lol. This one might be the all time slowest script I have ever used...even worse than mftoon.

[Soulhunter]

Originally posted by ChronoCross

This one might be the all time slowest script I have ever used...even worse than mftoon.

Wow, think I gonna keep this quote... :D


Thanks n' Bye

[ChronoCross]

Originally posted by Soulhunter
Wow, think I gonna keep this quote... :D


Thanks n' Bye

sweet. no joke though I can't test this on a more than 10 second clip...it's bad enough that it takes an hour per pass for that lol. but the results were what I wanted but until I get a new machine I couldn't even begin to do full episode tests.

at least with mftoon I got a solid 1-2fps lol. but good job on this script

[Soulhunter]

Originally posted by ChronoCross

but good job on this script

Thank Didée, he explained me how to write this script... ;)


Bye

[joshbm]

Originally posted by ChronoCross
Q: do you have HT technology?


Yes. I was somewhat impressed with the results of the script, but when the script takes hours to render seconds I'm definately not at all remotely interested.

:p

[ChronoCross]

try disabling HT by going into the taskmanger while encoding and right clicking on the vdubmod process. That might make a difference from what I've read in other threads.

[SpikeSpiegel]

Originally posted by ChronoCross
EDIT: This one might be the all time slowest script I have ever used...even worse than mftoon.
...then, I suppose you haven't already used antialiasing():

if you're REALLY lucky you can get 15, 20...even 25SPF (=Seconds Per Frame ) :D

[ChronoCross]

Originally posted by SpikeSpiegel
...then, I suppose you haven't already used antialiasing():

if you're REALLY lucky you can get 15, 20...even 25SPF (=Seconds Per Frame ) :D

lol either way both scripts are too slow for me to implement on my current rig.

[Didée]

ChronoChross, SpikeSpiegel:

What are you guys doing?

Just now I'm sitting at a P3 800MHZ. Feeding a 720*432 source (but w/o any deinterlacing, just applying the plain AA functions), AntiAliasing() is running @ 1.1 fps (HQedge) resp. 1.8 fps (sobel). AAA() is running @ 2 fps with everything turned on.

On more recent machines, these functions should run in the range of 4~10 fps.

Are you talking of some other script(s) than those posted in this thread?

[ChronoCross]

negative Didée. I tried both. the AAA mainly and it barely ran.....this is what the script looks like when I use it

NTSC Material

MPEG2Source("blah.d2v")
Telecide(Order=1,Guide=1).Decimate(cycle=5)
crop(numbers inside)
AAA()

that's it. 0fps-1fps barely. my setup isn't exactly good....please note that it's a intel mobile pentium 3 with it running most of the vob's off a USB 2.0 External Drive.

my average encoding speed is 8fps.
my max encoding speed ever(no filters) 24fps
mftoon speed 1-2fps
AAA speed 0-1fps

[lamer_de]

Hm, works pretty well:

http://s04.imagehost.org/0070/source_jpg.t.jpg (http://s04.imagehost.org/view.php?image=/0070/source.jpg)http://s04.imagehost.org/0070/anti-aliased_jpg.t.jpg (http://s04.imagehost.org/view.php?image=/0070/anti-aliased.jpg)

However, I had to use two instances of the filter to get rid of the aliasing, so it indeed run pretty sluggish (arund 1-2fps on athlon xp 1700). The high sharpening value of 50 caused edge halos though, so different sources might need different sharpening values.

CU,
lamer_de

[LB]

Originally posted by lamer_de
Hm, works pretty well:

http://s04.imagehost.org/0070/source_jpg.t.jpg (http://s04.imagehost.org/view.php?image=/0070/source.jpg)http://s04.imagehost.org/0070/anti-aliased_jpg.t.jpg (http://s04.imagehost.org/view.php?image=/0070/anti-aliased.jpg)

However, I had to use two instances of the filter to get rid of the aliasing, so it indeed run pretty sluggish (arund 1-2fps on athlon xp 1700). The high sharpening value of 50 caused edge halos though, so different sources might need different sharpening values.

CU,
lamer_de

To eliminate the halos try

AAA()
AAA()
HQDering(255)
HQDering(100)

Working friggin amazing, but haha I think I get negative fps speed.

[lamer_de]

It's way easier to just tone down the sharpening values when using AAA if you get halos :p Where's the point in sharpening when it creates so much artifacts you need a smoother to get rid of it? As I said, Soulhunters values which he made the standard seem to work ok for his sources or if only one instance of AAA is used. But yeah, HQDering does it's job on removing halos, although I like BlindDeHalo() better for that purpose. Often a bit harder to set up correctly, though.

The screenshots I posted were made with Unfilter values of 7 respectively 5 for the second instance.

CU,
lamer_de

[You Know]

is possible crate a dll filter for avisynth for antialias using directX function? I do not know if is possible apply directx function out the 3d poligon contex, for example on images or images loaded as texture on rectangle of (XXX, YYY) size.


should be more more fastest than a CPU implementation

[foxyshadis]

fft3dgpu does just that, so it's obviously possible. (Actually I don't know if it uses opengl or directx, but it's the same general idea.) As long as you pull in one or more frames and push one back out, you can do anything you can write in C++ to it (or C, or Delphi, etc). You might want to start from the fft3dgpu code if you intend to write something like that.

[Ramromulus]

Could someone post an example usage for SAA(), I can't seem to get it to work, avisynth keeps giving me invalid arguments or no such function.

[Boulder]

Do you have the SangNom and MaskTools v1.5.x dlls in your plugins folder?

[Ramromulus]

yes im just unclear how to set the width and height function

[Boulder]

What's your script and what exact error message do you get?

[legionator2]

I have tested some anti-aliasing script. Here are some pics :

Source :
http://i23.servimg.com/u/f23/11/31/41/30/shot210.jpg

AAA(xres=1024, yres=576, xshrp=20, yshrp=20, us=1, ds=2, chroma=true) :
http://i23.servimg.com/u/f23/11/31/41/30/shot410.jpg

AntiAliasing(th_luma=20, th_croma=20) :
http://i23.servimg.com/u/f23/11/31/41/30/shot510.jpg

SAA():
http://i23.servimg.com/u/f23/11/31/41/30/shot310.jpg

SAA()
SAA()
http://i23.servimg.com/u/f23/11/31/41/30/shot111.jpg

It seems like the SAA() script of Soulhunter gave the best result here. :) It even got rid of the halos, though the video looks a bit more blurry. It's a very nice function, but all of the anti-aliasing scripts seems to be slow like hell. I wonder if there's an equivalent method, but faster than this.

[foxyshadis]

You can get some low-budget anti-aliasing with TIsophote, or Lanczos4Resize(width*2,height*2).BilinearResize(width,height), but they don't look nearly as good as the slower methods.

[*.mp4 guy]

frfun7(1.01, 256, 256) Will give some very powerful, relatively quick antialiasing, but it blurs alot.

[Soulhunter]

frfun7(1.01, 256, 256) Will give some very powerful, relatively quick antialiasing, but it blurs alot.
A hint to write a "FRFun7@EdgesOnly" AA function? ;]


Thx n' Bye

[*.mp4 guy]

Nah, I actually already tried that, its not neccesarily any better or faster then other aproaches, once you take care of certain dificulties (mainly all the extra blur, which can cause problems with masking), which is why I never mentioned it before; Of coarse thats not to say someone else coudn't do a better job then me at it and make it competitive. It is, however, usefull without masking, on already "ruined" sources where you need extremely strong antialiasing.

[jethro]

A hint to write a "FRFun7@EdgesOnly" AA function? ;]


Thx n' Bye

Yes, edges like in msharpen mask with adjustable detection...

[Socio]

I tried SAA and antialiasing they are indeed hella slow and got me thinking;

If video games can use the antialiasing features on video cards, in some cases even be forced to, could there not be a way for Avisynth to do the same and deinterlacing as well for that matter.

The speed increases alone would be phenomenal I would think.

[Dark Shikari]

I tried SAA and antialiasing they are indeed hella slow and got me thinking;

If video games can use the antialiasing features on video cards, in some cases even be forced to, could there not be a way for Avisynth to do the same and deinterlacing as well for that matter.

The speed increases alone would be phenomenal I would think.Antialiasing on video cards relies on being able to render many samples per pixel and average them. You can't do this when you aren't rendering 3D graphics; when you're just using an ordinary input image, there's no real way to render more samples.

[Socio]

Antialiasing on video cards relies on being able to render many samples per pixel and average them. You can't do this when you aren't rendering 3D graphics; when you're just using an ordinary input image, there's no real way to render more samples.

Maybe one could have it take samples from 30 frames ahead and 30 frames back to get enough samples but that would probably require a lot more ram for that data and make it slower than the AA scripts we have now.

[scharfis_brain]

that's what is called temporal super resolution (TSR).
This also had been discussed here several years ago

http://forum.doom9.org/showpost.php?p=808013&postcount=19

http://forum.doom9.org/showthread.php?t=81868&highlight=tsr

[Chengbin]

Where do I get the function "edgemask"?

EDIT: Nevermind, I needed masktool.dll, apparently it didn't work with masktool 26.

Wait, how do I use this? I wrote antialiasing(), and it had no effect.

[maki]

Hi can someone help with this:

This is without SAA

import("C:\Program Files\AviSynth 2.5\plugins\AAA.avs")
import("C:\Program Files\AviSynth 2.5\plugins\SAA.avs")
import("C:\Program Files\AviSynth 2.5\plugins\Antialiasing.avs")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\SangNom.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\UnFilter.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")

MPEG2Source("C:\Documents and Settings\HP_Administrator\My Documents\Vocabulary\air sample.index\air sample.d2v",cpu=0,info=3)
AAA(xres=1024, yres=576, xshrp=20, yshrp=20, us=1, ds=2, chroma=true)
AntiAliasing(th_luma=20, th_croma=20)

http://i49.tinypic.com/2zoho55.jpg

http://i48.tinypic.com/2yl8gog.jpg

This is with SAA:

import("C:\Program Files\AviSynth 2.5\plugins\AAA.avs")
import("C:\Program Files\AviSynth 2.5\plugins\SAA.avs")
import("C:\Program Files\AviSynth 2.5\plugins\Antialiasing.avs")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\SangNom.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\UnFilter.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")

MPEG2Source("C:\Documents and Settings\HP_Administrator\My Documents\Vocabulary\air sample.index\air sample.d2v",cpu=0,info=3)
AAA(xres=1024, yres=576, xshrp=20, yshrp=20, us=1, ds=2, chroma=true)
AntiAliasing(th_luma=20, th_croma=20)
SAA()

http://i48.tinypic.com/mvn6v7.jpg

http://i49.tinypic.com/v5ctwh.jpg (http://i49.tinypic.com/v5ctwh.jpg)

So here's my question I believe the one without SAA to be better, which should I chose and also, I want to keep the resolution at 1920x1080, if isn't possible at least 1280x720. I know I need to change something so when I keep the resolution at 1920x1080 or 1280x720 it would be of the same quality as the one above.

What do I need to change here:

AAA(xres=1024, yres=576, xshrp=20, yshrp=20, us=1, ds=2, chroma=true)
AntiAliasing(th_luma=20, th_croma=20)

[canuckerfan]

^you're calling an anti-aliasing filter multiple times in both attempts. unless you've got some serious aliasing you won't need so many calls. post a sample of what you're encoding and we can have a better look at how to tackle this.

but for what it's worth I think the one without SAA looks betters.

as for your resolution is your source 1024 by 576? in that case I'd do something like this:

AAA(xshrp=20, yshrp=20, us=1, ds=2, chroma=true)
Spline36Resize(1920,1080)
AntiAliasing(th_luma=20, th_croma=20)

but as always it really is up to how you wanna do this. btw you might wanna check out MCTemporalDenoisePP(). it's AA feature is also quite good.

[Keiyakusha]

maki
Looks like this is Air BD. To be honest I don't see a single reason why you want keep it 1080p. There is no details for that. This is actually upscale from SD... so you should be fine with 720p. Especially with this horrible source since it needs heavy filtering. Unprocessed sample would be fine, maybe there is some tricks and someone will be able to suggest you a better script.

Anyway you probably need to change this:
AAA(xres=1280, yres=720, xshrp=20, yshrp=20, us=1, ds=2, chroma=true)

[maki]

Okay so here's the original mpv file this file is cut, so it's around 2 minutes long, I know that the size is big for a 2 minute video, but I believe that starting with the source is the way to go. Here you go:

http://www.megaupload.com/?d=7HGV2SL0

Here's a m2st file. It's around 86 mb. If this work as well than here:

http://www.megaupload.com/?d=BQ0LSSG1

Can someone help me here.

[maki]

Ok so I changed the resolution and this how it looks:

http://i49.tinypic.com/5o5k4g.jpg



Do I need to change anything---xshrp=20, yshrp=20, us=1, ds=2, chroma=true----- and ----AntiAliasing(th_luma=20, th_croma=20)----- so that it might look a little bit better, or does anyone know which filter can improve the video to a much better quality?

Can someone please help me here.

[martre]

Hello, your samples weight to much, almost 500mb. For what i see on your last picture your dvd is grainy and need chroma cleaning. For the colours you may use something like:

coloryuv(levels="tv->pc",gain_u=25).tweak(hue=10,sat=1.2,cont=.9)

You can keep on playin with the numbers yourself.
saluts

[maki]

Right now I just want to change the AntiAliasing. I want to get rid of the jitteriness outlining the characters. I want smooth black lines.

Like here:

http://i48.tinypic.com/2yl8gog.jpg

Except I want the resolution to be at 1280x720.

I used your filter, and this is what I got:

http://i46.tinypic.com/2nklv60.jpg

import("C:\Program Files\AviSynth 2.5\plugins\AAA.avs")
import("C:\Program Files\AviSynth 2.5\plugins\SAA.avs")
import("C:\Program Files\AviSynth 2.5\plugins\Antialiasing.avs")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\SangNom.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\UnFilter.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")

MPEG2Source("C:\Documents and Settings\HP_Administrator\My Documents\Vocabulary\air sample.index\air sample.d2v",cpu=0,info=3)
AAA(xres=1280, yres=720, xshrp=20, yshrp=20, us=1, ds=2, chroma=true)
AntiAliasing(th_luma=20, th_croma=20)
coloryuv(levels="tv->pc",gain_u=25)

I took out----.tweak(hue=10,sat=1.2,cont=.9)----. the color was too dark.

It's an improvement from this:

http://i49.tinypic.com/5o5k4g.jpg

A filter that might improve AntiAliasing would be nice. Any ideas?

[Lyris]

Thanks to the script writers - I used this on a source which had bad aliasing and DV compression artefacts which is going through CCE SP2 for DVD.

Before (200% pixel resize):

http://lyris-lite.net/temp/pre.jpg

After:

http://lyris-lite.net/temp/post.jpg

Loss of detail is minimal, pleased with the results. Shame the guys who shot it couldn't supply a lossless version when asked.

[Porsche_fan]

Go easy on me this is my first attempt to share something.:) All the credit goes to SpikeSpiegel & Didée among others. I am just sharing some modifications I made to the original script which I have found helpful…especially for HD content.

The main difference is the ability to fine tune the supersampling factor to include factional numbers instead of being locked into whole numbers. Fractional numbers help mitigate the trade off in supersampling. The larger the supersampling the less negative effects such as blurring but also the less aliasing is correctable…it is a balancing act.

I also prefer unsharpmask as a sharpener and positioned it in the script so sharpening is supersampled to reduce unwanted side effects. LimitedSharpenFaster is commented out for those who prefer it.

Lastly, I have found the original version of SangNom is about 4x faster on my quadcore than the more current versions. Unfortunately it does have a very slightly lower quality but it is hardly perceivable. http://mf.creations.nl/avs/filters/SangNom_old2.dll



# AAAmod
#
# Thanks @ SpikeSpiegel, Didée, mf, Akirasuto & ScharfisBrain...
#________________________________________________________________________________________
#
# Usage: AAAmod(Xres, Yres, Shrp, Us, Ds, chroma, Order, AAstr, ss_x, ss_y)
#________________________________________________________________________________________
#
# Xres/Yres = The final resolution... InputSize = OutputSize is the default
#
# Shrp = Unsharpmask strength... 30 is the default
#
# Us = Resizer for upsampling... 0 = PointResize (default) / 1 = Lanczos
#
# Ds = Resizer for downsampling... 0 = Bilinear (default) / 1 = Bicubic Catmull-Rom / 2 = Lanczos
#
# Chroma = Enable/disable chroma antialiasing... Disable = false (default) / enable = true
#
# Order = 0=Top Field, 1= Bottom Field, 2= Bob... default is 1
#
# AAstr = Anti-Aliasing strength in Sangnom... default is 48
#
# ss_x = Supersampling coefficient for x effective range 2.0-3.0... if using Pointresize then set to 2.0
#
# ss_y = Supersampling coefficient for y effective range 2.0-3.0... if using Pointresize then set to 2.0
#
#________________________________________________________________________________________
#
# Example...
#
#Import("C:\Programme\AviSynth 2.5\plugins\AAAmod.avs")
#
# AAAmod(656,480,40,1,2,chroma=false,order=1,AAstr=48,ss_x=2.15,ss_y=2.25)
#________________________________________________________________________________________

function AAAmod(clip clp, int "Xres", int "Yres", int "Shrp",
\ int "US", int "DS", bool "chroma", int "order", int "AAstr", float "ss_x", float "ss_y")
{
clp = clp.isYV12() ? clp : clp.ConvertToYV12()
ox = clp.width
oy = clp.height
Xres = default(Xres, ox)
Yres = default(Yres, oy)
Shrp = default(Shrp, 30)
us = default(us, 1)
ds = default(ds, 2)
chroma = default(chroma, false)
order = Default(order, 1) # 0=Top Field, 1= Bottom Field, 2= Bob
AAstr = Default(AAstr, 48)
ss_x = default(ss_x, 2) #For pointresize use 2.0
ss_y = default(ss_y, 2) #For pointresize use 2.0

xxs=round(ox*ss_x/8)*8
yys=round(oy*ss_y/8)*8





us==0 ? clp.PointResize(xxs,yys) : clp.Lanczos4Resize(xxs,yys)

TurnLeft()
SangNom(order,AAstr)


TurnRight()
SangNom(order,AAstr)



UnsharpMask(Shrp)
#LimitedSharpenFaster(ss_x=1.0,ss_y=1.0, Smode=4, strength=200)

ds==0 ? BilinearResize(Xres,Yres) :
\ ds==1 ? BicubicResize(Xres,Yres,0,.5) :
\ LanczosResize(Xres,Yres)



chroma ? MergeChroma(clp.Lanczosresize(Xres,Yres)) : last

}

[maki]

Hi me again XD!!

Can someone help!

I wan to fix the lines of this video. Here's a pic:

http://img684.imageshack.us/i/yuki1.png/

And also I used warpsharp at depth=50 and I was wondering if it's too much. Here's the pic with warpsharp, and also this is the one I want to edit:

http://img710.imageshack.us/i/yuki2b.png/

I meant to say-I want to fix the jagginess of the eyebrows and glasses and I know I need to de_halo it, but right I just want to fix the lines.

another thing here's script and I want to ask why does this happen( I am still a noob at this, but I try to read a little everyday, so please be patience with me XD).

P.S. sorry for posting the pics its just that imageshack wouldn't let me link to the image, I will fix it when they fix the problem.

without sharpaamcmod():

deblock_qed( quant1=25,quant2=25, aOff1=8,bOff1=8,aOff2=8,bOff2=8, uv=3 )
o = last
DH = o.dehalo_alpha(rx=1.75,ry=2.8,darkstr=0)
bb = DH.removegrain(11)
xD = mt_makediff(bb,bb.repair(bb.repair(bb.medianblur(2,-333,-333),1),1))
\ .mt_lut("x 128 - 2.49 * 128 +")
xDD = mt_lutxy(xD,mt_makediff(o,DH),"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")

DH.mt_adddiff(xDD,U=2,V=2)

FluxSmoothST(7,7)
WarpSharp(depth=20)
HQDering(strength=255, overall =0)

LimitedSharpenfaster( ss_x=1.0, ss_y=2.0, dest_x=1280, dest_y=720,
\ Smode=3, strength=255, radius=0,
\ Lmode=1, wide=true, overshoot=1,
\ edgemode=1, special=false,
\ exborder=1)

#deinterlace
#crop
LanczosResize(1280,720) # Lanczos (Sharp)
sharpaamcmod()
#denoise
ConvertToYV12()

http://img407.imageshack.us/i/yukiwithoutsharpmod.png/
(http://img407.imageshack.us/i/yukiwithoutsharpmod.png/)





with sharpaamcmod()

deblock_qed( quant1=25,quant2=25, aOff1=8,bOff1=8,aOff2=8,bOff2=8, uv=3 )
o = last
DH = o.dehalo_alpha(rx=1.75,ry=2.8,darkstr=0)
bb = DH.removegrain(11)
xD = mt_makediff(bb,bb.repair(bb.repair(bb.medianblur(2,-333,-333),1),1))
\ .mt_lut("x 128 - 2.49 * 128 +")
xDD = mt_lutxy(xD,mt_makediff(o,DH),"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")

DH.mt_adddiff(xDD,U=2,V=2)

FluxSmoothST(7,7)
WarpSharp(depth=20)
HQDering(strength=100, overall =0)

LimitedSharpenfaster( ss_x=1.0, ss_y=2.0, dest_x=1280, dest_y=720,
\ Smode=3, strength=255, radius=0,
\ Lmode=1, wide=true, overshoot=1,
\ edgemode=1, special=false,
\ exborder=1)

#deinterlace
sharpaamcmod()
#crop
LanczosResize(1280,720) # Lanczos (Sharp)
sharpaamcmod()
#denoise
ConvertToYV12()

P.S. I changed HQ to 100 I gain some of the dark line quality since this image.

I know that it isn't limitedsharpen since I changed each one of its settings.

http://img35.imageshack.us/i/yukiwithsharpmod.png/

and also is it possible to get the results of the second image without the loss of details and curvature lines.

Oh! look what I found. Somehow the same effect as the above image, but with jagginess and a bit thicker lines:

deblock_qed( quant1=25,quant2=25, aOff1=8,bOff1=8,aOff2=8,bOff2=8, uv=3 )
o = last
DH = o.dehalo_alpha(rx=1.75,ry=2.8,darkstr=0)
bb = DH.removegrain(11)
xD = mt_makediff(bb,bb.repair(bb.repair(bb.medianblur(2,-333,-333),1),1))
\ .mt_lut("x 128 - 2.49 * 128 +")
xDD = mt_lutxy(xD,mt_makediff(o,DH),"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")

DH.mt_adddiff(xDD,U=2,V=2)
FluxSmoothST(7,7)
WarpSharp(depth=20)

HQDering(strength=255, overall =0)
LimitedSharpenfaster( ss_x=1.0, ss_y=2.0, dest_x=1280, dest_y=720,
\ Smode=3, strength=255, radius=0,
\ Lmode=1, wide=true, overshoot=1,
\ edgemode=1, special=false,
\ exborder=1)
function ESharpAAMC(clip orig, float "ds", int "ShPre", int "ShPost", int "SmPost")
{
ds=default(ds,0.2) #strokes darkening strength
ShPre=default(ShPre,80) #Presharpening
ShPost=default(ShPost,300) #Postsharpening
SmPost=default(SmPost,100) #Postsmoothing

a=orig.assumetff()
w=width(a)
h=height(a)
m=logic( a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false)
b=a.Toon(ds).Warpsharp(depth=ShPre)
b=b.TurnLeft().EEDI2().TurnRight().EEDI2().spline64resize(w,h,0.5,-0.5,2*w+.001,2*h+.001)
b=b.LimitedSharpenFaster(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost)


mt_merge(a,b,m,Y=3,U=3,V=3)
sD=mt_makediff(a,last)
fv2 = a.MVAnalyse(isb=false,delta=2,idx=13)
fv1 = a.MVAnalyse(isb=false,delta=1,idx=13)
bv1 = a.MVAnalyse(isb=true, delta=1,idx=13)
bv2 = a.MVAnalyse(isb=true, delta=2,idx=13)

sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14)

reduc = 0.4
sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc)

a.mt_makediff(sDD,U=2,V=2)

return last
}

LanczosResize(1280,720) # Lanczos (Sharp)
#denoise
ConvertToYV12()

http://img715.imageshack.us/i/yukiwithnewscript.png/


I still prefer the second option. Also I still want to remove the jaggies. Help would be greatly appreciated.

[list]

can't see any artifact there
BTW...
#FASTER ANTI ALIASING FILTER
#REQUIERED:
# SangNom
# mt_masktools

function FAA(clip input,int "str") {
str = Default(str, 50)
vid = input.IsYV12() ? input : input.ConvertToYV12()
w=width(input)
h=height(input)

aa=vid.pointResize(w*2,h) . TurnLeft() . SangNom(AA=str) . TurnRight(). lanczosResize(w,h)

return mt_merge(vid , aa , vid.mt_edge().mt_inflate() )
}
Maybe it's not the best anti aliasing filter, but 3x faster, and the first i can use in real time with some sources.

[MenotBumba]

http://img196.imageshack.us/img196/8300/snapshot171720111004222.png

I used these two lines, but the aliasing problem is still not solved as you can see on the screenshot above.

AnimeIVTC(mode=1,aa=4,killcomb=2,chrfix=false,ifade=false)
AAA(640,480,15,15,1,2,false)

Anyone knows how to fix it?

[mastrboy]

That looks more like a dotcrawl problem, little hard to tell by an image only...

Have a look at Tcomb(mode=0) or Checkmate()... If i remember correctly setting chrfix=true in AnimeIVTC also activates some dotcrawl removal? Or was it derainbowing?

[Bloax]

Oh hey. I for some reason decided to mess around with this little script.

The end result ended up being twice as fast, though only because I reduced the initial resizing. :)

Well, basically you get a pair of extra options, and that's about it. (You can see it here.) (http://pastebin.com/4nymD72m) <- Cleaned the post up a bit.

Oh shameless me, but I also wanted to stabilize the antialiasing. (Which is pretty spatial to me.)

So I ended up doing my very first 'real' script edit. Feel free to flame the hell out of my clumsy and useless edit. :thanks:

(Found here) (http://pastebin.com/p5DmUXzk)

All I seem to accomplish is create quite a bunch of artifacts.
Though it does seem a bit more stable.

I'm such a shameless bugga. ;4

Edit:I made a new script today, with most of the dirty masktools business similar to these. (Updated, old version here. (http://pastebin.com/Aa95r4MN))
EditEdit: Another update, old version (Here) (http://pastebin.com/sAGWtZwH)
Edit^3: Updated once again, old script here. (http://pastebin.com/pYZgsAs3)


function m4(float x) {return(x<16?16:int(round(x/4.0))*4)} # Haha, didn't realize this wasn't internal. :p
# Me and my giant stash of scripts. (Well, more or less.)

function AntiAliasRG(clip orig,float "sx", float "sy",string "mode",int "thrl",int "thrc",bool "sharpen",float "sharpstr",bool "sharpmode") {

# Fast antialiasing using RemoveGrain. (It is dull no more!)

# > Requirements: MaskTools2, RemoveGrain(SSE3).dll, Repair(SSE3).dll <- For the sharpening.
# By Bloax, based on the "Antialiasing" script (>now pretty much original), authors of which are; SpikeSpiegel, Didée, mf, scharfis_brain and Soulhunter.

# This script is rather strong, especially when both sharpen and sharpmode are "true". At that point it looks like MSAA, which is quite sharp.
# It's also appliable to sources around 800x600 /before/ downscaling. ^ It's to be noted that it also denoises the video a bit.

# As of now, it seems better than the SangNom ones. Especially with the spooky results from sharpen/sharpmode=true.
# ^ Confirmed stuff, which turned out to be quite positive.

# UpdateUpdate: "sx" And "sy" are now the options to prevent (extreme) text smearing.
# Setting them below 1 works fine, though the speed gains aren't too big. (And the smearing gets much worse.)
# ^ Now, due to the addition of sharpen=true, in combination with sharpmode=true, you can actually get a nice speed boost at sx/sy=0.8 at next to no quality loss!

thrl = Default(thrl, 10) # Luma threshold for the mask(s).
thrc = Default(thrc, 10) # Chroma threshold for the mask(s). (Color, useless?)
sharpen = Default(sharpen, true) # See below.
sharpstr = Default(sharpstr, 0.4) # Added a sharpening filter. Experimental. (Effective though.)
sharpmode = Default(sharpmode, false) # Makes the edge mask use a sharper (custom) kernel. This nullifies the "mode" option though.
# Also degrades the performance a bit because of an extra blur filter. (Because why not.)
ox = orig.width
oy = orig.height
sx = Default(sx, 1.33) # Horizontal supersampling, reducing either to <1 will make text almost unreadable.
sy = Default(sy, 1.33) # Vertical supersampling, ^ unless you don't have sharpen=true, then it's alright to some degree.
mode = Default(mode, "laplace") # Kernel for detecting edges, I prefer this one.
# Possible strings (mode=>"option"<): roberts (sharp), sobel (too sharp), laplace (pretty soft), prewitt (iseethemedges), hprewitt (prewitt light?)
clp = orig.IsYV12() ? orig : orig.ConvertToYV12() # Is the source in YV12? If not, convert to it. (Shameless ripoff!)

##############

a=clp.RemoveGrain(mode=1) # Subpixel noise removal added.
b=a.BilinearResize(m4(ox*sx),m4(oy*sy))
c=sharpmode ? b.RemoveGrain(mode=5).RemoveGrain(mode=21).RemoveGrain(mode=19).BilinearResize(ox,oy).RemoveGrain(mode=22) : b.RemoveGrain(mode=5).RemoveGrain(mode=21).BilinearResize(ox,oy).RemoveGrain(mode=22)
# RemoveGrain mode 5 is for removing noise, mode 19 is a blurring filter (No longer used), and 21/22 are kind of like SangNom antialiasing.
# sharpmode=true Adds a pass of mode 19, because why not while we're at it.
# Well, except that they're insanely fast. ;)
e=sharpmode ? a.RemoveGrain(mode=0,modeu=-1).Mt_Edge("2 3 3 2 -18 2 3 3 2",thy1=0,thy2=thrl,thc1=0,thc2=thrc).Mt_Inflate().RemoveGrain(mode=9,modeu=-1).Mt_Deflate() : a.RemoveGrain(mode=0,modeu=-1).Mt_Edge(mode=mode,thy1=0,thy2=thrl,thc1=0,thc2=thrc).Mt_Inflate().RemoveGrain(mode=9,modeu=-1).Mt_Deflate()
# Expanding the mask quite a bit, just to make sure the antialiasing gets a more or less smooth area.
# Also making the mask Luma only. (ModeU=-1) > Update: sharpmode=true, as mentioned, uses a custom kernel that's sharper than laplace. (But otherwise pretty identical.)
f=sharpen ? Mt_Merge(a,c,e).Sharpen(sharpstr).Repair(a,mode=9) : Mt_Merge(a,c,e) # Do we require sharpening? If yes, do this. If no, do the same thing without it.
# Why do I even have two Mt_Merge? < To reduce the blurring perhaps?
Mt_Merge(a,f,e).RemoveGrain(mode=22) # Killing the jaggies that got past the mask. ;)
}

This one uses RemoveGrain(SSE3).dll instead of SangNom, however. So it's less prone to make bizarre artifacts from subpixel noise. (Now thanks to prefiltering those out.)
So basically this is a softer antialiasing script (that looks pretty nice in my eyes). I may add an option for additional softening to it though.

Enjoy!

Third (latest) update: Changed the MaskTools stuff to MaskTools2 stuff. Which sped it up by a bit.
Also added sub-pixel aliasing removal by using RemoveGrain(mode=1). Edit: Also added Mt_Deflate() at the end of the edgemask to make the script slightly sharper.
EditEdit: Also added sharpening, at the price of a bit of performance. (140-155 FPS before, 130-140 after.)
These two latest edits should improve cases of overshooting.
^ Edit: Also added sharpmode, which uses a custom kernel to get a sharper mask. With sharpen=true it gets 120-127 FPS though.

Still, whereas I got 60 FPS with my lousy edit of the initial AntiAliasing(), I usually get about 130-140 FPS on the same source with this one. :)

Second update: Changed the RemoveGrain modes, ended up getting a very nice, and really fast antialiasing filter.
Also made the mask a bit wider and blurrier by using RemoveGrain(mode=19) on it, and by removing the Deflate() at the end.

First update: Cleaned some useless parts up, changed re-resizing to BilinearResize (instead of Gauss), and tweaked the preset settings a bit.
Also tweaked the mask a bit. Overall, it's faster, and a bit prettier. (Thanks to sx/y tweaking.)

P.S: If you want a pretty nice, and absolutely terrifyingly fast antialiasing filter, use this:
RemoveGrain(mode=1).RemoveGrain(mode=5).RemoveGrain(mode=22).RemoveGrain(mode=21)

Mode 1 to kill the smaller noise, mode 5 to kill the bigger noise, and 22 and 21 for the antialiasing. (Which make funky shapes out of the now removed noise.)

Update: Here's the ultimately simple AA script:
RemoveGrain(21,19,19).RemoveGrain(22)

[MenotBumba]

I have tested some filters(TComb, GuavaComb, DeCrawl etc.) and I got the best quality/speed results using TComb(mode=2) (when I put chrfix=true for AnimeIVTC, the encode was too slow - ~1fps). With TComb(mode=2) and chrfix=false I got ~ 3 fps :)

[naoan]

I like edge detection from lsfmod better than any kernel, a comparison :

laplace : http://i.imgur.com/sIszB.png
lsfmod : http://i.imgur.com/hLjDG.png

[luke83]

function AntiAliasRG(clip orig,float "sx", float "sy",string "mode",int "thrl",int "thrc") {

# Fast antialiasing using RemoveGrain. (It is dull no more!)

# > Requirements: MaskTools2, RemoveGrain(SSE3).dll
# By Bloax, based on the "Antialiasing" script (stealing parts of ;)), authors of which are; SpikeSpiegel, Didée, mf, scharfis_brain and Soulhunter.

# Note: ?Confirm?? This script is pretty weak, so tough edges (huge, curved ones) end up looking plain blurred. < It's rather strong now.
# Though if said source is downscaled, everything looks alright. (More or less better than just downsized.) < Applying it BEFORE downscaling gives superior results, for the price of some speed.

# ?Confirm?? It is to be noted that this script loses to the SangNom ones in slightly aliased cases. (Eg. Warcraft 3: The Frozen Throne title screen.)
# Though for eliminating tons of aliasing in a noisy game (Eg. Doom), this is certainly better than the usual SangNom ones.
# (Well, not so much now.) Due to the fact that it doesn't smoothen out the noise to abstract shapes, and instead eliminates it. (RemoveGrain. :))

# Due to recent changes the above may or may not be absolutely true. :-)
# (Because I'm totally way too lazy to rewrite all of this!)

# UpdateUpdate: "sx" And "sy" are now the options to prevent (extreme) text smearing.
# Setting them below 1 works fine, though the speed gains aren't too big. (And the smearing gets much worse.)

thrl = Default(thrl, 10) # Luma threshold for the mask(s).
thrc = Default(thrc, 10) # Chroma threshold for the mask(s). (Color)
ox = orig.width
oy = orig.height
sx = Default(sx, 1.33) # Horizontal supersampling, reducing either to <1 will make text almost unreadable.
sy = Default(sy, 1.33) # Vertical supersampling, once again, setting both of those results in getting a slower RemoveGrain(mode=21) filter.
mode = Default(mode, "laplace") # Kernel for detecting edges, I prefer this one.
clp = orig.IsYV12() ? orig : orig.ConvertToYV12() # Is the source in YV12? If not, convert to it. (Shameless ripoff!)

##############

a=clp.RemoveGrain(mode=1) # Subpixel noise removal added.
b=a.BilinearResize(m4(ox*sx),m4(oy*sy))
c=b.RemoveGrain(mode=5).RemoveGrain(mode=21).BilinearResize(ox,oy).RemoveGrain(mode=22)
# RemoveGrain mode 5 is for removing noise, mode 19 is a blurring filter (No longer used), and 21/22 are kind of like SangNom antialiasing.
# Well, except that they're insanely fast. ;)
e=a.RemoveGrain(mode=0,modeu=-1).Mt_Edge(thy1=0,thy2=thrl,thc1=0,thc2=thrc,mode=mode).Mt_Inflate().RemoveGrain(mode=9,modeU=-1)
# Expanding the mask quite a bit, just to make sure the antialiasing gets a more or less smooth area.
# Also making the mask Luma only. (ModeU=-1)
# > Switching to MaskTools2 ended up making the smoothening better.
f=Mt_Merge(a,c,e) # I'll probably be messing around with those. < Why do I even have two of these?
Mt_Merge(a,f,e).RemoveGrain(mode=22) # Killing the jaggies that got past the mask. ;)
}

This one uses RemoveGrain(SSE3).dll instead of SangNom, however. So it's less prone to make bizarre artifacts from subpixel noise. (Now thanks to prefiltering those out.)
So basically this is a softer antialiasing script (that looks pretty nice in my eyes). I may add an option for additional softening to it though.

Third (latest) update: Changed the MaskTools stuff to MaskTools2 stuff. Which sped it up by a bit.
Also added sub-pixel aliasing removal by using RemoveGrain(mode=1).

Still, whereas I got 60 FPS with my lousy edit of the initial AntiAliasing(), I usually get about 130-140 FPS on the same source with this one. :)


Hi. I think your script would be amazing... If I could use it XD

It gives me an error window in witch it says that "m4 function doesn't exist"
In fact I looked for it in the script, but I didn't found it; or well: I found where you use it but I hasn't be declared before. I "loaded" in the avs script mt_masktools 2 and 26, RemoveGrainSSE3 0.9 and 1.0 but It doesn't work :(

This is my avs script

Where I make a mistake (If there's one)

LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\RemoveGrain-1.0\RemoveGrainSSE3.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\mt_masktools2.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Decomb.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("K:\Temp\Temp Megui\File 1\lupin.d2v", info=3)
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
# Deinterlacer
AssumeTFF().Telecide(guide=1).Decimate(mode=1,threshold=2.0)
# Cropper
crop( 10, 2, -12, -6)

# Resizer
Spline16Resize(704,528) # Spline16 (Neutral)
# Denoiser and Anti-aliasing
Import("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\Anti-aliasing\AARG.avs")
AntiAliasRG()


As you can read, I'm using MeGui but the problem is relative to avs script only

So, because you can run your script, one of us has made a mistake: you have forgotten something in you explanation or I'm a newbby :D

[Didée]

"m4 function" supposedly is this one

function m4(float x) {return(x<16?16:int(round(x/4.0))*4)}

[luke83]

"m4 function" supposedly is this one

function m4(float x) {return(x<16?16:int(round(x/4.0))*4)}

Why it isn't in the script? :confused:

However, I added before AntiAliasingRG function and now it works :)

Can I use Spli16Resize instead of Bilinear? This last one makes soft sharp. Spline16 is Neutral. I'm compressing in mkv my Lupin DVDs; the image is not defined as modern anime and the sharpening would be not good

EDIT:
I tried to use Spline16Resize instead of Bilinear but nothing is changed.

Your script is really fast compared with AAA (very slow) but this one (AAA) doesn't smooth as your function. The images are too smoothed.

Script with DeGrainMedian and AAA

http://img703.imageshack.us/img703/5231/immagineaaa.th.png (http://img703.imageshack.us/i/immagineaaa.png/)
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\Anti-aliasing\UnFilter.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\Anti-aliasing\SangNom.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\degrainmedian082\degrainmedian.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Decomb.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("K:\Temp\Temp Megui\File 1\lupin.d2v", info=3)
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
# Deinterlacer
AssumeTFF().Telecide(guide=1).Decimate(mode=1,threshold=2.0)
# Cropper
crop( 10, 2, -12, -6)

# Resizer
Spline16Resize(704,528) # Spline16 (Neutral)
# Denoiser
DeGrainMedian()
# Anti-aliasing
Import("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\Anti-aliasing\AAA.avs")
AAA()



Script with AntiAliasingRG

http://img853.imageshack.us/img853/2306/immagineaaarg.th.png (http://img853.imageshack.us/i/immagineaaarg.png/)
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\RemoveGrain-1.0\RemoveGrainSSE3.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\mt_masktools2.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Decomb.dll")
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("K:\Temp\Temp Megui\File 1\lupin.d2v", info=3)
LoadPlugin("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
# Deinterlacer
AssumeTFF().Telecide(guide=1).Decimate(mode=1,threshold=2.0)
# Cropper
crop( 10, 2, -12, -6)

# Resizer
Spline16Resize(704,528) # Spline16 (Neutral)
# Denoiser
# Anti-aliasing
Import("K:\Documents and Settings\Luca\Documenti\Software\Video Editing\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\Anime\Anti-aliasing\AARG.avs")
AntiAliasRG()


You can see the effect overall on the faces of the far people

[Bloax]

Why isn't it (m4) in the script? :confused:
Because I didn't realize it wasn't internal. ;P

Can I use Spli16Resize instead of Bilinear? This last one makes soft sharp. Spline16 is Neutral.
Spline16 should actually be a bit sharp, but I went for Bilinear to get a little extra performance. Also because haloed antialiasing is automatically bad.

Your script is really fast compared with AAA (very slow)
Glad it's good in terms of performance, I kinda went for that. ;)

The images are too smoothed.
That's what happens when you don't have anything but old games to tune for! :rolleyes:

But anyway, the only thing I can think of is replacing this line:

e=a.RemoveGrain(mode=0,modeu=-1).Mt_Edge(thy1=0,thy2=thrl,thc1=0,thc2=thrc,mode=mode).Mt_Inflate().RemoveGrain(mode=9,modeU=-1)
With this:
e=a.RemoveGrain(mode=0,modeu=-1).Mt_Edge(thy1=0,thy2=thrl,thc1=0,thc2=thrc,mode=mode).Mt_Inflate().RemoveGrain(mode=9,modeU=-1).Mt_Deflate()
Which should keep the Inflated smoothness of the mask, meanwhile being slightly compacted.

Because as far as I can see, it's the mask being way too wide.
And thus adding quite a bit of the blurry mess to the image.

If that doesn't work, try experimenting with the thrl option, and perhaps pumping sx/sy up to 1.8/2. (Both of which ought to at least slightly reduce said issue.)

And I said "ThrL" because "ThrC" should supposedly do absolutely nothing as of now, since I made the mask operate on a (supposedly) discolored image.

Oh, and by the way - I added the m4 function to the script now.
Thankies to you for it. :P

[luke83]

I GOT IT!!!!
Your mod didn't work so I think to use Unfilter. For my anime (Lupin 2nd Season) after AntialiasRG I put Unfilter(70,70) and the result is the same :D

EDIT:
Unfilter(60,60) otherwise I have light aliasing on the lips

[Bloax]

Hm, combining "mode="roberts"" with Mt_Deflate() at the end of the mask seems to provide sharper results. (Added that to the main script.)

But yeah, it does need some sharpening. If I can find something appropriately fast, I may add it in as an option.

As for now, that should work.

(I also added a little list of options for mode="blah", just because I felt like it.)

Edit: Also added sharpening now. Do test it. (sharpen=true/false,sharpstr=stuff)
Performance suffers a bit (140-155 FPS Average/Peak, now 130-140 Average/Peak on the same source), but that shouldn't be too much.

And I say "do test it" because I don't have too much to test it on. :-)

EditEdit: Oh boy, format breaking time!

But in other news, added YET ANOTHER sharpening feature, a sharpmode for a sharper kernel for the mask.
That, in addition to sharpen=true makes this look a bit like MSAA. (Oops!)

P.S: Hm, with both of those, sx/sy=0.5 don't really seem to degrade too much. Even 0.8 provide quite a bit of performance for no visible to me quality degradation.
Could someone investigate this one? (Fine wording for test? ;))


Next day: Confirmed some stuff, which is, updated the random comments inside it that describe it.

[bcn_246]

...Lastly, I have found the original version of SangNom is about 4x faster on my quadcore than the more current versions. Unfortunately it does have a very slightly lower quality but it is hardly perceivable. http://mf.creations.nl/avs/filters/SangNom_old2.dll...

Just wondered if anybody could re-upload this version. I can only find the beta (newest) version of the DLL (date=18/01/2004, crc32=4e79e77e).

TIA

[bcn_246]

...Lastly, I have found the original version of SangNom is about 4x faster on my quadcore than the more current versions. Unfortunately it does have a very slightly lower quality but it is hardly perceivable. http://mf.creations.nl/avs/filters/SangNom_old2.dll...
Just wondered if anybody could re-upload this version. I can only find the beta (newest) version of the DLL (date=18/01/2004, crc32=4e79e77e).

TIA
Probably shouldn't but *bump*... really am interested in getting hold of the above file...

Sorry guys :)

[Akai-Shuichi]

I'm curious if anybody could help me out with how to go about anti-aliasing in this anime?

Here is a sample clip: http://www.sendspace.com/file/lozi3h

It is almost as if it slightly wobbles with the aliasing there.

[Didée]

I've found three more versions of sangnom.dll on my HDD, which all do differ by CRC. <click> (http://www.mediafire.com/download.php?4l78sluv4t1e5jy)

Though, I've no clue if any of those is faster or slower or something. You try.

[bcn_246]

Thanks Didée :)

I'll try them and report back...

[bcn_246]

Okay...

On i7@4.2GHz with source on a ~450MB/s SSD, just a simple:AVISource("Uncompressed 1920x1080p60 YV12 Input.avi")
SangNom()SangNom.dll (CRC32: 4E79E77E) (http://www.mediafire.com/?uev3oftt76gpm49)
~45.5 FPS

sangnom_old.dll (CRC32: 60BE3D92)
~2.9 FPS

sangnom_old2.dll (CRC32: 232EF465)
~23.9 FPS

sangnom2.dll (CRC32: 725B1C38)
Couldn't get it to load (Access Violation)

[WooWoo]

Hi, I need some help. I am trying to encode to x264 using aaa() loading the latest versions of SangNom and UnFilter and importing AAA.avs (I got all the files today from the avisynth wiki). Everything looks right in the preview window for programs such as VirtualDub and MeGUI. However, when my x264 encode finishes, the height is double the size (width is the same), and the fps is halved. So instead of it being 640x480 @ 23.976 FPS, it is 640x960 @ 11.988 FPS. When I encode to XviD, everything comes out at the correct resolution and FPS using the same script. All I'm doing is loading the source file and plugins/scripts needed for AAA(), and then running AAA() in my avs script.

I then tried out each version of SangNom that Didée posted and bcn_246 tested out. They all resulted in the the same problem except for sangnom_old.dll (CRC32: 60BE3D92). This particular version worked perfectly without changing the resolution or FPS, however it reduced my FPS to a laughable ~2 FPS on my i5 quad core machine. Anybody else having this issue, or am I not loading the correct versions of everything, or what's going on exactly? Why is it changing my resolution and fps?

[luke83]

Hi, I need some help. I am trying to encode to x264 using aaa() loading the latest versions of SangNom and UnFilter and importing AAA.avs (I got all the files today from the avisynth wiki). Everything looks right in the preview window for programs such as VirtualDub and MeGUI. However, when my x264 encode finishes, the height is double the size (width is the same), and the fps is halved. So instead of it being 640x480 @ 23.976 FPS, it is 640x960 @ 11.988 FPS. When I encode to XviD, everything comes out at the correct resolution and FPS using the same script. All I'm doing is loading the source file and plugins/scripts needed for AAA(), and then running AAA() in my avs script.

I then tried out each version of SangNom that Didée posted and bcn_246 tested out. They all resulted in the the same problem except for sangnom_old.dll (CRC32: 60BE3D92). This particular version worked perfectly without changing the resolution or FPS, however it reduced my FPS to a laughable ~2 FPS on my i5 quad core machine. Anybody else having this issue, or am I not loading the correct versions of everything, or what's going on exactly? Why is it changing my resolution and fps?
post your script please. I'm using AAA from a lot of time and i haven't any problem yet. The only problem is that AAA is quite slow. However is the only anti-aliasing script that works good for me.

[byme]

how do to put the nnedi3 instead of BilinearResize in the SAA?
is it possible?
I'm not able...

Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().BilinearResize(OX,OY)

Now, for the use that I do, I'm putting spline36resize

Clp.PointResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().spline36resize(OX,OY)

[Keiyakusha]

Not possible. nnedi3 doesn't do downscaling.

[jofori89]

can't see any artifact there
BTW...
#FASTER ANTI ALIASING FILTER
#REQUIERED:
# SangNom
# mt_masktools

function FAA(clip input,int "str") {
str = Default(str, 50)
vid = input.IsYV12() ? input : input.ConvertToYV12()
w=width(input)
h=height(input)

aa=vid.pointResize(w*2,h) . TurnLeft() . SangNom(AA=str) . TurnRight(). lanczosResize(w,h)

return mt_merge(vid , aa , vid.mt_edge().mt_inflate() )
}
Maybe it's not the best anti aliasing filter, but 3x faster, and the first i can use in real time with some sources.

:goodpost: this one is good enough.

[spx]

Hi, i'm using the aaa() from wiki (http://avisynth.org/mediawiki/upload/6/68/AAA.avs), and i got a question: What adds suchs a heavy noise to picture?
I've got a pretty clean animation frame, then i add aaa(Xshrp=55,Yshrp=55,chroma=false), and i get a tremendous amout of temporal noise. it changes with every frame. I'm afraid to clean it up because of detail loss.
You guys have the same results?

here's an example: http://screenshotcomparison.com/comparison/33785

at first i thought it's because of unsharp values of 55, but it adds noise on default, too...

[Seedmanc]

In regards to ediaa(), wouldn't replacing
return a.EEDI2(field=1).TurnRight().EEDI2(field=1).TurnLeft().spline36resize(a.width,a.height,-0.5,-0.5,2*a.width+.001,2*a.height+.001)
with
return a.EEDI2(field=1).spline36resize(a.width,a.height,0,-0.5,0,2*a.height+.001).TurnRight().EEDI2(field=1).TurnLeft().spline36resize(a.width,a.height,-0.5,0,2*a.width+.001,0)

be better speed- and memory-wise? So that second EEDI2 doesn't have to deal with unnecessarily 2x wider clip when it only works with the height, not width.
For me it gives a speed increase from 1 fps to 1.6 on 1080p.

[wOxxOm]

AA scripts that use turnleft and/or sangnom may be significantly sped-up with sangnom2+fturn (http://forum.doom9.org/showthread.php?p=1638416) - for better results drop the chroma plane by using Y8 colorspace during processing and copy original chroma in the end (example (https://github.com/line0/avs-scripts/blob/master/maa2.avsi)). Realtime AA of 1080p @ 24fps on my pc...

[may24]

Hi all,
sorry for capturing this thread ;) , but I've a similar issue.
I got a poorly deinterlaced clip here (simple bob().SelectEven() ) with quite a lot of jaggy edges I'd like to fight.
This is not Anime material but captured from a VHS tape.
Your anti-alias-script uses SangNom, but that's for interlaced material - not ?
But my vid is already progressive encoded and now I'd like to reencode plus a bit upsizing.
So what would be the best "way to do it ?"

The original is lost only reencoded vid is here.
TIA

[real.finder]

my first function, MC DAA3 :D



#Motion-Compensated Anti-aliasing with contra-sharpening by A.SONY
#some ideas was taken from Didée scripts daa and this http://forum.doom9.org/showthread.php?t=153219
#created because when applied daa3 to fixed scenes, it could damage some details and other issues.
# v3.55

function mcdaa3(clip input) {
sup = input.HQdn3d().FFT3DFilter().MSuper(pel=2,sharp=1)
fv1 = sup.MAnalyse(isb=false,delta=1,DCT=2,Truemotion=false)
fv2 = sup.MAnalyse(isb=true,delta=1,DCT=2,Truemotion=true)
csaa = input.daa3mod()
momask1 = input.MMask(fv1, kind = 1, ml=2)
momask2 = input.MMask(fv2, kind = 1, ml=3)
momask = mt_average(momask1,momask2,u=3,v=3)
mt_merge(input,csaa,momask,u=3,v=3)
}


#############################

#Anti-aliasing with contra-sharpening by Didée with some modifications, v3.55

FUNCTION daa3mod(clip c1, val "slow", int "threads") {
avs25 = VersionNumber() < 2.60
isslod = defined(slow)

slow = default (slow , true)
isslob = isbool(slow)
issloi = Isint(slow)
Assert(isslob || issloi, "'slow' only accepts bool or int")
slowb = isslob ? slow : isslod
threads = default (threads , 0)

rescale = slowb && issloi ? slow > -1 : slowb
upsizer = rescale ? issloi ? slow!=0 ? "BilinearResize" : "Spline36Resize" : "Spline36Resize" : nop()
downsizer = slowb ? issloi ? slow!=0 ? "BicubicResize" : "Spline36Resize" : "Spline36Resize" : nop()

c = rescale ? threads!=1 ? Eval("""try { Eval("c1."+upsizer+"mt(c1.width, c1.height*3/2,threads=threads)") } catch(error_msg) { Eval("c1."+upsizer+"(c1.width, c1.height*3/2)") }""") : Eval("c1."+upsizer+"(c1.width, c1.height*3/2)") : c1

nn = issloi ? slow > 1 ? c.Padding(slow==2 ? 0 : 4,4,slow==2 ? 0 : 4,4).nnedi3(field=-2,threads=threads).crop(slow==2 ? 0 : 4,4,slow==2 ? 0 : -4,-4) : c.nnedi3(field=-2,threads=threads) : c.nnedi3(field=-2,threads=threads)
c =isyuy2(c) && avs25 ? c.Interleaved2Planar() : isyuy2(c) ? c.converttoyv16() : c
nn=isyuy2(nn) && avs25 ? nn.Interleaved2Planar() : isyuy2(nn) ? nn.converttoyv16() : nn
isslow4 = slowb && issloi ? slow > 3 || slow < -1 : false
istof = isslow4 ? c1.getparity : nop()
seto = isslow4 ? istof ? selecteven(nn) : selectodd(nn) : selecteven(nn)
seto = isslow4 ? threads!=1 ? Eval("""try { Eval("seto."+downsizer+"mt(c.width, c.height,src_top=-0.5,threads=threads)") } catch(error_msg) { Eval("seto."+downsizer+"(c.width, c.height,src_top=-0.5)") }""") : Eval("seto."+downsizer+"(c.width, c.height,src_top=-0.5)") : seto
sebo = isslow4 ? istof ? selectodd(nn) : selecteven(nn) : selectodd(nn)
sebo = isslow4 ? threads!=1 ? Eval("""try { Eval("sebo."+downsizer+"mt(c.width, c.height,src_top=0.5,threads=threads)") } catch(error_msg) { Eval("sebo."+downsizer+"(c.width, c.height,src_top=0.5)") }""") : Eval("sebo."+downsizer+"(c.width, c.height,src_top=0.5)") : sebo

dbl = mt_average(seto,sebo,U=3,V=3)
dblD = mt_makediff(c,dbl,U=3,V=3)
shrpD = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 11,Planar=true),U=3,V=3)
DD = shrpD.repair(dblD,13,Planar=true)
fd1 = dbl.mt_adddiff(DD,U=3,V=3)

fd2 = rescale ? threads!=1 ? Eval("""try { Eval("fd1."+downsizer+"mt(c1.width, c1.height,threads=threads"+((downsizer!="Spline36Resize") ? ",b=0.3, c=0.2" : "" )+")") } catch(error_msg) { Eval("fd1."+downsizer+"(c1.width, c1.height"+((downsizer!="Spline36Resize") ? ",b=0.3, c=0.2" : "" )+")") }""") : Eval("fd1."+downsizer+"(c1.width, c1.height"+((downsizer!="Spline36Resize") ? ",b=0.3, c=0.2" : "" )+")") : fd1
fd3 = isyuy2(c1) && avs25 ? fd2.Planar2Interleaved() : isyuy2(c1) ? fd2.converttoyuy2() : fd2
return fd3
}


I created this script because daa3 when applied to fixed scenes, it could damage some details such as horizontal lines on the screen

like this example:- original (http://i.imgur.com/pHfDcpJ.png) - daa3 (http://i.imgur.com/cmoZbvd.png) - mcdaa3 (http://i.imgur.com/up9geVZ.png)

And here apply of daa3 and mcdaa3 on a moving scene:- original (http://i.imgur.com/f9tW8bl.png) -daa3 (http://i.imgur.com/YlavKVb.png) -mcdaa3 (http://i.imgur.com/DmPcDw8.png)

download link (http://www.mediafire.com/download/wqkob7zx1p119e0/daa3mod.avsi) (it's old now)

edit1: mcdaa3 and normal daa can deal with ifade too, In fact, the daa function is the remnants Interlaced cleaner after IVTC

edit2: now you can use it like this smam(filter=daa3mod()) (http://forum.doom9.org/showpost.php?p=1750019&postcount=77) , this support YUY2 and more up to date

edit3: since smam now will add daa3mod() is some cases, if you use it like this smam(filter="daa3mod()") it will be 2 daa3mod(), so you must change smam settings like this smam(filter="daa3mod()",usedaa3mod=0)
or use it as I write in "edit2" above, or smam(filter=last)

edit4: if the clip has black border on top or bottom it beter to crop it before using any daa/daa3 method since it will add some Artifacts unless you set slow to -2 or more than 0 (better but slower as you go on with the number)

[mandarinka]

Isn't it just motion adaptive, instead of motion-compensated? /Or am I reading it wrong, I'm no good at any sort of programming/.

[Pier]

my first function, MC DAA3 :D



#Motion-Compensated Anti-aliasing with contra-sharpening by A.SONY
#some ideas was taken from Didée scripts daa and this http://forum.doom9.org/showthread.php?t=153219
#created because when applied daa3 to fixed scenes, it could damage some details and other issues.

function mcdaa3(clip input) {
sup = input.HQdn3d.FFT3DFilter.MSuper(pel=2,sharp=1)
fv1 = sup.MAnalyse(isb=false,delta=1,DCT=2,Truemotion=false)
fv2 = sup.MAnalyse(isb=true,delta=1,DCT=2,Truemotion=true)
csaa = input.daa3mod
momask1 = input.MMask(fv1, kind = 1, ml=2)
momask2 = input.MMask(fv2, kind = 1, ml=3)
momask = mt_average(momask1,momask2)
mt_merge(input,csaa,momask,u=3,v=3)
}


#############################

#Anti-aliasing with contra-sharpening by Didée with some modifications

FUNCTION daa3mod(clip c1) {
c = c1.Spline36Resize(c1.width, c1.height*3/2)
nn = c.nnedi3(field=-2)
dbl = mt_average(selecteven(nn),selectodd(nn),U=3,V=3)
dblD = mt_makediff(c,dbl,U=3,V=3)
shrpD = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 11),U=3,V=3)
DD = shrpD.repair(dblD,13)
return dbl.mt_adddiff(DD,U=3,V=3).Spline36Resize(c1.width, c1.height)
}


I created this script because daa3 when applied to fixed scenes, it could damage some details such as horizontal lines on the screen

like this example:- original (http://i.imgur.com/pHfDcpJ.png) - daa3 (http://i.imgur.com/cmoZbvd.png) - mcdaa3 (http://i.imgur.com/up9geVZ.png)

And here apply of daa3 and mcdaa3 on a moving scene:- original (http://i.imgur.com/f9tW8bl.png) -daa3 (http://i.imgur.com/YlavKVb.png) -mcdaa3 (http://i.imgur.com/DmPcDw8.png)

download link (http://www.mediafire.com/download/wqkob7zx1p119e0/daa3mod.avsi)

Not sure, but if i remember correctly "HQdn3d()" is faster than "HQdn3d", same for every other filter.

[Kein]

my first function, MC DAA3 :D



#Motion-Compensated Anti-aliasing with contra-sharpening by A.SONY
#some ideas was taken from Didée scripts daa and this http://forum.doom9.org/showthread.php?t=153219
#created because when applied daa3 to fixed scenes, it could damage some details and other issues.

function mcdaa3(clip input) {
sup = input.HQdn3d.FFT3DFilter.MSuper(pel=2,sharp=1)
fv1 = sup.MAnalyse(isb=false,delta=1,DCT=2,Truemotion=false)
fv2 = sup.MAnalyse(isb=true,delta=1,DCT=2,Truemotion=true)
csaa = input.daa3mod
momask1 = input.MMask(fv1, kind = 1, ml=2)
momask2 = input.MMask(fv2, kind = 1, ml=3)
momask = mt_average(momask1,momask2,u=3,v=3)
mt_merge(input,csaa,momask,u=3,v=3)
}


#############################

#Anti-aliasing with contra-sharpening by Didée with some modifications

FUNCTION daa3mod(clip c1) {
c = c1.Spline36Resize(c1.width, c1.height*3/2)
nn = c.nnedi3(field=-2)
dbl = mt_average(selecteven(nn),selectodd(nn),U=3,V=3)
dblD = mt_makediff(c,dbl,U=3,V=3)
shrpD = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 11),U=3,V=3)
DD = shrpD.repair(dblD,13)
return dbl.mt_adddiff(DD,U=3,V=3).Spline36Resize(c1.width, c1.height)
}


I created this script because daa3 when applied to fixed scenes, it could damage some details such as horizontal lines on the screen

like this example:- original (http://i.imgur.com/pHfDcpJ.png) - daa3 (http://i.imgur.com/cmoZbvd.png) - mcdaa3 (http://i.imgur.com/up9geVZ.png)

And here apply of daa3 and mcdaa3 on a moving scene:- original (http://i.imgur.com/f9tW8bl.png) -daa3 (http://i.imgur.com/YlavKVb.png) -mcdaa3 (http://i.imgur.com/DmPcDw8.png)

download link (http://www.mediafire.com/download/wqkob7zx1p119e0/daa3mod.avsi)

edit: mcdaa3 and normal daa can deal with ifade too, In fact, the daa function is the remnants Interlaced cleaner after IVTC

Which plugins it requires?
Or, rather, do I need to just invoke the functions with empty parameters or I need to specify clip?

[StainlessS]

Which plugins it requires?
Or, rather, do I need to just invoke the functions with empty parameters or I need to specify clip?

HQDn3D, FFT3DFilter, MvTools v2, Masktools v2, nnedi3. Repair from RemoveGrain.


function mcdaa3(clip input)


Above tells you everything you need to know (call with named clip or it will use implicit Last clip).

http://avisynth.nl/index.php/External_filters

[Kein]

HQDn3D, FFT3DFilter, MvTools v2, Masktools v2, nnedi3. Repair from RemoveGrain.


function mcdaa3(clip input)


Above tells you everything you need to know (call with named clip or it will use implicit Last clip).

http://avisynth.nl/index.php/External_filters

Thanks!

[tormento]

my first function, MC DAA3 :D
Does it support AVS+ color formats?

If not, could you give it a version bump?

Thank you ;)

[real.finder]

Does it support AVS+ color formats?

If not, could you give it a version bump?

Thank you ;)

it should https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/daa3MOD.avsi but as mandarinka https://forum.doom9.org/showpost.php?p=1639733&postcount=90 said it's motion adaptive (I was kinda noob back then)

but anyway, now it better to use smam(filter=daa3mod(slow=2)) or with Vinverse instead of daa3mod/daa3

[tormento]

but anyway, now it better to use smam(filter=daa3mod(slow=2)) or with Vinverse instead of daa3mod/daa3
As far as I can see they are both for combing restoration not for antialiasing. Am I wrong?

Where can I find smam?

I am just looking for a good AA for anime contents, now that they use 3D a lot and it's a mess as far as aliasing is concerned.

[real.finder]

As far as I can see they are both for combing restoration not for antialiasing. Am I wrong?


yes they are for combing, sometimes in old anime the fields not matched ideally for some reason like they use an analog master or 60i Effects

like this https://i.imgur.com/f9tW8bl.png it's aliasing because of fields not matched perfectly


Where can I find smam?

as always http://forum.doom9.org/showthread.php?t=174121 or https://github.com/realfinder/AVS-Stuff

I am just looking for a good AA for anime contents, now that they use 3D a lot and it's a mess as far as aliasing is concerned

then you must look for something else like maa2 or santiag

[tormento]

then you must look for something else like maa2 or santiag
Thanks for your explanation.

At the end, there is no "modernized" AA filter?

[orchid]

Thanks for your explanation.

At the end, there is no "modernized" AA filter?

There is, but as the anime community has entirely moved to vapoursynth sometime in 2017, such modernized filters remain exclusive to vapoursynth.