Fine anime antialiasing

[Mystery Keeper]

I am trying to encode .hack//SIGN and I've got an encode by KAA to compare. Those guys have lost awfully lot of details, but, on the bright side, they've got characters edge strokes smooth and thin. I tried many AA methods, and none could make the same result.

Here's KAA's encode
http://ipicture.ru/uploads/080804/8073/thumbs/4dqX1JoR08.png (http://ipicture.ru/Gallery/Viewfull/4606990.html)

Here's my frame without AA.
http://ipicture.ru/uploads/080804/8073/thumbs/z7Sxw7kfbk.png (http://ipicture.ru/Gallery/Viewfull/4606993.html)

Here's my frame with eedi2().TurnRight().eedi2().TurnLeft()
http://ipicture.ru/uploads/080804/8073/thumbs/SA13fN1TUg.png (http://ipicture.ru/Gallery/Viewfull/4606996.html)

And here's my frame with triple tweaked "antialiasing()" script
http://ipicture.ru/uploads/080804/8073/thumbs/W6zW8EiFVQ.png (http://ipicture.ru/Gallery/Viewfull/4606999.html)

As you can see, antialiasing() gives rather good result. BUT! It came to be rather unstable and gives heavy flickering in some motion scenes.

Does anyone know how I can make the edge strokes as thin and smooth? I tried asking around KAA, and they said their encoder is WoW addict, which they can't contact for months T_T

[Sagekilla]

Try using Toon() I remember that being very useful.

[Mystery Keeper]

Try using Toon() I remember that being very useful.

Very nice for darkening strokes. But 'tis not what I need.

[Comatose]

I think what you need is some kind of warpsharp, this is too big for AA filters to handle :E

[egrimisu]

i like kaa encode :) maybe warpsharp will help a bit like Comatose said

[Mystery Keeper]

I think what you need is some kind of warpsharp, this is too big for AA filters to handle :E

Hey, it's getting closer! But too strong. Carves out some objects too.

Upd: Thanks! I guess I can come up with something nice if I play with that.
Anyone got other suggestions?

[Mystery Keeper]

Ok, this draft gives nice result:

a=last
w=width(a)
h=height(a)
b=a.Warpsharp(depth=100).TurnLeft().EEDI2().TurnRight().EEDI2()
m=logic( b.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,b.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false).spline64resize(w,h)
b=b.spline64resize(w,h).Toon(0.5)
MaskedMerge(a,b,m,Y=3,U=3,V=3)

But it still gives a little flicker. Maybe it can be motion compensated?

[Sagekilla]

Sure, if you do this:

bvec = a.MVAnalyse(isb=true,delta=1,idx=1,ov=4,pel=2)
fvec = a.MVAnalyse(isb=false,delta=1,idx=1,ov=4,pel=2)
bcomp = a.MVFlow(bcomp, idx=1, thSCD1=500)
fcomp= a.MVFlow(fcomp, idx=1, thSCD1=500)
b = Interleave(bcomp,a,fcomp)

w=width(b)
h=height(b)
c=b.Warpsharp(depth=100).TurnLeft().EEDI2().TurnRight().EEDI2()
m=logic( c.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,c.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false).spline64resize(w,h)
c=c.spline64resize(w,h).Toon(0.5)
MaskedMerge(b,c,m,Y=3,U=3,V=3)
TemporalRepair(b,last)

That should work, I believe.

[Mystery Keeper]

Gives errors. If I correct them, it crashes.

[Didée]

@ Sagekilla: Not quite what I would've expected, but try & see how that turns out. Downside is that the EEDI2 interpolation now is done on 3 times more frames than before ...
Also, me thinks there's a SelectEvery(3,1) missing somewhere, to undo the interleave? ;)


My go (not tested though): MC'ed Soothe.

a=last.assumetff()
w=width(a)
h=height(a)
b=a.Warpsharp(depth=100).TurnLeft().EEDI2().TurnRight().EEDI2()
m=logic( b.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,b.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false).spline64resize(w,h)
b=b.spline64resize(w,h,0.5,-0.5,2*w+.001,2*h+.001).Toon(0.5)
MaskedMerge(a,b,m,Y=3,U=3,V=3)

sD=mt_makediff(a,last)

fv2 = a.MVAnalyse(isb=false,delta=2,idx=13)
fv1 = a.MVAnalyse(isb=false,delta=1,idx=13)
bv1 = a.MVAnalyse(isb=true, delta=1,idx=13)
bv2 = a.MVAnalyse(isb=true, delta=2,idx=13)

sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14)

reduc = 0.4
sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?") .mergeluma(sDD,1.0-reduc)

a.mt_makediff(sDD,U=2,V=2)

@ MysteryKeeper: you've had a 1/2-pel registration error between the original (a) and antialiased (b) clip, due to EEDI2. Corrected by the blue parts.

[Mystery Keeper]

Thank thee, Didée. I'll try to encode with that script and see if 'tis effective enough. Weird. I found an interlaced frame in the middle of anime o_0

[Sagekilla]

Ah yes, I forgot about SelectEvery(3,1). Also, why is it necessary for the AssumeTFF()? I was just assuming this source was progressive, and wouldn't need any interlace badgering.

Edit: Interlacing inside of anime isn't as uncommon as you might think. Unfortunately very annoying stuff happens in anime all the time :(

[Didée]

Progressive is what I expected, too. The AssumeTFF was just to make sure in which direction the frame gets shifted by EEDI2. It's a bad shortcut, yes, I'm too lazy and tired to look up the necessary parameters to make it the "proper" way. :p

[Mystery Keeper]

@Sagekilla, interlacing is usually introduced in the places of montage. Well, that single frame was at the scene change.

Well, thanks guys. I see that my script is highly imperfect, but as I said, 'twas draft. Now, thanks to you all, I've got the wrapper I need and can play with it and make an adjustable function... tomorrow. Good night ^_^

[Mystery Keeper]

Now this looks very good ^_^ Though prone to some details carving.

function SharpAAMC(clip orig, int "aastr", float "ds", int "SPre", int "SPost", int "BPost")
{
aastr=default(aastr,255) #antialiasing strength
ds=default(ds,0.2) #strokes darkening strength
SPre=default(SPre,100) #Presharpening
SPost=default(SPost,50) #Postsharpening
BPost=default(Bpost,10) #Postbluring

a=orig
w=width(a)
h=height(a)
m=logic( a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false)
b=a.Toon(ds).Warpsharp(depth=SPre)
b=b.spline64resize(w*2,h*2).TurnLeft().SangNom(aa=aastr).TurnRight().SangNom(aa=aastr)
b=b.Warpsharp(depth=SPost,blur=BPost).spline64resize(w,h)

MaskedMerge(a,b,m,Y=3,U=3,V=3)
sD=mt_makediff(a,last)
fv2 = a.MVAnalyse(isb=false,delta=2,idx=13)
fv1 = a.MVAnalyse(isb=false,delta=1,idx=13)
bv1 = a.MVAnalyse(isb=true, delta=1,idx=13)
bv2 = a.MVAnalyse(isb=true, delta=2,idx=13)

sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14)

reduc = 0.4
sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc)

a.mt_makediff(sDD,U=2,V=2)

return last
}

Feel free to tune it and make more versatile!

[Mystery Keeper]

It probably needs some other postsharpening method. But it must still leave lines smooth. Any suggestions?

[Sagekilla]

LSF tends to work great, I've used LimitedSharpenFaster(edgemode=1,strength=160,overshoot=1,soft=15) and it tends to avoid haloing while adding some nice sharpness.

[Mystery Keeper]

Ok. It's getting better and better. But now it seems to be sharpness VS strokes smoothness.

function SharpAAMC(clip orig, int "aastr", float "ds", int "ShPre", int "ShPost", int "SmPost")
{
aastr=default(aastr,255) #antialiasing strength
ds=default(ds,0.2) #strokes darkening strength
ShPre=default(ShPre,80) #Presharpening
ShPost=default(ShPost,300) #Postsharpening
SmPost=default(SmPost,100) #Postsmoothing

a=orig
w=width(a)
h=height(a)
m=logic( a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false)
b=a.Toon(ds).Warpsharp(depth=ShPre)
b=b.spline64resize(w*2,h*2).TurnLeft().SangNom(aa=aastr).TurnRight().SangNom(aa=aastr)
b=b.spline64resize(w,h).LimitedSharpenFaster(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost)


MaskedMerge(a,b,m,Y=3,U=3,V=3)
sD=mt_makediff(a,last)
fv2 = a.MVAnalyse(isb=false,delta=2,idx=13)
fv1 = a.MVAnalyse(isb=false,delta=1,idx=13)
bv1 = a.MVAnalyse(isb=true, delta=1,idx=13)
bv2 = a.MVAnalyse(isb=true, delta=2,idx=13)

sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14)

reduc = 0.4
sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc)

a.mt_makediff(sDD,U=2,V=2)

return last
}

[Sagekilla]

Well 300 is a bit -excessive- for sharpening. You can always do a second toon or use AAA() to smooth out the lines a bit after LSF.

[Mystery Keeper]

Nah. It's tuned to sharpen the eyes area. Greater SmPost will smooth the lines nice and clean, but blur that area again. What's bothering me a bit is that some spots in eyes get lessened. Well, fine with me though. All the parameters can be tuned to make lesser damage in exchange of being less effective.

[DarkT]

I dunno... I'm getting nice results with:

Function SAA(Clip Clp,Int"SS",Bool"CP")
{

OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)


Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()

Clp.BilinearResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().Spline36Resize(OX,OY)

CP ? Last : MergeChroma(Clp)

Return(Last)

}

ImageSource("untitled.bmp", start=1, end=587,fps=23.976023976023976023976023976024).converttoyv12()

SAA()
Toon(strength=0.3)
WarpSharp(96, 10, 128, -0.6)
LimitedSharpenFaster(smode=4, strength=77)

Anywya, my 2 cents...

[Adub]

Yeah, with my passed experiences with Sagnom, I lost way to many details, and with Spline36resize on top of that, it starts to look to blurry for my personal taste.

[Mystery Keeper]

Yeah, with my passed experiences with Sagnom, I lost way to many details, and with Spline36resize on top of that, it starts to look to blurry for my personal taste.

That's why 'tis masked. Well, warpsharp in this function still kills some details (carves edges), but it can be controlled by ShPre parameter.

Folk, critique is good, but don't just criticize, suggest improvements.

[Adub]

My post was referring to the post by DarkT directly above mine. I haven't even looked at your script. I will now though.

Edit: Wow!! This works wonders on a source I was looking at! It had some nasty jaggies from a bad deinterlace, but this pretty much eliminated them. It's a bit slow though, but for the quality, I'll take it.

Edit2: Okay, it is REALLY SLOW. I am getting around 0.39fps on a lossless encode of a 720p source. Anyway to speed this up? Maybe use the Masktools 2 version of DEdgemask? (If it even exists, I don't know)

[Mystery Keeper]

If you comment the lines between "MaskedMerge()" and "return last", it will disable the motion compensation and probably add you some speed.

Sorry, I'm not well versed in AviSynth ^_^'

[Adub]

Yes, but I like the motion comp. If there is nothing that can be done, I will except my fate. But I'll keep my fingers crossed!!

[Seraphic-]

Sorry, I'm not well versed in AviSynth ^_^'

You seem to know enough to be able to write an impressive script.
Did you find a guide or put it together from bits and pieces from searching around?

Also, could you post your full script code? I was trying to run your script from post #18, but I'm not able to get it to load.
I am most likely missing a few need things.

[martino]

Maybe use the Masktools 2 version of DEdgemask? (If it even exists, I don't know)
Nope, it doesn't. But with some fiddling you could get something similar with mt_edge's prewitt mode.

[TheRyuu]

Can we get a demuxed m2v or something I can play around with? (preferable with a couple scenes, 30 seconds or something, I dunno)

[Adub]

Seraphic, what was your problem? Meaning, what error were you getting?

[Seraphic-]

Seraphic, what was your problem? Meaning, what error were you getting?

"Return value was not a video clip"

[Adub]

Try copying and pasting the script again. Make sure you scroll down all the way to the bottom as well. It sounds like you missed a few lines.

[Comatose]

Mystery Keeper: Why not replace MaskedMerge(a,b,m,Y=3,U=3,V=3) with mt_merge(a,b,m,Y=3,U=3,V=3)?
Also, you're getting the eye "sparkle" loss because of SangNom ;_; They're small spots so it eats them :3 Search for starfield (in this forum).

[Mystery Keeper]

@Seraphic, I have only done the AA part, using people recommendations even there, and made an adjustable function. All other parts are written by somebody else.

How can I split a vob file? I'd love to give it to you for experiments. Folk, I know that's a wrong thread for it, but I'm desperate! I can't denoise it. Whatever I try spoils the video. Grid artifacts, ghosts, washed out details. Nothing seems to work for it.

[Mystery Keeper]

Mystery Keeper: Why not replace MaskedMerge(a,b,m,Y=3,U=3,V=3) with mt_merge(a,b,m,Y=3,U=3,V=3)?

Done. Thanks.

[Seraphic-]

Try copying and pasting the script again. Make sure you scroll down all the way to the bottom as well. It sounds like you missed a few lines.

Yes, seems I did miss a few lines.
It loads now, but doesn't seem to be changing the appearance of the video at all.
Was something else left out?

@Seraphic, I have only done the AA part, using people recommendations even there, and made an adjustable function. All other parts are written by somebody else.

The AA technique that I am familiar with is "EEDI2().TurnRight().EEDI2().TurnLeft()" then "spline36resize" down to the correct resolution.
It does a good job and removes a fair amount of edge jaggies, but also adds a fair amount of blur.

[Mystery Keeper]

The AA technique that I am familiar with is "EEDI2().TurnRight().EEDI2().TurnLeft()" then "spline36resize" down to the correct resolution.
It does a good job and removes a fair amount of edge jaggies, but also adds a fair amount of blur.

'Twas in the initial version of the script. Blur is dealt with alright. 'Tis just that using EEDI2 will make it much slower while not smoothing the lines as good as sangnom. Maybe I'm wrong. Need to test.

[martino]

You never called the function itself.

It does a good job and removes a fair amount of edge jaggies, but also adds a fair amount of blur.
By no way as bad as with SangNom, blur-wise.

[45tripp]

How can I split a vob file? I'd love to give it to you for experiments. .

dgindex.
'[' ']' select section, file -> 'save and demux'

[Seraphic-]

You never called the function itself.

How do you call the function?

[Mystery Keeper]

Just write SharpAAMC() somewhere in your script. Better do it after deinterlacing and denoising and before resizing.

[Mystery Keeper]

Ok, Seraphic, here's the function using EEDI2 for you. Twice as slow.
function ESharpAAMC(clip orig, float "ds", int "ShPre", int "ShPost", int "SmPost")
{
ds=default(ds,0.2) #strokes darkening strength
ShPre=default(ShPre,80) #Presharpening
ShPost=default(ShPost,300) #Postsharpening
SmPost=default(SmPost,100) #Postsmoothing

a=orig.assumetff()
w=width(a)
h=height(a)
m=logic( a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false)
b=a.Toon(ds).Warpsharp(depth=ShPre)
b=b.TurnLeft().EEDI2().TurnRight().EEDI2().spline64resize(w,h,0.5,-0.5,2*w+.001,2*h+.001)
b=b.LimitedSharpenFaster(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost)


mt_merge(a,b,m,Y=3,U=3,V=3)
sD=mt_makediff(a,last)
fv2 = a.MVAnalyse(isb=false,delta=2,idx=13)
fv1 = a.MVAnalyse(isb=false,delta=1,idx=13)
bv1 = a.MVAnalyse(isb=true, delta=1,idx=13)
bv2 = a.MVAnalyse(isb=true, delta=2,idx=13)

sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14)

reduc = 0.4
sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc)

a.mt_makediff(sDD,U=2,V=2)

return last
}

And now compare them. Left - sangnom, right - eedi2.

http://ipicture.ru/uploads/080807/8073/thumbs/AihJn6JCxS.png (http://ipicture.ru/Gallery/Viewfull/4864548.html)

[Adub]

Hell, I can't tell the difference. I'll stick with the sangnom one.

Although, I got maa() (from the animeivtc package) and some other filters to achieve essentially the same results, with a much faster render time. Sure, it's not exactly the same, but it works.

[Seraphic-]

Ok, Seraphic, here's the function using EEDI2 for you. Twice as slow.
function ESharpAAMC(clip orig, float "ds", int "ShPre", int "ShPost", int "SmPost")
{
ds=default(ds,0.2) #strokes darkening strength
ShPre=default(ShPre,80) #Presharpening
ShPost=default(ShPost,300) #Postsharpening
SmPost=default(SmPost,100) #Postsmoothing

a=orig.assumetff()
w=width(a)
h=height(a)
m=logic( a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false)
b=a.Toon(ds).Warpsharp(depth=ShPre)
b=b.TurnLeft().EEDI2().TurnRight().EEDI2().spline64resize(w,h,0.5,-0.5,2*w+.001,2*h+.001)
b=b.LimitedSharpenFaster(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost)


mt_merge(a,b,m,Y=3,U=3,V=3)
sD=mt_makediff(a,last)
fv2 = a.MVAnalyse(isb=false,delta=2,idx=13)
fv1 = a.MVAnalyse(isb=false,delta=1,idx=13)
bv1 = a.MVAnalyse(isb=true, delta=1,idx=13)
bv2 = a.MVAnalyse(isb=true, delta=2,idx=13)

sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14)

reduc = 0.4
sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc)

a.mt_makediff(sDD,U=2,V=2)

return last
}

And now compare them. Left - sangnom, right - eedi2.

http://ipicture.ru/uploads/080807/8073/thumbs/AihJn6JCxS.png (http://ipicture.ru/Gallery/Viewfull/4864548.html)

You didn't have to do that.
All I meant was I used EEDI2 to do some testing here and there.

I'm getting a "no function named DEdgeMask" error.
I have LimitedSharpenFaster, mt_masktools-26.dll, and warpsharp.dll
During a search said you needed those three, but still returns the error.

[martino]

You need the old masktools for DEdgeMask.


P.S. You need Toon, EEDI2 and MVTools too.

[Seraphic-]

You need the old masktools for DEdgeMask.


P.S. You need Toon, EEDI2 and MVTools too.

Yup, have those. Just having a hard time finding Toon.

[Mystery Keeper]

http://mf.creations.nl/avs/filters/Toon-v1.0.dll

[Seraphic-]

http://mf.creations.nl/avs/filters/Toon-v1.0.dll

Thanks. Quite an impressive script as the default setting removed a lot o the jagged edges without much smoothing.
It over smooths on some of my other videos, so I'll have to mess around with the settings.

I tried to load your EEDI2 version from post #42, but get no function named "SharpAAMC".
However, after searching, I found nothing on it so I'm not sure what I'm missing.

(No AA Top - With AA Bottom)
http://www.temp.seraphicgate.com/noaa.jpg
http://www.temp.seraphicgate.com/withaa.jpg

Edit2: Okay, it is REALLY SLOW. I am getting around 0.39fps on a lossless encode of a 720p source. Anyway to speed this up? Maybe use the Masktools 2 version of DEdgemask? (If it even exists, I don't know)

I'm getting 1.55fps on one of my lossless 720p captures.
For a 3:30 long video, that works out to about two hours. :p

[Mystery Keeper]

EEDI2 version is called ESharpAAMC.

[Seraphic-]

EEDI2 version is called ESharpAAMC.

Ah, see what you mean now. Thanks.
Yeah, it is much slower as you said.
Getting 1.15fps, down from 1.55fps with the other.

During your testing, how did you find these scripts to handle fast motion?

[Mystery Keeper]

Fast motion scenes are handled fine, but look a little jerky due to unusual sharpness. Actually, nothing special needs to be done for fast motion. Motion compensation is done for slow motion to compensate the flicker.

[Adub]

Hmm...I seem to see a couple of artifacts produced from this script, such as here:
http://i34.photobucket.com/albums/d125/Merlin7777/snapshot20080807082354.jpg


I assume there is a setting I can change to protect against this?

[Mystery Keeper]

If you are sure that it is caused by that function, just disable motion compensation. I'll make it adjustable in the function a bit later.

@Didée, how can we make it apply MC only to if scene changes a little? 'Twould be good if it took a scene change threshold as parameter.

[Adub]

Yep, this was the only function in the filter chain. It's funny, I have seen pictures of artifacts like this before, and there is a way to guard against it in MVTools, but I can't for the life of me recall what it is.

[Didée]

That artefact looks like if the sharpen-difference is merged in without motion compensation from a neighbor frame. From the way how MVAnalyse & MVDegrain work and how the script is layed out, I don't see how such a thing can sneak in at all.

Try truemotion=false,search=3,searchparam=256 :p

[Mystery Keeper]

0_0 WEIRD. I suddenly started getting an error "RemoveGrain: invalid mode 19 (LimitedSharpenFaster.avsi, line 106)"

Fixed

[Mystery Keeper]

Sorry, Merlin. I don't get such artifacts anywhere, so I can't help. Here's the function in which you can disable/enable MC easily.

function SharpAAMC(clip orig, int "aastr", float "ds", int "ShPre", int "ShPost", int "SmPost", bool "MC")
{
aastr=default(aastr,255) #antialiasing strength
ds=default(ds,0.2) #strokes darkening strength
ShPre=default(ShPre,80) #Presharpening
ShPost=default(ShPost,300) #Postsharpening
SmPost=default(SmPost,100) #Postsmoothing
MC=default(MC, true) #Motion Compensation

a=orig
w=width(a)
h=height(a)
m=logic( a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale.levels(0,0.8,128,0,255,false)
b=a.Toon(ds).Warpsharp(depth=ShPre)
b=b.spline64resize(w*2,h*2).TurnLeft().SangNom(aa=aastr).TurnRight().SangNom(aa=aastr)
b=b.spline64resize(w,h).LimitedSharpenFaster(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost)

b=mt_merge(a,b,m,Y=3,U=3,V=3)
sD=mt_makediff(a,b)
fv2 = a.MVAnalyse(isb=false,delta=2,idx=13)
fv1 = a.MVAnalyse(isb=false,delta=1,idx=13)
bv1 = a.MVAnalyse(isb=true, delta=1,idx=13)
bv2 = a.MVAnalyse(isb=true, delta=2,idx=13)

sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14)

reduc = 0.4
sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc)

b=MC ? a.mt_makediff(sDD,U=2,V=2) : b

return b
}

[Sagekilla]

IIRC, I saw somewhere that you could do a pseudo scene-change based on change in luminance, and depending on that it could ramp up a filter to process more if there was fast motion or less if low motion.

[martino]

http://avisynth.org/mediawiki/ConditionalFilter#Advanced_conditional_filtering:_part_III
It's not thought through well. I've got a badly modified version with some fixes, so if you want to use it as a reference or something just ask. Also masktools has some scenechange stuff, as well as mvtools IIRC.

[Adub]

Blast you Didee!!! Blast you!!

It takes nearly a minute to render 1 frame! That is insane!! Hell, I can't see the artifacts when the clip is in motion. It was pure chance that I managed to spot it when stepping through frames.

[Seraphic-]

Hi,

What function is related to "spline64resize"?
I'm getting a no function named error for "spline64resize".
Not sure why as the script was working the other day.

[Adub]

Did you recently uninstall and reinstall avisynth?

It is an internal function, but I don't think it comes in avisynth 2.57. I believe it was implemented in 2.58.

[Seraphic-]

Did you recently uninstall and reinstall avisynth?

It is an internal function, but I don't think it comes in avisynth 2.57. I believe it was implemented in 2.58.

Ah, yes I did. Must have installed the wrong version there. Thanks

Also, is there anyway to something like "threads=4" to take advantage of four CPU cores?

[Adub]

For what? There are filters that can do this, yes, but many don't.

Normally, the way to run filters multithreaded is to use MT. Just run a search and you will find it.

[thetoof]

As requested by PM, here my modded version with a new "MT" parameter. It will be used as aa=4 in the next version of animeivtc and will be included in animeivtc.avsi
I may add other parameters (such as overlap, pel, search...) later.
True = use josey_wells' multithreaded MVTools with the new syntax (default)
False = Old syntax, "official" development is up-to-date, but no MT
Also added tradius parameter if you want to use more frames to degrain. Default is 2 since SharpAAMC() uses MVDegrain2.

function SharpAAMC(clip orig, int "aastr", float "ds", int "ShPre", int "ShPost", int "SmPost", bool "MC", bool "MT", int "tradius")
{
aastr=default(aastr,255) #antialiasing strength
ds=default(ds,0.2) #strokes darkening strength
ShPre=default(ShPre,80) #Presharpening
ShPost=default(ShPost,300) #Postsharpening
SmPost=default(SmPost,100) #Postsmoothing
MC=default(MC, true) #Motion Compensation
MT=default(MT,true) # Use josey_wells' branch of MVTools
Tradius=default(Tradius,2) # 2 = MVDegrain2 / 3 = MVDegrain3 /// 28 = MVDegrain 28

a=orig
w=width(a)
h=height(a)
m=logic( a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale().levels(0,0.8,128,0,255,false)
b=a.Toon(ds).Warpsharp(depth=ShPre)
b=b.spline64resize(w*2,h*2).TurnLeft().SangNom(aa=aastr).TurnRight().SangNom(aa=aastr)
b=b.spline64resize(w,h).LimitedSharpenFaster(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost)

MaskedMerge(a,b,m,Y=3,U=3,V=3)
sD=mt_makediff(a,last)

fv2 = a.MVAnalyse(isb=false,delta=2,idx=13)
fv1 = a.MVAnalyse(isb=false,delta=1,idx=13)
bv1 = a.MVAnalyse(isb=true, delta=1,idx=13)
bv2 = a.MVAnalyse(isb=true, delta=2,idx=13)
allv = a.MVAnalyseMulti(refframes=tradius,idx=13)

sDD = (MT) ? sD.MVDegrainMulti(allv,idx=14) : sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14)

reduc = 0.4
sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc)

return MC ? a.mt_makediff(sDD,U=2,V=2) : b
}

[egrimisu]

what aa is aa4 ?

[Mystery Keeper]

sDD = (MT) ? sD.MVDegrainMulti(allv,idx=14) : (MT==false) ? sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14) : NOP()


o_0 ??? With what condition would that NOP() work?

[thetoof]

haha, you're right... dunno what I was thinking when I wrote that :p *edited the first post*

[Mystery Keeper]

By the way, thetoof. There's NNEDI antialiasing in your script which you say is my suggestion, modified by you. I suggested using EEDI2. What made you change it to NNEDI? NNEDI is more ACCURATE interpolation. 'Tis not as good for antialiasing as EEDI2.

[thetoof]

Woah, that's some bad mistake I made back then... I simply thought "nnedi is more accurate, so it must be better", changed it without testing and took it for granted afterwards.
I just tested it on the image we had in the "Denoise of tv-anime" thread and it's uber obvious that nnedi is ineffective where eedi2 works like wonders.
It will be changed in the next version animeivtc.

[thetoof]

@ MysteryKeeper: you've had a 1/2-pel registration error between the original (a) and antialiased (b) clip, due to EEDI2. Corrected by the blue parts.
May I ask you what is the purpose/theory/explanation behind 2*w+.001,2*h+.001?

edit: I also compared a random original and aaed clip by using the following script to find which values were appropriate (the image did not move in any direction after ediaa)
function ediaa(clip a) {return a.EEDI2(field=1).TurnRight().EEDI2(field=1).TurnLeft().spline36resize(a.width,a.height,-0.5,-0.5,2*a.width+.001,2*a.height+.001)}
interleave(source, source.ediaa())
Note that I used -0.5,-0.5 to make it work and that you recommended 0.5,-0.5Am I missing something?

[Gavino]

function ediaa(clip a) {return a.EEDI2(field=1).TurnRight().EEDI2(field=1).TurnLeft().spline36resize(a.width,a.height,-0.5,-0.5,2*a.width+.001,2*a.height+.001)}
interleave(source, source.ediaa())
Note that I used -0.5,-0.5 to make it work and that you recommended 0.5,-0.5. Am I missing something?
Didée's version did the eedi upsizing in a different order:
b=a.Warpsharp(depth=100).TurnLeft().EEDI2().TurnRight().EEDI2()
Since his horizontal resizing was done on a left-turned version, and yours on a right-turned one, the horizontal shift correction is reversed.

[thetoof]

Oh, you're right. Thanks!

Now about the 2*w+.001,2*h+.001 thingy...

[Gavino]

Now about the 2*w+.001,2*h+.001 thingy...
No, I don't get that bit either.

It may have been intended to force resampling to be done (avoiding the no-op optimisation for same-size resizing), but using a non-zero crop position does that already.

[Didée]

It's for backward compatibility with Avisynth 2.5.6. That version would have the resizer going NOP without the +.001 thingy, indeed.