Fine anime antialiasing

[DarkT]

I dunno... I'm getting nice results with:

Function SAA(Clip Clp,Int"SS",Bool"CP")
{

OX = Clp.Width
OY = Clp.Height
SS = Default(SS,2)
CP = Default(CP,True)


Clp = Clp.IsYV12() ? Clp : Clp.ConvertToYV12()

Clp.BilinearResize(OX*SS,OY*SS).SangNom().TurnRight()
\ .SangNom().TurnLeft().Spline36Resize(OX,OY)

CP ? Last : MergeChroma(Clp)

Return(Last)

}

ImageSource("untitled.bmp", start=1, end=587,fps=23.976023976023976023976023976024).converttoyv12()

SAA()
Toon(strength=0.3)
WarpSharp(96, 10, 128, -0.6)
LimitedSharpenFaster(smode=4, strength=77)

Anywya, my 2 cents...

[Adub]

Yeah, with my passed experiences with Sagnom, I lost way to many details, and with Spline36resize on top of that, it starts to look to blurry for my personal taste.

[Mystery Keeper]

Originally Posted by Merlin7777

Yeah, with my passed experiences with Sagnom, I lost way to many details, and with Spline36resize on top of that, it starts to look to blurry for my personal taste.

That's why 'tis masked. Well, warpsharp in this function still kills some details (carves edges), but it can be controlled by ShPre parameter.

Folk, critique is good, but don't just criticize, suggest improvements.

[Adub]

My post was referring to the post by DarkT directly above mine. I haven't even looked at your script. I will now though.

Edit: Wow!! This works wonders on a source I was looking at! It had some nasty jaggies from a bad deinterlace, but this pretty much eliminated them. It's a bit slow though, but for the quality, I'll take it.

Edit2: Okay, it is REALLY SLOW. I am getting around 0.39fps on a lossless encode of a 720p source. Anyway to speed this up? Maybe use the Masktools 2 version of DEdgemask? (If it even exists, I don't know)

[Mystery Keeper]

If you comment the lines between "MaskedMerge()" and "return last", it will disable the motion compensation and probably add you some speed.

Sorry, I'm not well versed in AviSynth ^_^'

[Adub]

Yes, but I like the motion comp. If there is nothing that can be done, I will except my fate. But I'll keep my fingers crossed!!

[Seraphic-]

Originally Posted by Mystery Keeper

Sorry, I'm not well versed in AviSynth ^_^'

You seem to know enough to be able to write an impressive script.
Did you find a guide or put it together from bits and pieces from searching around?

Also, could you post your full script code? I was trying to run your script from post #18, but I'm not able to get it to load.
I am most likely missing a few need things.

[martino]

Originally Posted by Merlin7777

Maybe use the Masktools 2 version of DEdgemask? (If it even exists, I don't know)

Nope, it doesn't. But with some fiddling you could get something similar with mt_edge's prewitt mode.

[TheRyuu]

Can we get a demuxed m2v or something I can play around with? (preferable with a couple scenes, 30 seconds or something, I dunno)

[Adub]

Seraphic, what was your problem? Meaning, what error were you getting?

[Seraphic-]

Originally Posted by Merlin7777

Seraphic, what was your problem? Meaning, what error were you getting?

"Return value was not a video clip"

[Adub]

Try copying and pasting the script again. Make sure you scroll down all the way to the bottom as well. It sounds like you missed a few lines.

[Comatose]

Mystery Keeper: Why not replace MaskedMerge(a,b,m,Y=3,U=3,V=3) with mt_merge(a,b,m,Y=3,U=3,V=3)?
Also, you're getting the eye "sparkle" loss because of SangNom ;_; They're small spots so it eats them :3 Search for starfield (in this forum).

[Mystery Keeper]

@Seraphic, I have only done the AA part, using people recommendations even there, and made an adjustable function. All other parts are written by somebody else.

How can I split a vob file? I'd love to give it to you for experiments. Folk, I know that's a wrong thread for it, but I'm desperate! I can't denoise it. Whatever I try spoils the video. Grid artifacts, ghosts, washed out details. Nothing seems to work for it.

[Mystery Keeper]

Originally Posted by Comatose

Mystery Keeper: Why not replace MaskedMerge(a,b,m,Y=3,U=3,V=3) with mt_merge(a,b,m,Y=3,U=3,V=3)?

Done. Thanks.

[Seraphic-]

Originally Posted by Merlin7777

Try copying and pasting the script again. Make sure you scroll down all the way to the bottom as well. It sounds like you missed a few lines.

Yes, seems I did miss a few lines.
It loads now, but doesn't seem to be changing the appearance of the video at all.
Was something else left out?

Originally Posted by Mystery Keeper

@Seraphic, I have only done the AA part, using people recommendations even there, and made an adjustable function. All other parts are written by somebody else.

The AA technique that I am familiar with is "EEDI2().TurnRight().EEDI2().TurnLeft()" then "spline36resize" down to the correct resolution.
It does a good job and removes a fair amount of edge jaggies, but also adds a fair amount of blur.

[Mystery Keeper]

Originally Posted by Seraphic-

The AA technique that I am familiar with is "EEDI2().TurnRight().EEDI2().TurnLeft()" then "spline36resize" down to the correct resolution.
It does a good job and removes a fair amount of edge jaggies, but also adds a fair amount of blur.

'Twas in the initial version of the script. Blur is dealt with alright. 'Tis just that using EEDI2 will make it much slower while not smoothing the lines as good as sangnom. Maybe I'm wrong. Need to test.

[martino]

You never called the function itself.

Originally Posted by Seraphic-

It does a good job and removes a fair amount of edge jaggies, but also adds a fair amount of blur.

By no way as bad as with SangNom, blur-wise.

[45tripp]

Originally Posted by Mystery Keeper

How can I split a vob file? I'd love to give it to you for experiments. .

dgindex.
'[' ']' select section, file -> 'save and demux'

[Seraphic-]

Originally Posted by martino

You never called the function itself.

How do you call the function?