Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
3rd August 2008, 21:57 | #1 | Link |
Beyond Kawaii
Join Date: Feb 2008
Location: Russia
Posts: 724
|
Fine anime antialiasing
I am trying to encode .hack//SIGN and I've got an encode by KAA to compare. Those guys have lost awfully lot of details, but, on the bright side, they've got characters edge strokes smooth and thin. I tried many AA methods, and none could make the same result.
Here's KAA's encode Here's my frame without AA. Here's my frame with eedi2().TurnRight().eedi2().TurnLeft() And here's my frame with triple tweaked "antialiasing()" script As you can see, antialiasing() gives rather good result. BUT! It came to be rather unstable and gives heavy flickering in some motion scenes. Does anyone know how I can make the edge strokes as thin and smooth? I tried asking around KAA, and they said their encoder is WoW addict, which they can't contact for months T_T
__________________
...desu! |
3rd August 2008, 22:55 | #6 | Link | |
Beyond Kawaii
Join Date: Feb 2008
Location: Russia
Posts: 724
|
Quote:
Upd: Thanks! I guess I can come up with something nice if I play with that. Anyone got other suggestions?
__________________
...desu! Last edited by Mystery Keeper; 3rd August 2008 at 23:00. |
|
3rd August 2008, 23:34 | #7 | Link |
Beyond Kawaii
Join Date: Feb 2008
Location: Russia
Posts: 724
|
Ok, this draft gives nice result:
Code:
a=last w=width(a) h=height(a) b=a.Warpsharp(depth=100).TurnLeft().EEDI2().TurnRight().EEDI2() m=logic( b.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3) \ ,b.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3) \ ,"max").greyscale.levels(0,0.8,128,0,255,false).spline64resize(w,h) b=b.spline64resize(w,h).Toon(0.5) MaskedMerge(a,b,m,Y=3,U=3,V=3)
__________________
...desu! Last edited by Mystery Keeper; 3rd August 2008 at 23:50. |
4th August 2008, 00:09 | #8 | Link |
x264aholic
Join Date: Jul 2007
Location: New York
Posts: 1,752
|
Sure, if you do this:
bvec = a.MVAnalyse(isb=true,delta=1,idx=1,ov=4,pel=2) fvec = a.MVAnalyse(isb=false,delta=1,idx=1,ov=4,pel=2) bcomp = a.MVFlow(bcomp, idx=1, thSCD1=500) fcomp= a.MVFlow(fcomp, idx=1, thSCD1=500) b = Interleave(bcomp,a,fcomp) w=width(b) h=height(b) c=b.Warpsharp(depth=100).TurnLeft().EEDI2().TurnRight().EEDI2() m=logic( c.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3) \ ,c.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3) \ ,"max").greyscale.levels(0,0.8,128,0,255,false).spline64resize(w,h) c=c.spline64resize(w,h).Toon(0.5) MaskedMerge(b,c,m,Y=3,U=3,V=3) TemporalRepair(b,last) That should work, I believe. |
4th August 2008, 00:21 | #10 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
@ Sagekilla: Not quite what I would've expected, but try & see how that turns out. Downside is that the EEDI2 interpolation now is done on 3 times more frames than before ...
Also, me thinks there's a SelectEvery(3,1) missing somewhere, to undo the interleave? My go (not tested though): MC'ed Soothe. Code:
a=last.assumetff() w=width(a) h=height(a) b=a.Warpsharp(depth=100).TurnLeft().EEDI2().TurnRight().EEDI2() m=logic( b.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3) \ ,b.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3) \ ,"max").greyscale.levels(0,0.8,128,0,255,false).spline64resize(w,h) b=b.spline64resize(w,h,0.5,-0.5,2*w+.001,2*h+.001).Toon(0.5) MaskedMerge(a,b,m,Y=3,U=3,V=3) sD=mt_makediff(a,last) fv2 = a.MVAnalyse(isb=false,delta=2,idx=13) fv1 = a.MVAnalyse(isb=false,delta=1,idx=13) bv1 = a.MVAnalyse(isb=true, delta=1,idx=13) bv2 = a.MVAnalyse(isb=true, delta=2,idx=13) sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14) reduc = 0.4 sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?") .mergeluma(sDD,1.0-reduc) a.mt_makediff(sDD,U=2,V=2)
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
4th August 2008, 00:56 | #12 | Link |
x264aholic
Join Date: Jul 2007
Location: New York
Posts: 1,752
|
Ah yes, I forgot about SelectEvery(3,1). Also, why is it necessary for the AssumeTFF()? I was just assuming this source was progressive, and wouldn't need any interlace badgering.
Edit: Interlacing inside of anime isn't as uncommon as you might think. Unfortunately very annoying stuff happens in anime all the time |
4th August 2008, 01:03 | #13 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
Progressive is what I expected, too. The AssumeTFF was just to make sure in which direction the frame gets shifted by EEDI2. It's a bad shortcut, yes, I'm too lazy and tired to look up the necessary parameters to make it the "proper" way.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
4th August 2008, 01:52 | #14 | Link |
Beyond Kawaii
Join Date: Feb 2008
Location: Russia
Posts: 724
|
@Sagekilla, interlacing is usually introduced in the places of montage. Well, that single frame was at the scene change.
Well, thanks guys. I see that my script is highly imperfect, but as I said, 'twas draft. Now, thanks to you all, I've got the wrapper I need and can play with it and make an adjustable function... tomorrow. Good night ^_^
__________________
...desu! Last edited by Mystery Keeper; 4th August 2008 at 01:55. |
4th August 2008, 13:37 | #15 | Link |
Beyond Kawaii
Join Date: Feb 2008
Location: Russia
Posts: 724
|
Now this looks very good ^_^ Though prone to some details carving.
Code:
function SharpAAMC(clip orig, int "aastr", float "ds", int "SPre", int "SPost", int "BPost") { aastr=default(aastr,255) #antialiasing strength ds=default(ds,0.2) #strokes darkening strength SPre=default(SPre,100) #Presharpening SPost=default(SPost,50) #Postsharpening BPost=default(Bpost,10) #Postbluring a=orig w=width(a) h=height(a) m=logic( a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3) \ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3) \ ,"max").greyscale.levels(0,0.8,128,0,255,false) b=a.Toon(ds).Warpsharp(depth=SPre) b=b.spline64resize(w*2,h*2).TurnLeft().SangNom(aa=aastr).TurnRight().SangNom(aa=aastr) b=b.Warpsharp(depth=SPost,blur=BPost).spline64resize(w,h) MaskedMerge(a,b,m,Y=3,U=3,V=3) sD=mt_makediff(a,last) fv2 = a.MVAnalyse(isb=false,delta=2,idx=13) fv1 = a.MVAnalyse(isb=false,delta=1,idx=13) bv1 = a.MVAnalyse(isb=true, delta=1,idx=13) bv2 = a.MVAnalyse(isb=true, delta=2,idx=13) sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14) reduc = 0.4 sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc) a.mt_makediff(sDD,U=2,V=2) return last }
__________________
...desu! Last edited by Mystery Keeper; 4th August 2008 at 13:42. |
4th August 2008, 15:54 | #18 | Link |
Beyond Kawaii
Join Date: Feb 2008
Location: Russia
Posts: 724
|
Ok. It's getting better and better. But now it seems to be sharpness VS strokes smoothness.
Code:
function SharpAAMC(clip orig, int "aastr", float "ds", int "ShPre", int "ShPost", int "SmPost") { aastr=default(aastr,255) #antialiasing strength ds=default(ds,0.2) #strokes darkening strength ShPre=default(ShPre,80) #Presharpening ShPost=default(ShPost,300) #Postsharpening SmPost=default(SmPost,100) #Postsmoothing a=orig w=width(a) h=height(a) m=logic( a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3) \ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3) \ ,"max").greyscale.levels(0,0.8,128,0,255,false) b=a.Toon(ds).Warpsharp(depth=ShPre) b=b.spline64resize(w*2,h*2).TurnLeft().SangNom(aa=aastr).TurnRight().SangNom(aa=aastr) b=b.spline64resize(w,h).LimitedSharpenFaster(edgemode=1,strength=ShPost,overshoot=1,soft=SmPost) MaskedMerge(a,b,m,Y=3,U=3,V=3) sD=mt_makediff(a,last) fv2 = a.MVAnalyse(isb=false,delta=2,idx=13) fv1 = a.MVAnalyse(isb=false,delta=1,idx=13) bv1 = a.MVAnalyse(isb=true, delta=1,idx=13) bv2 = a.MVAnalyse(isb=true, delta=2,idx=13) sDD = sD.MVDegrain2(bv1,fv1,bv2,fv2,idx=14) reduc = 0.4 sDD = mt_lutxy(sD,sDD,"x 128 - abs y 128 - abs < x y ?").mergeluma(sDD,1.0-reduc) a.mt_makediff(sDD,U=2,V=2) return last }
__________________
...desu! Last edited by Mystery Keeper; 4th August 2008 at 16:08. |
4th August 2008, 16:42 | #20 | Link |
Beyond Kawaii
Join Date: Feb 2008
Location: Russia
Posts: 724
|
Nah. It's tuned to sharpen the eyes area. Greater SmPost will smooth the lines nice and clean, but blur that area again. What's bothering me a bit is that some spots in eyes get lessened. Well, fine with me though. All the parameters can be tuned to make lesser damage in exchange of being less effective.
__________________
...desu! |
Tags |
anime, antialiasing, avisynth |
Thread Tools | Search this Thread |
Display Modes | |
|
|