Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 6th December 2011, 19:15   #1  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
AntiAliasHQ - Oh snap!

Oh snap, it's me!

And it's another antialiasing script, argh!!


Anyways, this is a new (whoah!) script by (silly) me.
It's made to kill jaggies, using both the old SangNom and the RemoveGrain mode 21.

And since it uses nothing else, it doesn't really smooth much.
It does include a no-edge sharpening, though it gets outside edges a lot at thr<5

It's kinda fast (read: kinda), too. Because a good additional script is a fast additional script.

Feel free to try it out, also brownie points to redfordxx for the awesome plugin. (I love messing with differences.)

Have fun!

Code:
function AntiAliasHQ(clip clp,int "thr",int "aa",bool "sharpen",float "sharp") {
# Another antialiasing script by, you guessed it - Bloax.
# Extra cookies to redfordxx for the masking simplification idea. (Now there are two masks, instead of two masks and two inversions!)

# I doubt this one's really usable for much, but I'll just go ahead and post it -
# Since it 'bypasses' a SangNom issue hackily, and combines it with RemoveGrain mode 21.

# It's also pretty fast. No supersampling for you though!
# But overall, it doesn't really "smooth" much, it does kill jaggies quite nicely though. (Game footage, anyone?)

# Requirements: RemoveGrain, RedAverage.dll (You can google it), SangNom, MaskTools2

# "thr" Is the threshold for detecting edges, higher values INCREASE the amount of edges found.
# It's operating in Pow2, so be careful about the values. (6 is 64, while 8 is >256< :P) [/basicmath]
# > Seems like lower values are softer. Lower values sharpen more though, whilst higher sharpen less.
# Due to the fact that when more edges are "detected", there's less overall space in the inverse edge mask.
# Which is what the sharpening is limited to.

# "aa" Is a silly little value for SangNom, I have no idea about what it does above 48-64.
# "sharpen" Controls whenever there's internal (non-edge) sharpening, and "sharp" controls the strength.

# Enjoy!

thr = Default(thr, 5)
aa = Default(aa, 1024)
sharpen = Default(sharpen, true)
sharp = Default(sharp, 0.33)

thrval = Pow(2,thr)

clp2 = IsYV12(clp) ? clp : clp.ConvertToYV12() # Don't we love to copy this? :-)

edgemsk=RAverageW(clp2,-thrval,clp2.RemoveGrain(19,-1),thrval,mode=4,u=0,v=0,bias=256)
aa1=clp2.SangNom(0,aa)
msk1=RAverageW(clp2,-64,aa1,64,mode=4,u=0,v=0,bias=-64)
aa2=clp2.SangNom(1,aa)
merge=RMerge(aa2,aa1,msk1.Mt_Expand().Mt_Deflate().Mt_Inflate(),mode=255)
clean=sharpen ? RMerge(merge,clp2.Sharpen(sharp),edgemsk.Mt_Deflate(),mode=255) : RMerge(clp2,merge,edgemsk.Mt_Deflate(),mode=255)
RMerge(clean.RemoveGrain(21),clean,edgemsk,mode=255)
}
P.S: Yes, I do mask PROPERLY here.
..Or do I!?

[Previous version]

Last edited by Bloax; 8th December 2011 at 09:22.
Bloax is offline   Reply With Quote
Old 6th December 2011, 22:59   #2  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Some speedup?
Code:
edgemsk=RAverageW(clp2,-thrval,clp2.RemoveGrain(19,-1),thrval,mode=4,u=0,v=0,bias=255+128)
Code:
RMerge(clean.RemoveGrain(21),clean,edgemsk,mode=255)
same with invmsk1 maybe?

Last edited by redfordxx; 6th December 2011 at 23:04.
redfordxx is offline   Reply With Quote
Old 7th December 2011, 08:11   #3  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
Hm, is doing the whole averaging faster than a simple reversion?
Bloax is offline   Reply With Quote
Old 7th December 2011, 12:52   #4  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by Bloax View Post
Hm, is doing the whole averaging faster than a simple reversion?
The thing is, you are adding the processing time of reversion to the processing time of averaging which happens anyway. And it can all be done in the averaging
And by the way, the filter works on two clips at once so the processing time of even and odd number of clips should be almost the same, provided you don't have bottleneck in the data transfer.
I could not mesure the exact speed coz I have bottle neck in data transfer (10GB/s = ~1000FPSFullHD and the filter is faster)
redfordxx is offline   Reply With Quote
Old 7th December 2011, 13:10   #5  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
I'm actually talking about running RemoveGrain(19,-1) again to get the edges in reverse, instead of just reversing the existing mask.
Bloax is offline   Reply With Quote
Old 7th December 2011, 13:35   #6  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
He's not running RemoveGrain() again -you also remove the assignment to 'edges' which is no longer used.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 7th December 2011, 13:36   #7  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Quote:
Originally Posted by Bloax View Post
P.S: Yes, I do mask PROPERLY here.
No, you don't. The masks are missing 50% of area-of-interest, because you're cutting the darker half.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 7th December 2011 at 14:07. Reason: "grammar"
Didée is offline   Reply With Quote
Old 7th December 2011, 15:52   #8  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
Hm, are you guys talking about instead of doing
Code:
RMerge(last,stuff,edgeshere) & RMerge(previousstuff,newstuff,invertedges)
Then just do
Code:
RMerge(stuff,last,invertedges) & RMerge(morestuff,previousstuff,invertedges) ?
If so, then I finally see your point. It's just that I was in a hurry when I wrote the previous post.

I'll mess with it some time later (maybe today), since I'm kinda busy on wednesdays.

Quote:
Originally Posted by Didée View Post
No, you don't. The masks are missing 50% of area-of-interest, because you're cutting the darker half.
D: !

Are we talking about the bias here?
If so, then it's for conservative edge-y reasons. Because the way I'm getting the edges is horribly hacky, and thus unbiased averaging results in a vast overkill of a detection.

What I'm talking about; Bias 0 - Bias -128

Now to find out whenever Mt_Edge is faster or not (LOL!), but I certainly find the amount of edges found appropriate.
As said, I'm a silly, silly guy.

Last edited by Bloax; 7th December 2011 at 15:53. Reason: D:
Bloax is offline   Reply With Quote
Old 7th December 2011, 16:30   #9  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
I don't get why are you talking about removegrain
Code:
edges=RAverageW(clp2,thrval,clp2.RemoveGrain(19,-1),thrval*-1,mode=4,u=0,v=0,bias=-128)
edgemsk=RAverageW(edges,-1,mode=8,u=0,v=0,bias=255) # Inverse edge mask, for removing SangNom weirdness outside edges.
[basicmath]
edges=thrval*x+thrval*-1*y-128
edgemsk=255-1*edges
==========
edgemsk=255-(thrval*x+thrval*-1*y-128)=255+128-thrval*x+thrval*y
[/basicmath]
You replace two lines of your code with the one I posted in the beginning, but you must also switch the sides in RMerge()

I believe invmsk1 is the same case.
redfordxx is offline   Reply With Quote
Old 7th December 2011, 17:10   #10  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
I did finally get it now, it's just that I was in a hurry when I wrote that post. (And thus didn't get the point.)

Now to try it out.

Alright, it seems a bit faster now. Cookies are to be handed out.

Edit:
Code:
edge=RAverageW(last,32,last.RemoveGrain(19,-1),-32,u=0,v=0,bias=-64,mode=4)
aa1=RemoveGrain(15)
msk=RAverageW(last,32,aa1,-32,u=0,v=0,mode=4)
RMerge(last,RMerge(RemoveGrain(16),aa1,msk.Mt_Inflate()),edge)
Seems like deinterlacers can be used for antialiasing in general. Hah!

Last edited by Bloax; 12th December 2011 at 15:54.
Bloax is offline   Reply With Quote
Old 12th December 2011, 22:26   #11  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
Hm, during my rather short observance, I noted that
Code:
(Deinterlancer).TurnRight().(Deinterlacer).TurnLeft()
Provides really sub-par results.

A better solution (for antialiasing at least), is to do this:
Code:
Merge(Deinterlacer(TopField),Deinterlacer(BottomField),0.5)
Sense? Dunno.
Good results? You bet.
Bloax is offline   Reply With Quote
Old 12th December 2011, 23:28   #12  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Quote:
Originally Posted by Bloax View Post
Sense? Dunno.
Good results? You bet.
Sure so!

It made sense and gave good results five years ago, did ever since, still does, and will continue to do.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 13th December 2011, 05:09   #13  |  Link
dansrfe
Registered User
 
Join Date: Jan 2009
Posts: 1,210
^ and that's the "oh snap" lol
dansrfe is offline   Reply With Quote
Old 13th December 2011, 07:15   #14  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
I wonder why you didn't scream in their face that they're doing something really stupid then. ;P
Since honestly, Merge(SangNom(0,255),SangNom(1,255),0.5) looks way better than SangNom(0,255).TurnRight().SangNom(0,255).TurnLeft()
Bloax is offline   Reply With Quote
Old 13th December 2011, 10:37   #15  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 712
They do different things. The former method generally erases any 1-pixel-high vertical detail. It is suited to post-process deinterlacing or IVTC problems (horizontal aliasing, residual combing, etc). It's the DAA() way.

You generally use the latter method on 2x-upscaled data to smooth a uniform aliasing (horizontal and vertical) caused by a bad digital conversion. It preserves pixel-sized details correctly. Of course you can upscale, DAA() and turn, but it involves twice more deinterlacing operations.
__________________
dither 1.28.1 for AviSynth | avstp 1.0.4 for AviSynth development | fmtconv r30 for Vapoursynth & Avs+ | trimx264opt segmented encoding

Last edited by cretindesalpes; 13th December 2011 at 10:42.
cretindesalpes is offline   Reply With Quote
Old 13th December 2011, 11:01   #16  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
You sure are right, but for Ye Olde Antialiasing (edge smoothening), the former is much better.

Though this:
Code:
PointResize(Width*2,Height*2)
SangNom(0,255).TurnLeft().SangNom(0,255).TurnRight()
Merge(SangNom(0,255),SangNom(1,255),0.5)
Looks pretty funky. (And the latter is indeed useful for 2x upscaled material.)
Though for antialiasing, it causes even worse jaggies, as it discards an entire field, and thus the existing jaggies get much wider. (And thus noticeable.)
Bloax is offline   Reply With Quote
Old 13th December 2011, 11:07   #17  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
@cretindesalpes - Yepp, just that the 2*upscaling part was actually missing in the scripts of this thread. The used deinterlacer throws away half of the pixels, although they (should) contain "good" data in game footage.

How many AA filters are available for Avisynth - a dozen? Two or three dozens if you count the "variants" of them? All code that makes sense is already out there (the LSharpAAF collection comes to mind). You only need to realize what your source is about, then choose an appropriate method.

Presuming that all pixels are valid (no interlacing or former de-interlacing) in game footage, a theoretically-sufficient filter would be

NNEDI3_rpow2(2,cshift="bicubicresize",fwidth=width,fheight=height)

A bit problematic could be if there are very "hard" or very "high contrast" contours - in such areas, NNEDI3 every now&then doesn't want to connect. (Other deinterlacers may show a similar problem.)
In this case, processing twice can help, maybe with a blur-bypass in between.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 13th December 2011 at 11:17.
Didée is offline   Reply With Quote
Old 13th December 2011, 11:39   #18  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
I have messed around with NNEDI3, but honestly, it's just too slow to be useful.
Antialiasing is supposed to be add-on filtering, not the heavyweight script that sets the FPS to <20

Talking about that, this seems to provide the highest quality:
Code:
Merge(SangNom(0,255),SangNom(1,255),0.5)
While this one is slightly faster (~25% on the source I test on):
Code:
Merge(SangNom(0,255),Yadif(2,1),0.5)
But provides slightly less smooth results. (Which may be preferable, actually.)
Bloax is offline   Reply With Quote
Old 13th December 2011, 12:17   #19  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
You surely didn't mean to say "the best", because really, it isn't.

As said, the "general" antialiasing methods are out there already.

For your specific case of game footage, advice is not possible without having seen something. In this thread you're randomly mixing one-axis processing with two-axis-processing (horizontal-only versus horizontal+vertical) - is the game using some asymetric AA from the start? It's not even clear what type of game footage it is - some modern FPS shooter, or Mario or Donkey Kong on an emulator? It's all different footage, with different footage having different problems, thus requiring different solutions.

Discussing in the blue, then saying "this script da best" is a bit ... nebulous.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 13th December 2011, 12:48   #20  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
I'm not saying that "This is the best", oops.
I actually mean "The best deinterlacer for the job", I tried NNEDI3 but it's inferior to SangNom in that aspect. (Also: Speed.)
YADIF is a pretty nice solution too, and fast. Though a bit too sharp for my tastes, it's still very useful.

Here's the clip, it's kind of a mixed bag (both big and small aliasing), which is why I use it for testing.
I don't exactly have much material to work with, since I don't really have a lot of (modern) games.
Bloax is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:59.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.