Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 2nd September 2015, 08:44   #1  |  Link
thecoreyburton
Registered User
 
Join Date: Jul 2015
Posts: 104
3D Game Anti-Aliasing (FXAA, SMAA, MLAA)

I've seen a posts about this from previous years, but none with a definitive answer so I thought I'd ask the experts.

When you play a 3D game there are generally two categories of Anti-Aliasing. Traditional Anti-Aliasing, and post-Anti-Aliasing.

Traditional Anti-Aliasing methods include supersampling (SSAA, FSAA) and multi-sampling (MSAA).

Post Anti Aliasing methods include FXAA, SMAA and MLAA.

My understand is that the traditional category is done by the GPU and cannot be changed once the scene is rendered as it relies on the scene or polygons to be rendered at a higher resolution and then shrunk down. The "Post AA" category is different however. It relies on the scene to be rendered and then applies the effect on top of the image as a filter.

Like many others, my card (AMD R9-285) can force different types of AA. If I force MLAA in game, it gives a slightly blurred, but anti-aliased image (as I would expect). If I capture footage from the game with a program such as fraps and then play it back, the effect is not there. Consequently, if I force Media Player Classic to use MLAA, when I play back the footage in it, I get the exact same effect.

My point is, a lot of the time I see this question answered with reference to traditional anti-aliasing and comments regarding the fact you'd need the game itself to do this sort of effect. I however get the same results applying the effect in game as I do applying the effect to a video file recorded without the effect.

Are there any sort of filters or plugins that can achieve this sort of effect? I'd be looking to polish some videos of games that don't support any sort of anti-aliasing and figured here was the place to ask!

Thank you for your time, and sorry about the big post!
thecoreyburton is offline   Reply With Quote
Old 2nd September 2015, 09:00   #2  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,538
Upscale with eedi3 (sclip=nnedi3()) and then scale it back
Repeat it for a few times
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 2nd September 2015, 12:00   #3  |  Link
thecoreyburton
Registered User
 
Join Date: Jul 2015
Posts: 104
Thank you for your fast reply. It works, although the results aren't too nice. It takes quite a few times before the edges see any sort of effect and by then the image is quite distorted. The trade off for distortion vs. visible aliased edges of polygons remaining is almost not worth it.
thecoreyburton is offline   Reply With Quote
Old 2nd September 2015, 13:50   #4  |  Link
vivan
/人 ◕ ‿‿ ◕ 人\
 
Join Date: May 2011
Location: Russia
Posts: 649
http://avisynth.nl/index.php/Externa...#Anti-aliasing
Try daa, it's based on this approach but is a bit more sophisticated. There're also few other filters to try.
vivan is offline   Reply With Quote
Old 2nd September 2015, 14:16   #5  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,375
I love QTGMC for AA on game footage - it does some kind of temporal filtering, in addition to normal AA, to soften the twinkling/shimmering that I find annoying and hard to compress. There is an "EEDI3+NNEDI3" mode, although with my material I don't notice an improvement.
Code:
/* PLUGIN REQUIREMENTS (YMMV)
## mvtools2.dll == AutoLoad, in Plugins folder
## libflowsse.dll == AutoLoad, in Plugins folder
LoadPlugin("mt_masktools-26.dll")
LoadPlugin("RemoveGrainSSE2.dll")
LoadPlugin("nnedi3.dll")
LoadPlugin("eedi3.dll")
LoadPlugin("FFT3DFilter.dll")
*/
#QTGMC(preset="Very Slow", FPSDivisor=2, EdiMode="EEDI3+NNEDI3")
QTGMC(preset="Slower", FPSDivisor=2) ## good enough

Last edited by raffriff42; 2nd September 2015 at 14:19.
raffriff42 is offline   Reply With Quote
Old 2nd September 2015, 14:17   #6  |  Link
thecoreyburton
Registered User
 
Join Date: Jul 2015
Posts: 104
Here is a screenshot I took playing Mario Party through an emulator. It was the quickest decent example I could find. I used ImageSource to load it up from a png file and apply the anti aliasing. The outcome is similar, although with less noticeable distortion and a sharper look within the characters, though the outsides are noticeably aliased still. Daa was definitely faster.

Original:

Daa, five times:

EEDI3, five times:


Hopefully these screenshots will help a bit too.

Edit: I only saw the QTGMC post after this. I just tried it and found that the results provided the least distortion but also the least correction in relation to the above methods. It's definitely the best visually so far.

Last edited by thecoreyburton; 2nd September 2015 at 14:21.
thecoreyburton is offline   Reply With Quote
Old 2nd September 2015, 15:45   #7  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,538
u sure u did it correctly?



Code:
ImageSource("w7mzqg.png")

interleave (showred ("yv12"),showgreen ("yv12"),showblue("yv12"))
eedi3 (dh=true,alpha=1,beta=0,gamma=0,nrad=3,mdis=40,sclip=nnedi3 (dh=true, nsize=3, nns=4, qual=2, etype=1))
turnleft ()
eedi3 (dh=true,alpha=1,beta=0,gamma=0,nrad=3,mdis=40,sclip=nnedi3 (dh=true, nsize=3, nns=4, qual=2, etype=1))
turnright ()
bicubicresize (640, 240, b=-1, c=0)

eedi3 (dh=true,alpha=1,beta=0,gamma=0,nrad=3,mdis=40,sclip=nnedi3 (dh=true, nsize=3, nns=4, qual=2, etype=1))
turnleft ()
eedi3 (dh=true,alpha=1,beta=0,gamma=0,nrad=3,mdis=40,sclip=nnedi3 (dh=true, nsize=3, nns=4, qual=2, etype=1))
turnright ()
bicubicresize (640, 240, b=-1, c=0)

mergergb (selectevery (3,0),selectevery (3,1),selectevery (3,2))
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 2nd September 2015, 15:54   #8  |  Link
thecoreyburton
Registered User
 
Join Date: Jul 2015
Posts: 104
Quote:
Originally Posted by feisty2 View Post
u sure u did it correctly?
I thought I did, but I guess not! I didn't do any rotation or interleaving so that's probably where I went wrong. That result looks beautiful.
thecoreyburton is offline   Reply With Quote
Old 2nd September 2015, 16:09   #9  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,538
Quote:
Originally Posted by thecoreyburton View Post
I thought I did, but I guess not! I didn't do any rotation or interleaving so that's probably where I went wrong. That result looks beautiful.
yeah, I said "upscale it", not "deinterlace it"
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 2nd September 2015, 17:05   #10  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,145
FXAA sounds pretty simple to implement, but I can't quite get a full grasp of it from the documents I've found. Is there any interest in it, or is multi-NNEDI3 the preferred (if presumably slower) approach?
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is online now   Reply With Quote
Old 2nd September 2015, 17:19   #11  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,538
Quote:
Originally Posted by davidhorman View Post
Is there any interest in it
Quality is the one and only thing I care about, if the quality is lower than the script at #7, then no, for me
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 2nd September 2015, 17:56   #12  |  Link
Desbreko
Registered User
 
Desbreko's Avatar
 
Join Date: Feb 2015
Posts: 55
feisty2's script is a good method of antialiasing, provided you don't mind the slow speed of eedi3, but the center shift should be corrected. It can also be sped up by not processing the unused chroma channels, and using FTurn instead of the internal turn functions will speed it up some more unless you're using AviSynth+.

Code:
ImageSource("w7mzqg.png")

interleave (showred ("yv12"),showgreen ("yv12"),showblue("yv12"))
eedi3 (field=1,dh=true,U=false,V=false,alpha=1,beta=0,gamma=0,nrad=3,mdis=40,sclip=nnedi3 (field=1, dh=true, U=false, V=false, nsize=3, nns=4, qual=2, etype=1))
turnleft ()
eedi3 (field=1,dh=true,U=false,V=false,alpha=1,beta=0,gamma=0,nrad=3,mdis=40,sclip=nnedi3 (field=1, dh=true, U=false, V=false, nsize=3, nns=4, qual=2, etype=1))
turnright ()
bicubicresize (640, 240, b=-1, c=0, src_left=0.5, src_top=-0.5)

eedi3 (field=1,dh=true,U=false,V=false,alpha=1,beta=0,gamma=0,nrad=3,mdis=40,sclip=nnedi3 (field=1, dh=true, U=false, V=false, nsize=3, nns=4, qual=2, etype=1))
turnleft ()
eedi3 (field=1,dh=true,U=false,V=false,alpha=1,beta=0,gamma=0,nrad=3,mdis=40,sclip=nnedi3 (field=1, dh=true, U=false, V=false, nsize=3, nns=4, qual=2, etype=1))
turnright ()
bicubicresize (640, 240, b=-1, c=0, src_left=0.5, src_top=-0.5)

mergergb (selectevery (3,0),selectevery (3,1),selectevery (3,2))
__________________
Twitter, AviSynth scripts

Last edited by Desbreko; 2nd September 2015 at 17:59.
Desbreko is offline   Reply With Quote
Old 2nd September 2015, 18:20   #13  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,190
Quote:
Originally Posted by feisty2 View Post
mergergb (selectevery (3,0),selectevery (3,1),selectevery (3,2))
I was wondering what this line was about.

Temporal anti-aliasing. Smart.
MysteryX is offline   Reply With Quote
Old 2nd September 2015, 18:25   #14  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,538
Quote:
Originally Posted by MysteryX View Post
I was wondering what this line was about.

Temporal anti-aliasing. Smart.
Sadly, it's 100% spatial
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 2nd September 2015, 23:56   #15  |  Link
pbristow
Registered User
 
pbristow's Avatar
 
Join Date: Jun 2009
Location: UK
Posts: 262
Quote:
Originally Posted by davidhorman View Post
FXAA sounds pretty simple to implement, but I can't quite get a full grasp of it from the documents I've found. Is there any interest in it, or is multi-NNEDI3 the preferred (if presumably slower) approach?
The key to it seems to be the depth-aware edge detection. How do you do that, if you don't have a 3D model to get the depth info from? And is the technique even relevant, if you don't have a distinction between edges that need de-aliasing (because of how they were created in the game) and ones that don't (because they're from stored textures)?

If your source is a 2D image, then you've either got to try to infer depth from features of the image (lots of processing, and probably very error-prone); or else just process all detected edges equally, which means you're not really doing FXAA.
pbristow is offline   Reply With Quote
Old 3rd September 2015, 08:24   #16  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,145
Quote:
The key to it seems to be the depth-aware edge detection.
FXAA isn't depth-aware.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is online now   Reply With Quote
Old 3rd September 2015, 16:29   #17  |  Link
pbristow
Registered User
 
pbristow's Avatar
 
Join Date: Jun 2009
Location: UK
Posts: 262
Quote:
Originally Posted by davidhorman View Post
FXAA isn't depth-aware.
Hmmm...? [GOES GOOGLING]

Ah! It seems I have been led astray by a Wikipedia entry ( https://en.wikipedia.org/wiki/Fast_a..._anti-aliasing ). The original nVidia App Note ( http://developer.download.nvidia.com...WhitePaper.pdf ) doesn't mentioning anything about depth awareness, as you say, so I wonder what the Wikipedia author was on about. Time to submit a correction, methinks...

[EDITED TO ADD:] I've put something in the talk section about it. Didn't want to edit the article directly, as I'm not at all sure now what it *should* say, and don't really have time to write a new one from scratch... =:o\

Last edited by pbristow; 3rd September 2015 at 17:06.
pbristow is offline   Reply With Quote
Old 4th September 2015, 16:38   #18  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,375
@feisty2 (Post #7) & Desbreko (Post #12)
Beautiful results, but my puny i3 spins for ~15 seconds per frame @ 1080p!
I had to look for something based on your ideas, but faster (this one is about 15x faster for me)

(EDIT now with RGB support)

Code:
#Import("QTGMC-3.32.avsi")

ImageSource("E:\Data\Downloads\w7mzqg.png") ## Post #6, "Original"
#ConvertToYV12(matrix="Rec601", chromaresample="spline16") 

return pow2Qt

##################################
### Antialiasing (AA) for game footage (YV12 only)
### (blending nnedi3_rpow2 & QTGMC) 
##
## @ sharpen - post-process Sharpen to mitigate QTGMC smoothing
##             (on double-size image, so apparent effect is 1/2)
##             (0.2-0.3 best in initial tests; default 0.3)
## @ cpoint  - upscale pass2 chroma with PointResize (sharper)
## @ doublesize - 2x enlargement, but AA less effective
##
## v0.42.02 - support RGB32, RGB24, Y8 (and YV12 as before)
##
function pow2Qt(clip C, float "sharpen", bool "doublesize", bool "cpoint")
{
    Assert(C.IsRGB || C.IsYV12 || C.IsY8, 
    \   "pow2Qt: source must be RGB, YV12 or Y8")

    shp = Default(sharpen, 0.0)
    dbl = Default(doublesize, false)
    cpt = Default(cpoint, false)

    return 
    \   (C.IsYV12) ? C._pow2QtYV12(sharpen, doublesize, cpoint)
    \ : (C.IsY8)   ? C._pow2QtY8  (sharpen, doublesize)
    \ :              C._pow2QtRGB (sharpen, doublesize)
    ## Y8   : 25 fps
    ## YV12 : 13 fps
    ## RGB24:  8 fps
    ## RGB32:  7 fps
}
##################################
function _pow2QtYV12(clip C, float "sharpen", bool "doublesize", bool "cpoint")
{
    Assert(C.IsYV12, "_pow2QtYV12: source must be YV12")

    shp = Default(sharpen, 0.3)
    cpt = Default(cpoint, false)
    dbl = Default(doublesize, false)

    C
    wid = Width
    hgt = Height

    ## pass 1: nnedi3 double size, scale back

    Y  = nnedi3_rpow2(rfactor=2, nsize=0, nns=3, qual=2, 
    \                 cshift="Spline64Resize")    
    UV = (cpt) 
    \  ? PointResize(2*wid, 2*hgt, src_top=0.25)
    \  : Spline64Resize(2*wid, 2*hgt, src_left=+0.5, src_top=+0.5)
    YToUV(UV.UToY, UV.VToY, Y.ConvertToY8)

    BicubicResize(wid, hgt, b=-1, c=0, src_left=-0.5, src_top=+0.5)

    ## pass 2: nnedi3 double size, QTGMC de-shimmer, sharpen, scale back

    Y = nnedi3_rpow2(rfactor=2, nsize=0, nns=3, qual=2, 
    \                 cshift="Spline64Resize") 
    UV = (cpt) 
    \  ? PointResize(2*wid, 2*hgt, src_top=0.25)
    \  : Spline64Resize(2*wid, 2*hgt, src_left=-0.5, src_top=-0.5)
    YToUV(UV.UToY, UV.VToY, Y.ConvertToY8)
    
    QTGMC(preset="Medium", InputType=1)
    Sharpen(shp)

    return (dbl) 
    \  ? Last 
    \  : BicubicResize(wid, hgt, b=-1, c=0, src_left=0.5, src_top=-0.5)
}
##################################
function _pow2QtY8(clip C, float "sharpen", bool "doublesize")
{
    Assert(C.IsY8, "_pow2QtY8: source must be Y8")

    shp = Default(sharpen, 0.3)
    dbl = Default(doublesize, false)

    C
    wid = Width
    hgt = Height

    ## pass 1: nnedi3 double size, scale back 

    ConvertToYV12
    ### {
        nnedi3_rpow2(rfactor=2, nsize=0, nns=3, qual=2, 
        \            cshift="Spline64Resize")    

        BicubicResize(wid, hgt, b=-1, c=0) #, src_left=-0.5, src_top=+0.5)

        ## pass 2: nnedi3 double size, QTGMC de-shimmer, sharpen, scale back

        nnedi3_rpow2(rfactor=2, nsize=0, nns=3, qual=2, 
        \             cshift="Spline64Resize") 

        QTGMC(preset="Medium", InputType=1, RepChroma=false)
    ### }
    ConvertToY8
    Sharpen(shp)

    return (dbl) 
    \ ? Last 
    \ : BicubicResize(wid, hgt, b=-1, c=0) #, src_left=0.5, src_top=-0.5)
}
##################################
function _pow2QtRGB(clip C, float "sharpen", bool "doublesize")
{
    Assert(C.IsRGB, "_pow2QtRGB: source must be RGB")

    shp = Default(sharpen, 0.3)
    dbl = Default(doublesize, false)

    return (C.IsRGB32) 
    \ ? MergeARGB(
    \       C.ShowAlpha("Y8")._pow2QtY8(sharpen, doublesize),
    \       C.ShowRed  ("Y8")._pow2QtY8(sharpen, doublesize),
    \       C.ShowGreen("Y8")._pow2QtY8(sharpen, doublesize),
    \       C.ShowBlue ("Y8")._pow2QtY8(sharpen, doublesize)
    \   )
    \ : MergeRGB(
    \       C.ShowRed  ("Y8")._pow2QtY8(sharpen, doublesize),
    \       C.ShowGreen("Y8")._pow2QtY8(sharpen, doublesize),
    \       C.ShowBlue ("Y8")._pow2QtY8(sharpen, doublesize),
    \       pixel_type="RGB24"
    \   )
}

Last edited by raffriff42; 17th March 2017 at 00:11. Reason: (fixed image link)
raffriff42 is offline   Reply With Quote
Old 4th September 2015, 16:56   #19  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,538
Cretindesalpes' eedi3 works pretty fast, give it a try
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 4th September 2015, 17:26   #20  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 319
Code:
function AA_SangNom2(clip in, int "aggression", float "sharpness")
{
    sharp = Default(sharpness, 0.25)
    str = Default(aggression, 48)
    a = in
    Upright = Merge(Sangnom2(in,0,str),Sangnom2(in,1,str))
    Left = in.FTurnLeft()
    Rotated = Merge(Sangnom2(left,0,str),Sangnom2(left,1,str)).FTurnRight()
    Merge(Upright,Rotated)
    return Merge(a,sharp)
}
If you want it quick then sangnom2 or yadif is the way to go, but of course - they're completely spatial.
Code:
ImageSource("a.png")
ConvertToYV24()
AA_Sangnom2(64,0.1)


Even faster would be only applying sangnom2 to the Y plane, and I know for a fact that this is not the first example of sangnom being used for exactly this purpose.

Last edited by Bloax; 4th September 2015 at 17:31.
Bloax is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:55.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.