LSharpAAF - AntiAliasing Combo

[Great Dragon]

v2.00 even doesn't work as it should with default settings

[Overdrive80]

I can not reproduce it.

function maa2(clip c, int "mask", bool "chroma", float "ss", int "aa", int "aac", int "threads", int "show", int "maskt")
{
    chroma = Default(chroma, false)
    mask   = Default(mask, 1)
    maskt   = Default(maskt, 1)
    mtresh = (mask < 0) ? -mask : 7
    show   = Default(show, 0)
    uv     = (chroma) ? 3 : 1
 
    Assert(c.IsY8 || c.IsYV12 || c.IsYV24 || c.IsYV16, "MAA2: Input must be Y8, YV12, YV16 or YV24")
    Assert(0 <= show <= 2, "MAA2: Parameter 'show' must be between 0 and 2")
 
    # create mask
    if (mask != 0) {
        m = (maskt != 1) ? c.mt_edge("min/max", 0, mtresh, 0, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv) : c.mt_edge("sobel", mtresh, mtresh, mtresh-6, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv)
    }
 
    # run sangnom2-based aa
    if (!chroma || show > 0) {
        c_aa = c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads)
    }
    else if (c.IsYV16) {
        c_aa_u = c.UtoY8().Sangnom2AA(ss, aac, threads=threads)
        c_aa_v = c.VtoY8().Sangnom2AA(ss, aac, threads=threads)
        c_aa = YToUV(c_aa_u, c_aa_v, c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads))
    }
    else { c_aa = c.Sangnom2AA(ss, aa, aac, threads) }
 
    # prepare chroma planes for mask overlay
    if (show == 1) {
        c_aa = (c.IsY8) ? c.ConvertToYV12().mt_lut(y=2, u=0, v=0)
                      \ : c.mt_lut("x 2 /", y=2, u=3, v=3)
    }
    else if (show == 2) {
        c_aa = (c.IsY8) ? c_aa.ConvertToYV12().mt_lut(y=2, u=0, v=0)
                      \ : YtoUV(c.UtoY8(), c.VtoY8(), c_aa).mt_lut("x 2 /", y=2, u=3, v=3)
    }
 
    # merge aa'ed lines into source
    if (mask == 0) {
        return mt_logic(c_aa, "and", y=4, u=2, v=2)
    }
    else if (show > 0) {
        if (c.IsYV16) {
            m_uv = BilinearResize(m, m.width/2, m.height)
            return mt_merge(c, c_aa, YtoUV(m_uv, m_uv, m), u=3, v=3)
        }
        else {
            return (c.IsYV24) ? mt_merge(c, c_aa, m.YtoUV(m,m), u=3, v=3)
                            \ : mt_merge(c.ConvertToYV12(), c_aa, m, u=3, v=3, luma=true)
        }
    }
    else {
        return (chroma) ? mt_merge(c, c_aa, m, u=3, v=3)
                      \ : mt_merge(c, c_aa, m, u=2, v=2)
    }
}

This function not use avisynth language, I think.

EDIT: I am using avisynth 2.6 and I dont get error with my source that is yv12 (if use converttoyv24 or converttoy8 get error because lsfmod is limited yv12 colourspaces but not for maa2).

Is possible that you didn't remove dll of removegrain, repair, etc... from pack of kassandro?? Please, use plugins listed in dependencies http://forum.doom9.org/showthread.ph...77#post1716077 and remove old plugins.

I use maa2 http://avisynth.nl/index.php/MAA2

[Great Dragon]

I'm using Avisynth+ but I'll check plugins version. However I'm pretty sure they're up to date.

UPDATE:
I'm using 0.41mod version of maa2 http://pastebin.com/fqR4CMbV
There is also 0.42+ mod http://pastebin.com/F0GLtrx5 I guess it's made specially for Avisynth+ however 0.41mod work fine for me too.

UPDATE2:
@Overdrive80 can you please check if those mods works for you with Avisynth 2.6 ?

[Overdrive80]

Originally Posted by Great Dragon

@Overdrive80 can you please check if those mods works for you with Avisynth 2.6 ?

Ok, but I have convert sintaxis for maa2 on avisynth from your avisynth+ mod.

[Overdrive80]

Ok, with code below of maa2 0.42+ runs fine on avisynth 2.6:

/*
MAA2MOD v0.42 
=========
 
Updated version of the MAA antialising script from AnimeIVTC.
MAA2 uses tp7's SangNom2, which provide a nice speedup for SangNom-based antialiasing,
especially when only processing the luma plane.
The defaults of MAA2 match up with MAA, so you'll get identical output (save for the more accurate border region processing of SangNom2)
when using this script as a drop-in replacement.
 
MAA2 supports Y8, YV12, YV16 and YV24 input.
 
Requirements:
 
   * AviSynth
   * SangNom2 0.3+
   * Masktools 2.0b1
 
Parameters:
 
   + [int] mask (1)
       *   0: Disable masking
       *   1: Enable masking
       *  -i: Enable masking with custom treshold
   + [bool] chroma (false)
       *   false: Don't process chroma channels (copy UV from the source clip if present)
       *   true: Process chroma channels
   + [float] ss (2.0)
       *   Supersampling factor (sensible values are between 2.0 and 4.0 for HD content)
   + [int] aa (48)
       *   Sangnom2 luma antialiasing strength
   + [int] aac (aa-8)
       *   Sangnom2 chroma antialiasing strength
   + [int] threads (4)
       *   Number of threads used by every Sangnom2 instance
   + [int] show (0)
       *   0: Don't overlay mask
       *   1: Overlay mask only
       *   2: Overlay mask and run antialiasing on the luma plane
   + [int] maskt (1)
       *   1: sobel
       *   2: min/max
 
*/
 
function maa2mod(clip c, int "mask", bool "chroma", float "ss", int "aa", int "aac", int "threads", int "show", int "maskt")
{
    chroma = Default(chroma, false)
    mask   = Default(mask, 1)
    maskt  = Default(maskt, 1)
    mtresh = (mask < 0) ? -mask : 7
    show   = Default(show, 0)
    uv     = (chroma) ? 3 : 1

    Assert(c.IsY8 || c.IsYV12 || c.IsYV24 || c.IsYV16, "MAA2: Input must be Y8, YV12, YV16 or YV24")
    Assert(0 <= show <= 2, "MAA2: Parameter 'show' must be between 0 and 2")

    # create mask
    m = mask  != 0 ? c.mt_edge("sobel", mtresh, mtresh, mtresh-6, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv) : \
        maskt != 1 ? c.mt_edge("min/max", 0, mtresh, 0, mtresh-6, u=uv, v=uv).mt_inflate(u=uv, v=uv)           : nop

    # run sangnom2-based aa
    c_aa_u = (c.IsYV16) ? c.UtoY8().Sangnom2AA(ss, aac, threads=threads) : nop 
    c_aa_v = (c.IsYV16) ? c.VtoY8().Sangnom2AA(ss, aac, threads=threads) : nop 
 
    c_aa = (!chroma || show > 0) ? c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads)                         : \
                      (c.IsYV16) ? YToUV(c_aa_u, c_aa_v, c.ConvertToY8().Sangnom2AA(ss, aa, threads=threads))  : c.Sangnom2AA(ss, aa, aac, threads)
 
    # prepare chroma planes for mask overlay
    c_aa = (Show == 1 && !c.IsY8)  ? c.mt_lut("x 2 /", y=2, u=3, v=3)                                  : \
           (Show == 1 &&  c.IsY8)  ? c.ConvertToYV12().mt_lut(y=2, u=0, v=0)                           : \
           (show == 2 && !c.IsY8)  ? YtoUV(c.UtoY8(), c.VtoY8(), c_aa).mt_lut("x 2 /", y=2, u=3, v=3)  : \
           (show == 2 &&  c.IsY8)  ? c.ConvertToYV12().mt_lut(y=2, u=0, v=0)                           : c_aa
	
    # merge aa'ed lines into source
    m_uv = (c.IsYV16)  ? BilinearResize(m, m.width/2, m.height) : nop

    last = (mask == 0)                          ? mt_logic(c_aa, "and", y=4, u=2, v=2)                      : \
           (mask != 0 && show > 0 && c.isYV16)  ? mt_merge(c, c_aa, YtoUV(m_uv, m_uv, m), u=3, v=3)         : \
           (mask != 0 && show > 0 && c.IsYV24)  ? mt_merge(c, c_aa, m.YtoUV(m,m), u=3, v=3)                 : \
           (mask != 0 && show > 0)              ? mt_merge(c.ConvertToYV12(), c_aa, m, u=3, v=3, luma=true) : \
           (mask != 0 && show == 0 && chroma)   ? mt_merge(c, c_aa, m, u=3, v=3)                            : mt_merge(c, c_aa, m, u=2, v=2)

   return last		       
}




function Sangnom2AA(clip c, float "ss", int "aa", int "aac", int "threads")
{
    threads = Default(threads, 4)
    aa = Default(aa, 48)
    aac = Default(aac, aa-8)
    aac = (aac < 0) ? 0 : aac
    ss = Default(ss, 2.0)
    ss_w = int(round(c.width*ss/4.0)*4)
    ss_h = int(round(c.height*ss/4.0)*4)
 
    Assert(ss > 0, "MAA2: Supersampling factor must be > 0")
 
    return c.Spline36Resize(ss_w, ss_h).TurnLeft() \
            .SangNom2(threads=threads, aa=aa, aac=aac).TurnRight().SangNom2(threads=threads, aa=aa, aac=aac).Spline36Resize(c.width, c.height)
}

Please, check if all code is fine for you, I am not tested too much.

Note: I rename function to maa2mod

[Great Dragon]

It works for me and produces identical result compare to original maa2+ 0.42 mod

[Overdrive80]

Ok, I have incluided in LSharpAAF and runs fine on avisynth 2.6 (maybe problem with avisynth+??), I dont get error for yv12 input.

https://dl.dropboxusercontent.com/u/...arpAAF2.01.zip

[Great Dragon]

No errors this time but the script still does nothing. Picture remains the same no matter what aatype I choose.
So maybe it has some Avisynth+ compatibility issue.

[real.finder]

Originally Posted by Great Dragon

No errors this time but the script still does nothing. Picture remains the same no matter what aatype I choose.
So maybe it has some Avisynth+ compatibility issue.

this will work in both avs and avs+

MAA2 v0.432 mod https://pastebin.com/8qz1kUbM

try it with

maa2(mask=-100, maskt=2, ss=3)

ss=3 is good for noisy source

[kuchikirukia]

If I rewrite my Santiag v1.6 to use Resize8 for the frame shift instead of the internal Spline36Resize, will it avoid the chroma shift bug in the internal resizers? Or is that already taken care of in the frame shift?

[cretindesalpes]

Originally Posted by kuchikirukia

If I rewrite my Santiag v1.6 to use Resize8 for the frame shift instead of the internal Spline36Resize, will it avoid the chroma shift bug in the internal resizers? Or is that already taken care of in the frame shift?

Santiag doesn’t take any chroma shift bug into account. But you don’t need to rewrite it to use your own scaler. Just provide it as scaler_post argument in the call (it should have the same prototype as the standard avisynth scalers), or change the default value.

I’ll check how chroma location issues affect the final resizing and will publish a fix if required.

[killerteengohan]

Hey cretindesalpes, why is it when I try to use the Santiag v1.6 you posted "aa=24" parameter I get this error every time.

(The named argument "aa" was passed more than once to santiag_dir)
santiagv1.6.avsi, line 30

[cretindesalpes]

Thanks, typo fixed.