Anti-aliasing Suggestions?

[Brassplyer]

I'm changing the title to more accurately reflect what I'm looking to accomplish which is antialiasing.

Thanks for all input.

[Groucho2004]

My ultimate goal is to use Sangnom for doing anti-aliasing. The wiki says I need masktools 1.5.8
1. Sangnom is outdated (~ 12 years). Consider Sangnom2 (http://avisynth.nl/index.php/SangNom2)
2. Sangnom(1 or 2) does not depend on masktools
3. What wiki?

[Brassplyer]

1. Sangnom is outdated (~ 12 years). Consider Sangnom2 (http://avisynth.nl/index.php/SangNom2)
2. Sangnom(1 or 2) does not depend on masktools
My wording could have been clearer. The AA page I was looking at said Sangnom and Masktools 1.5.8 are needed for a particular AA plugin.

3. What wiki?
http://avisynth.nl/index.php/AntiAliasing

[Groucho2004]

As I mentioned, these plugins are pretty much obsolete. I'm sure that something like MAA2 (http://avisynth.nl/index.php/MAA2) or similar is much better for this purpose.

[LemMotlow]

That depends. Brassplayer likes to deinterlace and upscale SD to HD and always mentions QTGMC as part of the chain, but SangNom and SangNom2 are deinterlacers anyway that are sometimes used just for anti-alias. Likely you're correct and brassplayer is using the wrong anti-alias plugin if he insists on SangNom. MAA2 or a newer version of Santiag would be less destructive, and there are even more alias fighters. But it depends on the video and what the aliasing looks like.

Otherwise mt_masktools-26.dll and the older masktools.dll can both exist at the same time in Avisynth's plugins, as they have different internal function names.

[Brassplyer]

That depends. Brassplayer likes to deinterlace and upscale SD to HD and always mentions QTGMC as part of the chain, but SangNom and SangNom2 are deinterlacers anyway that are sometimes used just for anti-alias.
My goal is doing anti-aliasing. I ran across a thread that talked about using Sangnom for doing this but I'm not devoted to using it. In fact I'm changing the thread title.

So far not having success with MAA2, I believe I've downloaded all the required components. It tells me the source has to be YV12. I tried converting using ConverToYV12 but per Mediainfo the resulting file is RGB and the script is still choking on it.

FFmpegsource2("W:\sourcefile.avi")
maa2 (mask=1, chroma=false, ss=2.0, aa=48, aac=40, threads=4, show=0)

[raffriff42]

upscaling (feisty2's idea) combined with QTGMC (http://avisynth.nl/index.php/QTGMC) for spatial-temporal AA
http://forum.doom9.org/showthread.php?p=1737201#post1737201

[LemMotlow]

FFmpegsource2("W:\sourcefile.avi")
maa2 (mask=1, chroma=false, ss=2.0, aa=48, aac=40, threads=4, show=0)That's weird. Why are you using ffmpegsource for an AVI? Is it encoded as MPEG or MPEG4? Where is anything that says you're converting to YV12? Is this .avi file interlaced, or what? Why are you setting chroma anti-aliasing strength in MAA2 but using chroma-false? Why are you even bothering to type MAA2's defaults?

AviSource("W:\sourcefile.avi")
ConvertToYV12(interlaced=true) #<-- or false, whatever applies
MAA2()

Are you getting RGB by saving output in VirtualDub using full processing mode and taking VDub's RGB default? No wonder.

[Brassplyer]

That's weird. Why are you using ffmpegsource for an AVI? Is it encoded as MPEG or MPEG4?
I find that if there's audio with the avi file, using Avisource I get glitches and artifacts in the resulting video. I spent a fair amount of time trying to track down why, never could come up with a solution. Included a fresh install of XP. Finally said heck with it, ffmpegsource2 works. I have to either use that or have the audio and video separate. If you have any insights as to why glitches happen only when there's audio included using Avisource I'd be happy to hear possible solutions.

I'm sure your next question is going to be "why are you using XP??" - the answer is because I have a bunch of hardware and software that works smoothly with it and have no interest in introducing more aggravation starting over with a different O/S until and unless I absolutely have to.

Where is anything that says you're converting to YV12? Is this .avi file interlaced, or what?
I used convertToYV12 in a different script, which is also what QTGMC requires and it does the trick there but MAA2 still chokes on the resulting file which apparently still isn't YV12.

Why are you setting chroma anti-aliasing strength in MAA2 but using chroma-false? Why are you even bothering to type MAA2's defaults?
It's the example script on the MAA2 page Avisynth Wiki (http://avisynth.nl/index.php/MAA2). Bear in mind I'd never heard of it until it was suggested in this thread. I assumed it should at least load the file even if the parameters need further tweaking, but so far no success. My experience with Avisynth has been to do some basic tasks as need arises, I'm not at all deeply versed in its use and I'll be the first to admit there are gaping holes in my knowledge of video tech.

AviSource("W:\sourcefile.avi")
ConvertToYV12(interlaced=true) #<-- or false, whatever applies
MAA2()
Thanks, I'll give it a try.

Are you getting RGB by saving output in VirtualDub using full processing mode and taking VDub's RGB default? No wonder.
I'm not sure why I'm getting RGB - see my previous comment re: gaping holes in video tech knowledge. The compression I use is HuffYUV. Used the ConvertToYV12 in a simple two line script that specifies the source file and ConvertToYV12.

FFmpegsource2("W:\sourcefile.avi")
ConvertToYV12()

I assumed that should work since it lets the QTGMC scripts I've used work.

[StainlessS]

AviSource("W:\sourcefile.avi") # Unless problems use AviSource (more stable)
# Info # Uncomment to check that you are already getting YV12, then comment back out again
ConvertToYV12(interlaced=true) # <-- or false, whatever applies. If above Info SAYS YV12, then dont need this line, comment out
MAA2()


Then In VirtualDub, select Video/Fast Recompress and
Video/Compression/ HuffYUV : [Configure : RGB Compression Method=ConvertToYUY2 : YUY2 Compression Method=Predict Left (fastest)]
Or alternatively if UtVideo available select compression as UtVideo (ULY0)
then saveAs

If you do not select Video/Fast Recompress and leave at "Full Processing Mode", it will result in RGB output (HuffYUV will compress as RGB).

[LemMotlow]

I'm sure your next question is going to be "why are you using XP??" - the answer is because I have a bunch of hardware and software that works smoothly with it and have no interest in introducing more aggravation starting over with a different O/S until and unless I absolutely have to.Lots of people use XP, for the same reasons. Windows 8/10 both suck for video work.

I used convertToYV12 in a different script, which is also what QTGMC requires and it does the trick there but MAA2 still chokes on the resulting file which apparently still isn't YV12. Note previous remarks from Stainless on why that's happening and how to prevent it.


It's the example script on the MAA2 page Avisynth Wiki (http://avisynth.nl/index.php/MAA2). Bear in mind I'd never heard of it until it was suggested in this thread. I assumed it should at least load the file even if the parameters need further tweaking, but so far no success. My experience with Avisynth has been to do some basic tasks as need arises, I'm not at all deeply versed in its use and I'll be the first to admit there are gaping holes in my knowledge of video tech. I see. Well, that "example" in the wiki isn't an example script, it's documentation on the several possible parameters that you can set when using the filter, and further details about the default values that work whether you state them explicitly or not.

For example, QTGMC's default preset is "Slow" and Avisynth's default field order is BFF. So typing this:
AssumeBFF()
QTGMC(preset="Slow")
is the same as typing this:
QTGMC()

Using QTGMC, you're also assuming many other parameter defaults including sharpness settings, denoise procedures, and a couple of dozen other parameters. It would take 10 minutes to type out all the QTGMC parameters and their defaults every time you type a script that uses it.

Just thought I would save you some trouble.

[real.finder]

I did some changes in maa2 since long time, manly to make it faster http://forum.doom9.org/showpost.php?p=1725500&postcount=69

and about the topic, it's depends on the source, but generally daa (daa3mod (http://forum.doom9.org/showthread.php?p=1639679#post1639679) and others) family for the IVTC problems, and the AA that use eedi family is useful for bad down scale problems, and so the others AA has their strong points in some source

[manono]

I'm changing the title to more accurately reflect what I'm looking to accomplish which is antialiasing.
That's all very nice but the Forum Rules (http://forum.doom9.org/forum-rules.htm) prohibit asking what's best. I've changed it for you. You are welcome to change it to something else that doesn't ask for what's best.