SharpAAMCmod: is it a AA script?

[Chengbin]

I'm currently using SharpAAMCmod to AA some animes. I got a problem, is it a AA filter? Because from comparison pictures, it seems like a line darkener, and no AA.

This is my avs line

sharpaamcmod(aastr=5, ds=0.2, shpre=80, shpost=300, smpost=100, tradius=2, mt=true, pel=1, ov=4, blk=8, aatype="sangnom")

Here are 2 pics.

no AA

http://i27.tinypic.com/2cffthi.png

AA (AA strength 5)

http://i26.tinypic.com/v42hpl.png

I tried adjusting the AA strength from 5 to 10 and 50, but there is no difference.

What am I doing wrong?

[Nightshiver]

Yes, it is AA. Have you looked at the AnimeIVTC docs to see what is included in the function? It uses line darkening, warpsharping, post sharpening and smoothing, sangnorm, eedi2, LSF, etc. Besides this, you have other problems besides AA. Mind posting a short vid clip?

[Chengbin]

I know it does all that, but is that image AAed?

And I'm not using AnimeIVTC, because all the extra functions is eating RAM like a buffet.

Can you tell me what are the other problems in this image (besides needing a strong AA)?

I'll post a video later, uploading a lossless footage is not fun, so I have to encode it.

[Chengbin]

Here is a CRF 10 encode.

http://www.megaupload.com/?d=KLD6A3U9

[Nightshiver]

I never said to use AnimeIVTC. I said to check the docs. And by video sample, I don't mean encoded. Video sample's should either be .vob's or .m2v's.

[Chengbin]

I never said to use AnimeIVTC. I said to check the docs. And by video sample, I don't mean encoded. Video sample's should either be .vob's or .m2v's.

Did you download it? You'll see why I didn't upload the original. I encoded a CRF 10, so it would be small enough, and be very close to the source.

[thewebchat]

Your image has not been antialiased. It has only been mutilated by various sharpening filters that the authors of SAAMCMod thought we should all be cursed with. However, the main reason your image has not been antialiased are:

1) You are using a masked antialiasing script when your source has no details to begin with.
2) You are calling SangNom with an aastr of 5, which is essentially WasteLotsOfCPUCyclesForNoReason()

Applying the traditional combination of DeHalo_alpha and SangNom, I was able to get the following (source is the first screenshot you linked):

http://img197.imageshack.us/img197/1825/antialias.png

[Chengbin]

@thewebchat

Well that's good AA, the image looks really soft.

Am I expecting too much from AA? There is this image comparison at the AnimeIVTC page. While I don't expect quality like that, I expect something better than what I'm getting.

[thewebchat]

You antialias an image by blurring it. That's the plain and simple truth. The only difference between elaborate edge-directed antialiasing scripts and a dumb Blur(1.53) is where the blur is targeted. In your case, there was no "sharpness" in your image to begin with, only halos and aliasing. Like I suggested to begin with, if you do a Resize(last.width/2,last.height/2), your image will actually have real sharpness in it and not just aliasing.

[Didée]

That source is not "true" HD. It is just plain SD that has been upscaled. And that's why the usual AA filters won't (can't) do much good when applied directly to the pseudo-HD footage.

In loose and easy terms: Those filters expect aliasing that is "sharp" on the per-pixel level. But that upscaled stuff doesn't have that; instead, the aliasing has been blurred by the scaling. So, in a sense, the AA filters are looking for something that is not there. Hence, the improvement is little.

You might try downscaling to SD, then upsize again with NNEDI2. There's not much to lose, since SD is all that's in it.

[stpdrgstr]

Did you download it? You'll see why I didn't upload the original. I encoded a CRF 10, so it would be small enough, and be very close to the source.

You don't have to upload the entire video, you know...

That's why it's called a sample.

[Chengbin]

@Didee

The footage is in HD, just that even in HD anime, there is few detail.

@stpdrgstr

So you want the original 20 Mbps video from NTV? There isn't much difference from the source and the CRF 10 encode..

[thewebchat]

Except that it is not in HD and is 100% upscaled if you take the time to stop staring at the stupid "HV" logo and just look at the damn picture.

[Didée]

@Chengbin - if you insist on believing the world is a disk, that's fine with me. But that source is upscaled SD. Absolutely definetly. There's not even room for discussing, since it's blataneously obvious.

You can even measure it out, directly from that screenshot. Looking e.g. at the upper left border of the girl's left sleeve (the diagonal line), it can easily be counted that six of the blurry vertical stripes occupy 16 pixels in horizontal direction. 16 by 6 is 2.666. And surprise, 1920 by 720 is 2.666. See a relation?

[thewebchat]

Chengbin, this is an HD animu. Notice how it doesn't look like your screenshot:

http://img269.imageshack.us/img269/7329/byousoku5centimeter2ccf.png

[Firebird]

@Chengbin
Your video is oblivious upscale. You can downscale and upscale it again and you would not see any difference.

And what about rule №6?

[Chengbin]

@Chengbin
Your video is oblivious upscale. You can downscale and upscale it again and you would not see any difference.

And what about rule №6?

OK, then I guess the producers didn't make this in 1080p resolution (which is very understandable for this kind of anime)

Asking help for recorded material is not breaking rule 6, I specifically asked neuron2 for that.

So what I'm getting from AA is about as good as AA can get? I'm not doing anything wrong right? I just need to know that. I will probably resize it.

[Firebird]

So what I'm getting from AA is about as good as AA can get? I'm not doing anything wrong right? I just need to know that. I will probably resize it.
As i said, you can try downscale it, filter it, then upscale again. That would be at least faster.
So in your case i would do something like

Downscale to 480p
dehalo_alpha(darkstr=0, brightstr=1)
your AA
upscale via nnedi2
awarpsharp2(depth=8)

[Chengbin]

I decided to try this.

This is the script

dfttest()
sharpaamcmod(aastr=5, ds=0.2, shpre=80, shpost=300, smpost=100, tradius=2, mt=true, pel=1, ov=4, blk=8, aatype="sangnom")
aWarpSharp(depth=12,blurlevel=4,thresh=0.3,cm=1)
LimitedSharpenFaster(edgemode=1,strength=100)
deen("a2d",3,6,6)
Dehalo_alpha()

This is the output

http://i30.tinypic.com/nw0n6a.png

As you can see, the lines have 2 lines closely together. Is it possible to remove them and make it a single line like it was supposed to be?

[Nightshiver]

From your strong filtering, you have introduced banding. You need a debander now, Gradfun2db. You do know you are using 3 different sharpeners, right?

LSF
deen
sharpaamcmod has it's own sharpener, which is actually LSF again.

[Chengbin]

From your strong filtering, you have introduced banding. You need a debander now, Gradfun2db. You do know you are using 3 different sharpeners, right?

LSF
deen
sharpaamcmod has it's own sharpener, which is actually LSF again.

Yes, I realize that (BTW, you missed awarpsharp)

I think banding isn't produced by the sharpeners, but the denoiser (dfttest)

[thewebchat]

Spline36Resize(704,480)

There, I just fixed every problem in your source, tripled the quality, and reduced the required bitrate by 6x.

[Nightshiver]

awarpsharp is not a sharpener, it is a line thinner. You think wrong. Over sharpening can cause banding as well as strong denoising. You have both. That makes it worse. Dfttest is an absurdly strong denoiser, you may want to think of toning it down or using a "safer" one, like mdegrain or fft3d with tweaking. Have you also not taken into account the other more experienced scripter's and users here have suggested? Heh and besides, thewebchat is right. Just downsizing the video drastically improves it without any filtering, except it should be Spline36Resize(720,400), as his produces an odd almost 4:3 picture.

[stpdrgstr]

except it should be Spline36Resize(720,400), as his produces an odd almost 4:3 picture.

You'd fix it later with setting the AR flag in the container.

[Chengbin]

Dfttest is an absurdly strong denoiser, you may want to think of toning it down or using a "safer" one, like mdegrain or fft3d with tweaking.

I noticed that as well. I'm probably gonna use MVDegrain 1.

Have you also not taken into account the other more experienced scripter's and users here have suggested? Heh and besides, thewebchat is right. Just downsizing the video drastically improves it without any filtering,

Yes I did, and I plan to do that. I just feel like to do some playing around and see what I can achieve if I don't resize it.

except it should be Spline36Resize(720,400), as his produces an odd almost 4:3 picture.

Yes, that is a 4:3 stretched to 16:9 (that's why I need AA). For some really unknown reason, Japan's 1080 HDTV is 1440x1080 with a 16:9 AR flag(!!!!!!!!!!!!!)

You guys have SERIOUSLY good eyes. How do you see everything? You can be detectives, picking out all those details from just my snapshots.

[BigDid]

Hi,

Interesting exercice. I tried a few things and what seems a good compromise is:
1/ downsize, factor of /2 or more, up to the user
2/ filter, attenuate halos, antialias
3/ upsize, splineresize used but could be nnedi2..
4/ optional, warpsharp or sharp+denoise (not in production) etc..

I agree that a using lower resolution resolves a lot of problems, but it seems the OP absolutely wants full HD viewing :devil:

The picture is step 1 to 3 as in the script below:

setmemorymax(512)
dss2("O:\Films HD Mkv\Anims\test_x264_crf10_HD.MP4", fps=23.976)
###resize
spline36resize(1920/2,1080/2)
###filter
dehalo_alpha(darkstr=0, brightstr=1)
maa()
###resize
spline36resize(1920,1080)
#nnedi2_rpow2(rfactor=2)
#nnedi2_rpow2(rfactor=2,cshift="spline36resize")
###postprocess
#awarpsharp2(depth=8)
#softsharpen(net=8,debr=4,flou=0)

http://i31.tinypic.com/rm77t2.png

[Nightshiver]

You guys have SERIOUSLY good eyes. How do you see everything? You can be detectives, picking out all those details from just my snapshots.

Nah, we've just been doing this for a long time. When you have practice looking for and identifying the same things in different sources, it comes out a little easier.

[Adub]

BigDid,

You got some serious halo issues there. Most likely do to the numerous resizing steps.

[BigDid]

BigDid,

You got some serious halo issues there. Most likely do to the numerous resizing steps.

I agree :D

This is why I have previously stated:
* "2/ filter, attenuate halos, antialias" ...
* "I agree that a using lower resolution resolves a lot of problems, but it seems the OP absolutely wants full HD viewing."

Any strong params/script for halo removing or this particular source you could share?

Did