Avisynth filters for anime encoding

[Ramscoop]

1.) I'm encoding anime using h.264 and I was interested in looking into Avisynth filters that could increase the viewing quality of the video. It seems that with animation in general you have large fields where the color is pretty much consistent; it seems there should exist a decent filter for preparing the video for encoding to emphasize the nature of the video source and produce better quality/compression. Does anyone have recommendations/favorites for this situation? (I googled around and was overwhelmed with the amount of information I got back, both recent and dated).

2.) Would this primarily be a pre or post-encoding procedure?

Thanks!

[thewebchat]

It is impossible to "increase the quality of a video".

For processing of cartoons, you can use the same filters you use on everything else and turn the strength up really high.

[Ramscoop]

I didn't literally mean increasing the quality in terms of bit-rate but, just increasing the viewing quality (i.e field/edge smoothing or maybe some sort of soften/blur filter). I'm very new to Avisynth and am curious about popular filters that could help with anime.

[thewebchat]

Bitrate has nothing to do with quality. The point I was making is that filters can only ever reduce quality - they can NEVER increase it.

Try the following:

AddGrainC - increases detail
aWarpSharp - makes things sharp
BiFrost - kills rainbows
Checkmate - kills dotcrawl
Deen - kills mosquito noise
DeHalo_alpha - kills halos
EEDI2 - kills jagged edges
FastLineDarken - puts edges back after smoothing
GradFun2DB - fixes gradients
SSIQ - kills more rainbows
Tweak - makes colors more interesting

[Ramscoop]

One thing I would be very interested in (and the only solution may be to increase the bit-rate) is some sort of post-processing function that can intelligently apply blurring/softening/anti-aliasing (one of em') to the color cells within anime while preserving the borders.

I generally end up with =~ 1500Kbit/s jobs and when I preview the completed video there are artifacts within the color cells as if the encoder is trying to display detail that is not there in the first place.

Either, 1) Blurring/Softening the artifacts out or, 2) Performing some sort of pre-processing on the video source that will allow the encoder to realize, "oh, that's a solid color cell w/ consistent color throughout that doesn't really need very aggressive compression."

[thewebchat]

Example of said artifacts (with source for reference)?

[Ramscoop]

Here's a screen, I'll post a sample of the source when I can.

http://usera.ImageCave.com/eigen_bro/My_Neighbor_Totoro.JPG

Zoom in to see the artifacts in the color cell.

http://usera.ImageCave.com/eigen_bro/My_Neighbor_Totoro-Zoomed.jpg

[thewebchat]

A screenshot of the source would have been just fine. Without it however, this looks like a combination of 1) MPEG-2 garbage and 2) AQ failure. Recommendations: 1) Filter it in Deen first and 2) turn off AQ or at least set it to 0.5 or lower.

[Ramscoop]

Your totally right, mpeg-2 garbage! I just realized I ripped a series of DVD9's to a DVD5 file structure and completely ruined the video source. Dang, gotta redo them all... :P