Anime Codec

[anime_encoder]

Some codec's perform really well with cel shading. Others fail miserably.

In pictures, Gif (which uses a palette) does really well.
JPG fails at anime style shading.

I'm looking for a codec that does really well with cel shading. It's likely it will have a palette. I remember one that would compress anime to 20% but wasn't beneficial, or was even a bad performer when it came to natural scenery.

Ooh... Ogg Theora now has the source of VP3... interesting.

http://forum.doom9.org/archive/index.php/t-11511.html

Does anyone know of any codecs which perform extremely well for large area's of a single color or 'layered' colors?

For instance.... http://www.libpgf.org/ is not what I want.

I remember the codec being mentioned, one which performed really well... but unfortunately, I don't know where. So, I suppose I'll ask here.

[Astrophizz]

Try x264. It's pretty good... and it's palette is YV12...

[Dark Shikari]

Large areas of a single color are easy. The problem is the edges, not the flat areas.

An anime-specific codec would probably have some sort of directional transform along with enhanced H.264-like intra prediction. It would almost certainly have in-frame motion vectors for intra coding.

x264 is obviously the best out there currently, but there's certainly a lot of room to improve.

[benwaggoner]

VC-1 generally does pretty well with cel animation in the current implementations if you use:
Open and Adaptive GOP
I-frame Differential Quantization
In-loop deblocking and overlap transform

At higher bitrates, Differential Quantization on P-frames can be useful, but at lower bitrates the current implementation can wind up hurting texture detail more than it helps smooth detail.

Using a player like WMP that implements adaptive postprocessing can help a bunch as well in ensuring the smooth regions stay smooth.