Anime-only codec idea

[kile]

I watched "Waking Life" last night and the thing that often came to my mind is how bad anime can look in MPEG based codecs. I know that the problem lies in the nature of the MPEG compression which works much better with smooth edges and gradual color change.

But what if there is a way to easily compress and store sharp edged, solid-color surfaces in some vector format? To achieve this the encoder would require to approximate the edges of the surfaces in the bitmap source to some cubic curve, and the decoder would have to quickly draw these edges and flood-fill the surfaces.

I think (at the moment this is only an educated guess) that images stored this way would require less space and would surely have much, much better visual quality then the current MPEG-based compression algorithms. Also, since all the data is stored in a vector format, the output resolution of the decoder and the quality of the rendered image would only be limited by the processing power of the machine the decoder is running on.

However, this "codec" would be only be good for compressing sharp-edged material like old WB cartoons, some japanise anime, and, I think, almost perfect for "Waking Life" type of computer-generated animated material.

At the moment this is only my crazy idea and I would like to know if anyone has thought of something like this before? More importantly: has anyone tried something like this before and failed? Is there something like this already out there? I know that something similar can be found in the ShockWave "cartoons" but not in the form of a video compression codec.

If something like this does not exist, maybe it would be nice to start up a SourceForge project and run some inital tests (size vs speed etc.)? :D

Any comments?

[ChristianHJW]

.. a lot of folks consider wavelet based codecs to be superior for anime IIRC. Have you tried the Rududu codec from nicolas yet ? make a search on the keyword 'rududu' and the link should pop up where you can get it. Its the only 'working' wavelet based codec i know of ..

[kile]

Thanks for the info, but I think that wavelet compression is something very different from my original idea. I want to find the edges of solid-color surfaces and store them as cubic splines.

Maybe I'll make a test during the weekend and see if I can come up with anything...

[ChristianHJW]

You are moving into an 'OpenGL' direction my friend ... not sure if this could still be considered being a 'codec' in the overall sense of it, as you are obviously more trying to 'synthesize' anime ...

[kile]

not sure if this could still be considered being a 'codec' in the overall sense of it, as you are obviously more trying to 'synthesize' anime ...

:D Exactly!

I just wonder if this is possible at all...

[hanfrunz]

hi kile,

i like your idea :)

i think in the coreldraw-package there is a tool which can convert bitmaps to vector-graphics and i also remember that it worked well on VERY simple graphics. so i think it would be possible, but the final fileformat will not be avi anymore but something like flash...

hanfrunz

[kile]

so i think it would be possible, but the final fileformat will not be avi anymore but something like flash...

It's just a matter of packing the data in a stream in the avi file. Each of the consecutive 'frames' will be indexed in the stream and quickly accessed. Putting this in avi wouldn't be a problem at all.

I've read some stuff on approximating these curves. It turns out that in a 1h45m 15fps film it would be possible to store on average around 2500 curves/frame without any additional compression (zip or the like). My guess is that this would be enough to preserve enough details from the original image for a fairly decent quality.

I'm also not sure how fast one image can be 'rendered'. There seem to be a few fast algorithms for drawing the curves during 'decoding'. And they could also be easily optimized using MMX or SSE.

Details about this at:

http://www.tinaja.com/cubic01.asp

[SirDavidGuy]

However, where would you get the source? A DVD will have quant/DCT artifacts, even if they're not visible, and cable will have random noise problems. You're best bet would probably to have a reduction to around 32 colors, and storing any differences in a grid with all zeroes, but some error correction values, which would then be RLL or huffman encoded. For animated material in general, this would probably be optimal.

[kile]

However, where would you get the source? A DVD will have quant/DCT artifacts, even if they're not visible, and cable will have random noise problems.
Some smoothing/filtering will have to be done on the source. I've made a simple test with one frame that came from a not-so-good divx. Even without any filtering and with a very simple flood-fill, I'm able to detect large solid-color surfaces.

You're best bet would probably to have a reduction to around 32 colors
Not at all. For each surface we could use the full color from the source (it's just 24 bits/surface at most). This could either be the average color or the color that most pixels from the surface have.

[SirDavidGuy]

Not at all. For each surface we could use the full color from the source (it's just 24 bits/surface at most). This could either be the average color or the color that most pixels from the surface have

24 bits vs. 5, per texture, with about 100 textures per scene? With huffman or whatever with the error codes, do you really think the compensation will take that huge amounts of space?

[Prettz]

Your idea would probably be the best one for the actual animations, but what are you going to do about the backgounds?

[esby]

I think the 'codec for anime' is utopic.

Why?
Because anime tends more & more to be complex like in real movie.
Of course i'm not talking of the Simpsons, and most of the animated
production of the usa, but of real anime.

a practical exemple:
If you put a look on a serie like arjuna a bit old in my calendar,
some scene have the complexity of the nature, making it deadly for an anime based compressor.

Another example, take a look on Stratos 4 opening sequence...
the same goes...


esby

[MasterYoshidino]

trying to create a codec for anime is not the way to go , even though anime is tough to encode since blockiness in the screen is much less tolerable since anime should look smoothly shaded, rather than look blocky, as "real life" movies are more tolerable at.

a codec that "renders" is defeating the idea of a codec, in fact it would not be dvd archiving but creating pseudo flash movies.

hmmm :D
be nice to see some anime encoded into flash lol

[kile]

@MasterYoshidino
a codec that "renders" is defeating the idea of a codec

Why? The way I see it, anything that stores data in a stream that can be later extracted into a series of bitmaps is a video codec, right?

@Prettz
Your idea would probably be the best one for the actual animations, but what are you going to do about the backgounds?

Yeah :( You are right. I saw a few others films since, and it turns out that the backgrounds would be a really big problem.

@esby
Because anime tends more & more to be complex like in real movie.

That type of anime would be better compressed with a standard MPEG codec.

[Mango Madness]

you really need some high level calculus to find function and curve fitting for the borders and parametric values and so forth. I've always had an idea for a codec that would look at a movie three dimensionally and find large volumes (say the sky background over 3000 frames) and find the volume as it relates to "shapes" and find integrated three dimensional equations to accurately describe the sky over those 3000 frames in a lossless manner. So instead of describing the sky in macroblocks over many frames, you'd have a "simple" equation that when solved would graph the pixels accurately over this spacial and temporal dimension. But that's beyond the scope of anything.

[mf]

This is possible, if you have the original project of the anime :). Anime nowadays is digital, so it's just a bunch of animated layers with a soundtrack, which is later rendered for TV broadcast ;). So if you have the original... ;)

Anyway, a tip for people who only watch anime and never watch real-life material: try setting ffdshow's deblocking strength to the max, it kindof flickers in still image (a bug which i haven't reported yet), but it looks way better with anime. Real-life stuff is blurred to death though :D.