Log in

View Full Version : I'd like to hear some feedback on a filter design document I made


Chainmax
14th November 2006, 21:44
The main points I'd like to cover are wether this will do its intended work and what would be the easiest/fastest way to make an Avisynth dll filter out of it. Here it is:

Here it is:

-------------------------------------------------------------------------------------------------------------------
Filter name: to be determined

The purpose of this filter is to make a source look like a flash animation, more or less like the following example:

http://img211.imageshack.us/img211/8876/cartcomp8dl.png

This filter works in RGB progressive videos. For other kinds of source, colorspace conversions and SeparateFields().[filter].Weave() or similar methods should be enough.

Operation: a matrix of user-defined dimensions is created around a pixel. Averages of the R, G and B values of every pixel in the matrix are calculated and the resulting "average color" is compared against each color from the palette. The color of the examined pixel will then be replaced to the palette color that's closest to this "averaged color". If no palette is selected, the examined pixel will be replaced by the "averaged color".
As optional postprocessing options, the user will be able to select border highliting and/or HQAA antialiasing.


PARAMETERS:
Palette: can be set to 0 (no palette), 1 (8-color palette), 2(16-color palette).


ML,MR,MU,MD: integer. It indicates how many pixels to the left, right, top and bottom of the examined one will comprise the matrix. For example:

ML=2,MR=3,MU=1,MD=0:
http://img189.imageshack.us/img189/183/mex1db.png (http://imageshack.us)

Defaults: ML=MR=MU=MD=1

If the matrix around a given pixel has some elements that fall outside the frame, they are not considered, for example:
http://img83.imageshack.us/img83/6716/ignex2fp.png (http://imageshack.us)


FM: filling method. Can be set to 1 (only the examined pixel's RGB values are changed) or 2 (all the pixels in the valid matrix's RGB values are changed).

Default: 1


UOF: use original frame (boolean). When operating on a given pixel, if set to true it uses the original frame and if set to false it takes into account the changes made up to then.

Default: true



POSTPROCESSING OPTIONS:

border: integer, 0 to 16 or 0 to 8. It uses one of the colors from the selected palette. If set to 0 or if palette was set to 0 then no border will be created.

Default: 1 (black)

Border creation algorithm: for every pixel, its eight (three for corner pixels and five for border pixels) surrounding neighbors are checked. If any of those surrounding pixels has a different color, then that pixel and the examined one are recolored with the selected border color. For example, with yellow border color:
http://img439.imageshack.us/img439/8217/bex0da.png (http://imageshack.us)

HQAA: a 1X version of Maxim Stepin's magnification filters (http://www.hiend3d.com/hq3x.html) meant to do antialiasing on the result.
-------------------------------------------------------------------------------------------------------------------

Any kind of feedback will be welcome.

jmac698
14th November 2006, 22:51
Hi, it's a great document, I haven't compiled simplesample yet so it may be a while before I implement it. However, does not fxtoon, mftoon, or msu cartoonizer work for you? Note: virtualdub filters can be used in avisynth also.
http://www.compression.ru/video/cartoonizer/index_en.html
http://manao4.free.fr/MaskTools.htm#mftoon
http://flaxen.edwardk.info/

Chainmax
15th November 2006, 00:42
Those tools are great, but none of them do what this filter would be supposed to make:

- MSU Cartoonizer is geared towards making footage look like a painting or a pencil drawing.

- mfToon is designed for animated content. It colors all outlines in black and sharpens the picture. It was superceded by vmToon (which comes with MT_MaskTools).

- I donīt know FXToon, but according to the readme it's a line darkener and sharpener akin to mfToon.

jmac698
15th November 2006, 01:19
Ok, I read again and I see what you mean. But that should be possible in script. You are first blurring (and a generalized matrix filter can be used for this), then you are quantizing to a palette. I don't know any filter to do that offhand. I think masktools lookup command could be used for this. That's the first part I would work on, the rest is easy.
Give me a picture and a palette and I'll try to encode it.
There's other ways to do this also, old freeware was used to make animated gifs to a "websafe" palette. Photoshop can probably do this too, and some video programs use photoshop filters on each image. There's lots of other places to look.

Chainmax
15th November 2006, 03:39
The following image should be a good test of a difficult situation:

http://img82.imageshack.us/img82/4384/donkey1arp750pxcl6.jpg (http://imageshack.us)

As for the palette, I'd say this one (in RGB values) would be ok:

(0,0,0)
(255,255,255)
(128,128,128)
(255,0,0)
(0,255,0)
(0,0,255)
(255,255,0)
(255,0,127)
(150,75,0)
(0,127,255)
(139,0,255)
(255,165,0)
(205,133,63)
(255,253,208)
(48,213,200)
(34,139,34)

We should also define what to use as a measure of distane between colors. A simple solution would be the euclidean distance between the vectors defined by their RGB values. We could also look into CIE's and DIN's definitions.

jmac698
15th November 2006, 04:11
The distance measurement is not so important, you can adjust the palette itself to compensate.

Chainmax
15th November 2006, 04:16
I'm not so sure about that, wouldn't adjusting the palette lead to dithering artifacts?

foxyshadis
15th November 2006, 12:43
If you don't want any dithering effects, you swallow the smallest areas with surrounding larger areas, either until everything's at least a certain size or you're down to a certain number of areas. That's how CelForeground works. In fact, CelForeground fulfills your basic requirements, though it's oh-so-slow, and its design documentation has quite a bit that would be applicable here. Even if you built something that worked in a different way.

krieger2005
15th November 2006, 13:32
http://www.avisynth.org/vcmohan/WaterShed/Watershed.html maybe?

Chainmax
15th November 2006, 14:05
foxyshadis: that's a great recommendation, thanks :). I think I recall you recommending me CelForeground as a basis for this some time ago. Probably CelBackground also has useful starting points since this is intended to work on the whole picture. Both those filters came with sources, right?

krieger2005: that doesn't sound too similar to what I want to achieve, but thanks for the heads-up :).


Everyone: please post more feedback regarding faster / more efficient ways to do what I want to do, and if the 16-color palette is ok or you think other colors should be used.

foxyshadis
15th November 2006, 16:10
Watershed (and ReTouch) tend to use very small areas, CelForground tends toward very large areas, which is usually what you want if you're trying to "paint" something. It'll do borders, too.

CelBackground was unfortunately stillborn, and I don't think it was ever really documented except as a preview. It wasn't made to paint, but rather to stitch together a whole scene's highly detailed backgrounds (to reduce per-frame noise in it).

Unfortunately, no sources, unless Clouded ever sent them to someone else. They'd have been mostly assembly with a little C++ glue, anyway, hardly comprehensible to us mortals. The documentation is pretty good, though.

Get CelForeground, and try out this script, just to see if you like the effect:

source=last
seeds = source.FastGaussBlur(2).LocalMaxima
simplegraph = source.ConstructFloodGraph(seeds, 75, 180)
source.ColourGraphByAverage(simplegraph, source)


75 and 180 are quite strong, play with those to trade off detail vs blobbiness. 25,120 are the recommended defaults for normal cel animation.

Example: This is easily the most complex source I have: Original (http://foxyshadis.slightlydark.com/random/frond.png), and Processed (http://foxyshadis.slightlydark.com/random/frond_anim.png).

Chainmax
15th November 2006, 16:30
That's an amazingly good start :eek:. Could you please try that in the donkey and Pennywise pictures and post the results here?

Wilbert
15th November 2006, 21:50
Unfortunately, no sources, unless Clouded ever sent them to someone else.
He sent me the sources of all his plugins. I promise i will upload them on Friday. If i don't, for some reason, please send me a reminder.

Chainmax
20th November 2006, 06:28
That's an amazingly good start :eek:. Could you please try that in the donkey and Pennywise pictures and post the results here?

Ok then, I'll try to do it myself once I get some free time (too many DVD projets in the queue :)).

Chainmax
20th November 2006, 21:31
He sent me the sources of all his plugins. I promise i will upload them on Friday. If i don't, for some reason, please send me a reminder.

Link to the thread: http://forum.doom9.org/showthread.php?t=118430

Chainmax
24th November 2006, 15:25
I tried this script:

source=last
seeds = source.FastGaussBlur(2).LocalMaxima
simplegraph = source.ConstructFloodGraph(seeds, 100, 210)
source.ColourGraphByAverage(simplegraph, source)

And these are the results:

Pennywise on Segway:
http://img375.imageshack.us/img375/1816/pennysegxn6.png (http://imageshack.us)

Donkey munching on grass:
http://xs309.xs.to/xs309/06475/DonkeyMunch.png (http://xs.to)


As you can see, the result on Pennywise is not what I'm looking to achieve. I think my filter would achieve very similar results to those with Palette=0, ML=MR=MU=MD=1, FM=1, UOF=false, border=0, HQAA=false.


By the way, is this:

function FastGaussBlur(clip clp, float rx, float "ry")
{
ry = default(ry, rx)
rx = rx+1.0
ry = ry+1.0
ox = clp.width
oy = clp.height
oxs1 = int(round(ox/4.0/sqrt(rx))*4)
oys1 = int(round(oy/4.0/sqrt(ry))*4)
oxs2 = int(round(sqrt(ox*oxs1)/4.0)*4)
oys2 = int(round(sqrt(oy*oys1)/4.0)*4)
oxs1 = oxs1<16 ? 16 : oxs1
oys1 = oys1<16 ? 16 : oys1
clp.bicubicresize(oxs1,oys1)
rx>9.0||ry>9.0 ? bicubicresize(oxs2,oys2,0.5,0.25) : last
return bicubicresize(ox,oy,1,0)
}

the FastGaussBlur function?

Chainmax
27th November 2006, 19:22
I'd like to try and create the filter. Taking into account that I'd have to learn from scratch, could you recommend me what to read in order to take this on?

jmac698
27th November 2006, 23:20
yes, simplesample. Search for: Humbly requested, simple filter SDK.
I might try this weekend.

vcmohan
2nd December 2006, 10:21
Happened to see this thread. With my retouch plugin I get the following results.
Frond
http://img154.imageshack.us/img154/659/frondretouch0wz8.jpg (http://imageshack.us)
Donkey:

http://img91.imageshack.us/img91/7513/donkeyretouch0tm3.jpg (http://imageshack.us)

Chainmax
4th December 2006, 20:52
That looks more similar to what I'm looking. Could you post the results on the OP's image?

vcmohan
5th December 2006, 10:53
Below on left is retouch output. Center and right are your images. The script used is:
retouch(separate = false, low = false, seed = false,sh = 20, st = -40)

http://img218.imageshack.us/img218/1728/segwayretouch0wg2.th.jpg (http://img218.imageshack.us/my.php?image=segwayretouch0wg2.jpg)

The results may not be as expected by you. For want of time I did not try many different values for params.

Chainmax
7th December 2006, 06:35
Yeah, ReTouch's output on the OP image is not really similar to what I'm looking for (i.é: image on the right in the OP).