Can someone port fxtoon?

[digitize]

I would like to request a filter called fxtoon (http://flaxen.dynip.com/vdf/) be ported over from vdub to avs2.5, if it is possible. Im sure alot of anime encoders would appreciate it. Thank you in advance.
url - http://flaxen.dynip.com/vdf/

[Dreassica]

good call!! ^_^
would be nice indeed.

[Kurosu]

1) The author is the first one to contact in this case...
2) Without source-code, such filter's effects could be reproduced, but you would never have the exact same effect.

was: Are you satisfied by warpsharpYV12 for instance ?
is: Compare for instance the effect of the different flavours of warpsharp (Vdub's, Sh0dan's avs2.0x version, some japanese guy's, and the YV12 version).


The first option is really the best, as the filter only works on Y plane (from YIQ colorspace, but that doesn't matter). I bet it needs a lot of rewritting, but natively working with Y plane should help him a lot.

[digitize]

Warpsharp & this filter are very different. The fxtoon filter darken's lines, but does make them a bit thicker, which is why warpsharp is used, to thin the lines a bit. But I will e-mail the author, not too sure why I didn't think of that, thank you.

[Tofusensei]

fxToon is an amazing filter because of its uniqueness. I used it to clean up 33 meg Hajime no Ippo raws way back in the day ^_^

Because of this filter, I was able to smooth at much higher levels than I could have normally, because I was able to "redraw" in the black lines that smoother/cleaners tear up. I would love to see an avs 2.5 version of it.

-Tofu

[mf]

Originally posted by Tofusensei
fxToon is an amazing filter because of its uniqueness. I used it to clean up 33 meg Hajime no Ippo raws way back in the day ^_^
Ouch! I've had one of those one time, damn it was horrible -_-;;.


Anyway, re-writing it for avisynth completely might be better than porting it, the settings are very meager and I can't get it to work on supersampled content properly. I'd need it to recognize the thick grey stripes (I can't call them lines anymore cause at 1920x1440 they're really thick) that xsharpen produces, and maybe recognize the original thickness from the level of luma difference between the stripe and the surrounding pixels, and warp the surrounding pixels in it to thin the line.

Edit: I did a theoretical example of what I'd want the filter to do. You can find it here:
http://mf.onthanet.com/linedarkening/

[Tofusensei]

Cool, looks like it'll be a perfect filter if you can pull it off :)

I'm totally reliant on avs 2.5's yv12 color space now, so it'd rock if it happened. Best of luck!

[Kurosu]

@digitize

Any news? *looks at the attach field*

[digitize]

@Kurosu
Nope, I e-mailed him, but haven't gotten a response. But I like mf's idea, it's a brilliant filter, well with anime at least. But it's just a matter of someone actually writing it. Oh well, I'll continue to check my e-mail and see if he responds.

[Kurosu]

Ok, I have a good deal of thinking about that filter.

The idea of mf is hardly feasible. How to detect those kind of lines? Their thickness totally depends on the supersampling you choose, and you (the filter) may not distinguish them from adjacent flat areas of different luminance. Lines are seen as edges, ie strong variations of luminance. When you run FxToon and look at the map, you'll see it detects the adjacent pixels from different areas (like shadows in anime) as an edge, which isn't good. The distinction is pretty hard.

I've choosen another way, which alas has other drawbacks. My plan is to darken single pixel-wide lines, of the like warpsharp produces. The drawback is that it still darken unwanted parts (quite less). Moreover, line luminance can suddenly decrease, producing an unpleasing 'flash' or discontinuities.

I'll try to link pictures of the results I have so far.

Original source (http://kurosu.inforezo.org/avs/Without.png)
Processed output (only the filter) (http://kurosu.inforezo.org/avs/With.png)

You can see that lines are rather well identified. This can be shown with the Mask (http://kurosu.inforezo.org/avs/Mask.png), where the brighter pixels will be the most darkened (proportionally to their original luminance and a strength parameter, here set to maximum).

The source is rather clean (512x384 xvid encode) and the lines not too much blurred, so that could be have helped a lot. But please notice than the hairs came untouched (although, here and there some faulty pixel was put).

Ah! And another consideration, of speed: the plain C function of the filter takes around 22.1ms per frame (all in all, the speed was around 38-40fps) while the unfiltered source is decoded at around 70fps.

I'm trying to improve the filter, make some mmx optimizations, and I'll release it in the wild.

[mf]

What would be so hard about detecting supersampled xsharpened lines ?
Some common characteristics:


Lines are always a common thickness between (thinnest line thickness) and (thickest line thickness).

Lines will always be between (thinnest line darkness) and (thickest line darkness) darker than the surrounding pixels.

Lines will always be thick stripes.


You can use these characteristics in detecting them, and have configurable thinnest line thickness and darkness, and thickest line thickness and darkness thresholds.
The good part about this, is that the line line thickness (amount of blur) always relates to the line darkness (original line thickness). So, you can always estimate the target thickness and darkness very precisely.

And as an added bonus, you're working on 3 or 4 times the final size. The filter doesn't need to be 100% perfect, as you might have noticed xsharpen makes a lot of weird edges as well, but you don't see that downsized (in fact, those little ragged edges you see supersampled cause that truly blurred things don't really get sharpened, so a gradual blur will look nice).

Anyway, I challenge you to try to make an algorithm that detects the edges in the following picture:

WARNING!! This image is HUGE (4MB), do NOT download if you have dial-up and/or are low on allowed traffic.
http://mf.onthanet.com/naruto-huge.png

Meanwhile I'll see if I can edit it by hand in PSP :D.

[Kurosu]

What would be so hard about detecting supersampled xsharpened lines ?

I'm eagerly waiting for your filter :p

More seriously, the idea of thickness seems too hard to tackle to me. We're slightly leaving the world of pure calculus to join the word of interpretations. It seems I'm the only one interested in coding a filter, so I don't know if other plugin writers have some thoughts about that kind of search.


You can use these characteristics in detecting them, and have configurable thinnest line thickness and darkness, and thickest line thickness and darkness thresholds.

And the in-between thickness/darkness values can be linearly derived from those parameters?
As soon that a negative edge is detected, you have to search between minthin and maxthin for a positive edge. And you have to do that for vertical, horizontal and both diagonal? This sounds awfully slow, and as far as my mmx abilities go, can't really be accelerated...


And as an added bonus, you're working on 3 or 4 times the final size.

Strange, I would have seen it as a huge penalty from the start ;)
Having to supersample a picture to apply a convolution filter sounds a bit weird, or suggest that the filter produces too much garbage.

Trying to translate your words into calculus, I only see problems arising; I'll have a look, but frankly, I don't intend to implement your idea into a working filter.

[mf]

Originally posted by Kurosu
This sounds awfully slow, and as far as my mmx abilities go, can't really be accelerated...

The VDub version of xsharpen isn't optimized either. It runs at 1fps supersampled, so I'm used to slow filtering :D.

Strange, I would have seen it as a huge penalty from the start ;)
Having to supersample a picture to apply a convolution filter sounds a bit weird, or suggest that the filter produces too much garbage.

All in the name of anti-aliasing :). It's a speed penalty, but a quality advantage. Little errors will be averaged and unnoticable.

Trying to translate your words into calculus, I only see problems arising; I'll have a look, but frankly, I don't intend to implement your idea into a working filter.
Hm, that's sad then. I REALLY need a proper cartoon filter, as working with fxToon is annoying me more and more. It just doesn't work properly. The same with warp sharp. It can get some nice results, but it should just apply them only to the lines.

[Kurosu]

The VDub version of xsharpen isn't optimized either. It runs at 1fps supersampled, so I'm used to slow filtering.

I guess the biggest impatc is due to the size of the image, which provokes enormous memory transfers.


All in the name of anti-aliasing.

It's a working solution, as shown by the graphics cards. Yet, if a sharpening filter produces aliasing at the normal resolution, I wonder what the quality of the filter/source is :)


Hm, that's sad then. I REALLY need a proper cartoon filter, as working with fxToon is annoying me more and more. It just doesn't work properly.

The good news: your idea works (well, of course it was going to work, that needed a bit of pseudo-mathematical rephrasing).
See this 4MB png (http://kurosu.inforezo.org/avs/naruto-huge-processed.png) for the output
The 1st bad news: I can't manage to make it work on vertical hunt
The 2nd bad news: it produces garbage, and is increadibly slow (but if you encode at 1fps, you won't mind 3 or 4 more hours of encoding for a 24-minutes-long episode :D)
The worst news: although it works, the practical use seems very little to me because of its slowness and it's bugged. Which means I'll only release the source for someone to take it on when the bugs are fixed, which is not in the near future.

[digitize]

Despite the fact there are artifacts I think this filter will prove quite useful when developed more / debugged. Well done Kurosu.

[mf]

Wait! I was replying to this... Damn, I must have started watching anime, then rebooted... Sorry Kurosu, I'd have replied earlier, but uhm, I didn't finish it. I'll write one again later.

[Kurosu]

Despite the fact there are artifacts I think this filter will prove quite useful when developed more / debugged.

I'd like so, but the idea used so far is a dead-end in my opinion. I'm releasing the source-code now (no plugin compiled in the package, because it isn't intended to be used as such), which means I'm done with it.

I think another possible way to do the job mf expects is to apply fxtoon (or an equivalent) to the source before supersampling (or to a downsampled version of the supersampled version), supersample it to the size of the supersampled version, and use that mask for darkening pixels. Get it here (http://kurosu.inforezo.org/avs/darkthick.zip)

I'm also releasing the filter darkening single-pixel-wide lines. this filter is quite a lot faster. But it also seems unusable, unless used for sharpening, which I don't intend to implement for now. Get it here (http://kurosu.inforezo.org/avs/Lsharpen.zip)

[Popovk]

question :
is this plugin (http://shelob.mordor.net/dgraft/msharpen/msharpen.html) correspond to what you are looking for ?
This plug gived me the best result i had ever get, but D.G. don't seems in a hurry to improve the speed (much slower than xsharpen)

[mf]

Originally posted by Popovk
question :
is this plugin (http://shelob.mordor.net/dgraft/msharpen/msharpen.html) correspond to what you are looking for ?
This plug gived me the best result i had ever get, but D.G. don't seems in a hurry to improve the speed (much slower than xsharpen)
Nope, it's a special unsharp mask filter (which is a contrast-enhancing sharpener).