New anime filter: mfToon

[mf]

Ok, so some of you might have heard of this by reading the AVISynth development forum, however not all of you do, so I'm posting here, now that I reckon I won't be doing much about it anymore.

I have mfToon v0.32 ready, and I think it's good enough for normal usage now. You can follow development from v0.1 onward here (http://forum.doom9.org/showthread.php?s=&threadid=52066).

The script (it's a function, not a real filter dll) can be found here (http://mf.onthanet.com/linedarkening/mfToon-v0.32.avs).

What does it do: in default operation, it performs line darkening, Xsharpening, and warp sharpening. Sharpening is slow, and because of that I've made it optional.
Because of colorspace conversions, most of the operations are done on luma only. Warp sharpening can be done either on luma only or on luma and chroma (chroma linked to luma). Input requires to be YV12, this is not checked, it's probably safe to use other colorspaces but the output will always be YV12. I'm not responsible for what happens if you use something other than YV12 :D.

Usage and parameters:

Import("mfToon-v0.32.avs")
AVISource/MPEG2Source/WhateverSource("file")
mfToon()


Normal parameters:
type param - description - range - default

int strength - line darkening strength - 0-255 - 255
bool sharpen - sharpening - on/off - true
bool cwarp - chroma warp - on/off - true
float wdepth - warping depth - ?-? - 16.0
int wblur - warping blur level - ?-? - 2
float wthresh - warping threshold - 0.0-1.0 - 0.5
int ssw - supersample factor horizontally - 0-inf - 4
int ssh - supersample factor vertically - 0-inf - 4

Advanced parameters, you're not likely to be tweaking these:
type param - description - range - default

int xstren - xsharpening strength - 0-255 - 255
int xthresh - xsharpening threshold - 0-255 - 255
float ublur - unsharp mask blur level - 0.0-1.58 - 1.58
int ustren - unsharp mask strength - 0-255
int urange - unsharp mask range - 0-255 - 115
float uboost - unsharp mask boost - 0.0-10.0 - 1.0
int dstren - detail strength - 0-255 - 255
int drange - detail range - 0-255 - 64
float dboost - detail boost - 0.0-10.0 - 10.0
int dlimit - detail limiter - 0-255 - 30

Explanation of advanced parameters: mfToon uses masks, and these parameters influence them. Strength means opacity of the effect, range means the value compression, boost means an opaque/transparency boost, where a value below 1.0 is negative, and limiter means the final range compression.

Yeah, these are quite some parameters, but I love configurability :D.

Initially I wanted to hide the drawbacks, and let you figure them out (they're hardly noticable), but nahh :devil:.
Drawbacks: Sometimes a strange "dark edge" effect on seemingly low-detail surfaces, and a VERY slight darkening along non-line edges (this is the effect you get from sharpeners ala MSharpen and aSharp, so you probably won't notice it).

Dependencies are:
Special(!) Warp Sharp package (http://mf.onthanet.com/linedarkening/WarpSharp.dll)
MaskTools (http://kurosu.inforezo.org/avs/MaskTools.zip)
aWarpSharp (http://ziquash.chez.tiscali.fr/aWarpSharp%20beta%201.zip)

Thanks go to:
Anonymous author of the new WarpSharp package (http://www.geocities.co.jp/SiliconValley-PaloAlto/2382/)
neuron2 (for the original XSharpen) (http://shelob.mordor.net/dgraft/)
Kurosu (http://kurosu.inforezo.org/avs/)
Marc FD (http://ziquash.chez.tiscali.fr)
The whole forum and its users :) (http://forum.doom9.org)

Have fun!

PS: Just for fun, here's a nice challenge. I just noticed I accidentally made one obsolete parameter. See if you can spot it ;).

[digitize]

For all of those who have not tried this filter.... well you're missing out ^^. It is directed towards anime... not sure how and if it'd really work with live action stuff. Anyway, it is able to darken lines and sharpen the picture very well. The one drawback atm is the speed, but speed isn't of the largest concern, quality is, and that's where this filter does well. Well done mf :p

[jarthel]

another drawback: slow even on my overclocked system :( but quality is good. :)

[zeus163]

(OK, I just re-read some of the txt files and it appears that I need to install 2.51). I'll try that tomorrow and get back to you.
Does this work with 2.08? I'm trying to use it and it errors out every time I pull it into CCE. It doesn't say an error, just a picture with an exclamation error. I just tried to open the script in VDub and it states there is a script error: There is no function named Edge Mask.I took all the downloadable files from this page and just can't seem to get them set up right. I must be doing something wrong. My sample script is below.

Import("C:\Program Files\AviSynth2\plugins\mfToon-v0.32.avs")
AVISource("F:\anime\naruto episodes\naruto test.avi")
mfToon()
BicubicResize(704,440)
AddBorders(8,20,8,20)
ResampleAudio(44100)

I can't wait to try this. It looks cool. Especially since I've been converting Naruto fansubs to DVD!

[digitize]

@zeus
You need to load all the dll's that you downloaded:
awarpsharp
warpsharp
MaskTools

Load all of those into the avs, and it should work.

[dvd2svcd]

Here's a funny thing. If you put the 3 above plugins into the Avisynth 2.51 Plugin folder, dvd2svcd fails when using the native VFW routines of Windows (used by preview and frameselection). I haven't gone into depth with the issue, but if anybody cares I can investigate further.

[Wilbert]

Here's a funny thing. If you put the 3 above plugins into the Avisynth 2.51 Plugin folder, dvd2svcd fails when using the native VFW routines of Windows (used by preview and frameselection). I haven't gone into depth with the issue, but if anybody cares I can investigate further.
Problem is already known (and described in both AviSynth faq's). The problem is warpsharp.dll, and the solution is to put it _not_ in the AviSynth plugindir, but somewhere else.

[dvd2svcd]

Sorry, my bad. Note to self "use the freaking search button"

However, I find it funny that mf apparently doesn't need to load warpsharp manually. (if you look in the mfToon function it isn't loaded anywhere and therefore mf should get the same problem when previewing an avs in MediaPlayer 6.4)

[mf]

Originally posted by dvd2svcd
Sorry, my bad. Note to self "use the freaking search button"

However, I find it funny that mf apparently doesn't need to load warpsharp manually. (if you look in the mfToon function it isn't loaded anywhere and therefore mf should get the same problem when previewing an avs in MediaPlayer 6.4)
Actually, I load them before I do Import() :D.

[Defiler]

I didn't need to explicitly load any plugins to make this work. Thanks for the script.
I'm encoding a 90 minute anime to one CD right now. I'll reply with my results if they are interesting. I'm also using the latest XviD features, like Trellis, etc.. so it will be a while. Heh.

[mf]

So, nobody spotted the obsolete parameter yet? ;)

[Sorrow]

I'll take a shot... is the obsolete parm

int urange - unsharp mask range - 0-255 - 115


hehe wild guess here. ;)

[Defiler]

That's my guess as well.

[mf]

Originally posted by Sorrow
int urange - unsharp mask range - 0-255 - 115
Nope :).

[Defiler]

I give up. I just read through the script like 10 times, and every defined parameter seems like it is used in a function that actually passes data to one of the three clips that is used for the final output. 18 "independent" parameters, 18 are defined, 18 are documented.

Can we get a hint? :)

[zeus163]

I loaded all the plug-in's into my script and that did not solve the problem. Although the error it reported was something like screen size not accepted. I'm at work so don't remember what it says. I think I have to install avisynth 2.51, but whenever I do that I seem to have problems modifying my working scripts for fielddeinterlace (i think it's something to do with mpeg2dec that I'm trying to use). When I get home, I'll try this again and upgrade to 2.51 and see if I can get it to work.

[Defiler]

zeus: You did read this, right?
Input required to be YV12, this is not checked, it's probably safe to use other colorspaces but the output will always be YV12. I'm not responsible for what happens if you use something other than YV12

[Mr_Khyron]

zeus163: Try the new MPEG2Dec3 v1.04 (http://forum.doom9.org/showthread.php?s=&threadid=53164) and AviSynth 2.5.1 RC 3.1 (http://cultact-server.novi.dk/kpo/avisynth/avs_cvs.html)

[Dreassica]

detail limiter - 0-255 - 30
maybe? :P

[mf]

Originally posted by Defiler
I give up. I just read through the script like 10 times, and every defined parameter seems like it is used in a function that actually passes data to one of the three clips that is used for the final output. 18 "independent" parameters, 18 are defined, 18 are documented.

Can we get a hint? :)
Well actually, there are two parameters that do the same, and you can use either one of them and get the same effect :).

Originally posted by Dreassica
detail limiter - 0-255 - 30
maybe? :P
Nope :).

[Defiler]

Are they dboost and dstren? They seem to do essentially the same thing, one right after the other.

[WyldeF1r3]

I personally don't like the thinning of the lines in my anime so i try to minimize all the warping settings as much as possible, but this is definately a useful "filter".

I'm not much of a programmer, and I honest don't feel like testing to see if my guesses on redundancy are correct, but to me it seems like
detail range and detail limiter are somewhat redundant, despite having a different range of perameter values. But also wondering, will cwarp work if you omit cwarp2 (does it default to 0/1 thus enable it's typing to be redundant, perhaps making that your "obsolete")

cwarp = Default(cwarp, true) # chroma warp on/off
cwarp2 = cwarp ? 1 : 0
aWarpSharp(cm=cwarp2

you also don't ask cwarp2 to be identified in mftoon() only cwarp, yet i don't see cwarp anywhere, only cwarp2

[mf]

Originally posted by Defiler
Are they dboost and dstren? They seem to do essentially the same thing, one right after the other.
You're getting close, but no.

Originally posted by WyldeF1r3
I personally don't like the thinning of the lines in my anime so i try to minimize all the warping settings as much as possible, but this is definately a useful "filter".

I'm not much of a programmer, and I honest don't feel like testing to see if my guesses on redundancy are correct, but to me it seems like
detail range and detail limiter are somewhat redundant, despite having a different range of perameter values. But also wondering, will cwarp work if you omit cwarp2 (does it default to 0/1 thus enable it's typing to be redundant, perhaps making that your "obsolete")

cwarp = Default(cwarp, true) # chroma warp on/off
cwarp2 = cwarp ? 1 : 0
aWarpSharp(cm=cwarp2

you also don't ask cwarp2 to be identified in mftoon() only cwarp, yet i don't see cwarp anywhere, only cwarp2
About the thinning: line darkening thickens the lines. Warping just puts them back at their original thickness.
And about the cwarp: nope :). cwarp is boolean, and the line you pasted converts the boolean into int. I did this because I originally didn't know there were 3 values for aWarpSharp's "cm" :o. But I left it in there cause cm=2 will usually be more crappy than cm=1 anyway.

[MasterYoshidino]

The thing that is unacceptable in this filter is the overall darkening of the picture, as MPEG4 tends to overcompress dark colors that are still noticeable.
It also would not be a useable one since the only accurate way to use a contrast filter is before deinterlace/decomb. Since this filter will interpolate lines, there is no way to use it there, so it is not a useable filter :rolleyes:

You can do all the sharpening you want, but as soon as I saw the bicubic resize in the script = interpolation = tough to decomb

[mf]

Originally posted by MasterYoshidino
The thing that is unacceptable in this filter is the overall darkening of the picture, as MPEG4 tends to overcompress dark colors that are still noticeable.
It also would not be a useable one since the only accurate way to use a contrast filter is before deinterlace/decomb. Since this filter will interpolate lines, there is no way to use it there, so it is not a useable filter :rolleyes:

You can do all the sharpening you want, but as soon as I saw the bicubic resize in the script = interpolation = tough to decomb
Maybe you should do some more research.
A) It doesn't darken the whole picture anymore. That was fixed in v0.30.
B) What you say only applies to lumi-masking.
C) You can simply turn off the interpolated sharpening, just do sharpen=false.
D) You should never use these kind of filters on interlaced material, except if it has support for it.

[Kurosu]

1) People tend to overlook your strength at hiding this obsolete parameter while looking for it, isn't it ? :D
2) Levels(0, 1, 255, 255, 0) = Invert(Y=3,U=2,V=2) from my filters's package, AFAIK. But it might be slower...
3) There could be smarter edge detectors than "sobel", especially if they only detect dark lines. But that's would be of little help anyway.

[mf]

Originally posted by Kurosu
1) People tend to overlook your strength at hiding this obsolete parameter while looking for it, isn't it ? :D

Even at the last moment, seconds before uploading it to my webspace, I had to look twice to notice it, and it had been there all that time :D.

2) Levels(0, 1, 255, 255, 0) = Invert(Y=3,U=2,V=2) from my filters's package, AFAIK. But it might be slower...
3) There could be smarter edge detectors than "sobel", especially if they only detect dark lines. But that's would be of little help anyway.
Invert says it only runs on YV12, and I found sobel to work the best for me.

[Kurosu]

Invert says it only runs on YV12

Nevermind, then; I didn't check the colorspace.

[MasterYoshidino]

:) looks allrite with the newer versions. (did not look around lol)
but I could always do a chain SmartSmooth to emphasize edges. ;)
SmoothHiQ(5,8,16,256,6).SmoothHiQ(5,8,16,256,6).SmoothHiQ(3,8,16,256,4).SmoothHiQ(3,8,16,256,4)

lol, makes them ****** PinkPineapple DVD's look nice and focused
:p

[mf]

Originally posted by MasterYoshidino
:) looks allrite with the newer versions. (did not look around lol)
but I could always do a chain SmartSmooth to emphasize edges. ;)
SmoothHiQ(5,8,16,256,6).SmoothHiQ(5,8,16,256,6).SmoothHiQ(3,8,16,256,4).SmoothHiQ(3,8,16,256,4)

lol, makes them ****** PinkPineapple DVD's look nice and focused
:p
Nice, however clearly beside the point. mfToon darkens lines. SSHiQ doesn't.

[BoNz1]

Just a quick question for you mf or actually maybe not so quick. Ok, so I have been trying to modify mftoon to work for a dvdrip but I can only seem to get it to work one way. Here is my script:

LoadPlugin("C:\Video\Avisynth Plugins\MPEG2Dec3.dll")
LoadPlugin("C:\Video\Avisynth Plugins\Decomb.dll")
Import("C:\Video\Avisynth Plugins\mfToon-v0.32.avs")

MPEG2Source("C:\DVD\VIDEO_TS\DVD.d2v",idct=5,cpu=2)
Telecide(post=false).Decimate(5).FieldDeinterlace(full=false)
crop(12,2,704,476)
mfToon(sharpen=false,cwarp=false,ssw=3,ssh=3)
BicubicResize(640,480,0,0.5)

But, if I do it this way this means that I am resizing three times, first to supersample it and then to resize again to do mergeluma() and then again after mfToon(), I can get mfToon to work like this if I eliminate mergeluma() but I would like use it and also only resize it twice. Is there any way that you can think of to do this? I have been trying for quite a while now and I can't seem to figure it out. I don't think there will be any loss in quality by resizing three times like this but definitely it will slow it down. I guess, the real problem is that the frame size for mergeluma() must be the same so if I change orig.width and orig.height to 640 and 480 respectively everywhere obviously it doesn't work, I guess I could also resize to 640x480 before the first mergeluma() but then I still have to resize three times. Maybe you or someone else can think of a way around this since I can't seem to. :)
EDIT: Don't bother actually, I made my own script and stole some of the routines you used in mfToon, seems just as good since the source is fairly clean plus it is a lot faster ;)

[zeus163]

Cool. Now that I have Avisynth 2.5.1 working on my machine, I want to play around with this. I created a script that CCE accepted, but when I hit decode it thinks about it for about 5 seconds and then it thinks it's done. Of course, nothing has been produced except a small file that doesn't amount to anything.

I noticed that somebody had shown some screen shots of Naruto. I wonder if the original source files were .avi files (fansubbed divx files). Does mftoon work with these type of files?

I'm sure the answer is right in front of me and I'm missing something.

thanks.

[mf]

YAY! Feedback! ^__^

Bonzi: You have sharpen=false set, so the resizing routines are left alone. mfToon isn't resizing with sharpen=false set. So ssw and ssh have no influence either. And warping isn't done either, so cwarp is ignored too, in this mode.

Zeus: I don't know for sure, but I think CCE only eats YUY2. Try putting ConvertToYUY2() at the end of your script.

[zeus163]

@mf

adding the convert worked like a charm. I never had to do that with 2.0.8 and CCE. I just modified my 2.0.8 (simple .avi script) and had to add the convert to get that to work it. So, that must be it!

thanks.

[digitize]

@zeus163
You never had to do that with 2.08 b/c 2.08 works in yuy2 color space, 2.5 is in yv12.

[zeus163]

Ahh....Thanks. I appreciate you letting me know that.

I encoded a 1 minute segment and it took about 25 minutes (just did a CBR) and will burn it to a DVD+RW and check it out later on the TV to see if I like the results. If they are promising, then I don't mind the extra time involved.

Thanks for all your answers and help! I appreciate it.

[mf]

I appreciate you guys' feedback too. Thanks everyone.

[digitize]

@zeus163 No problem.
@mf Thanks for the filter ^^.

[mf]

Originally posted by digitize
@mf Thanks for the filter ^^.
Yeahyeah I know, you want that new version huh :p.

[crusty]

Import("mfToon-v0.32.avs")
AVISource/MPEG2Source/WhateverSource("file")
mfToon()

I like the 'WhateverSource' part....can we have a function like that in avisynth please?:D :D :D

[onisama]

When I try to run this script I get an avisynth open failure message. It says awarpsharp is not an avisynth 1.0 plugin, I am using avisynth 2.07.

Here is my script:

LoadPlugin("C:\Program Files\DVD2SVCD\Avisynth2 Plugins\awarpsharp\aWarpSharp.dll")
LoadPlugin("C:\Program Files\DVD2SVCD\Avisynth2 Plugins\masktools\MaskTools.dll")
LoadPlugin("C:\Program Files\DVD2SVCD\Avisynth2 Plugins\WarpSharp.dll")
Import("C:\DOCUME~1\BadAss\Desktop\dvdtemp\BATTLE~1\ep7\RESAMP~1.AVS")
Import("mfToon-v0.32.avs")
LoadPlugin("C:\PROGRA~1\DVD2SVCD\AVISYN~1\Mpeg2dec\MPEG2D~1.DLL")
UnalignedSplice(mpeg2source("C:\Documents and Settings\BadAss\Desktop\dvdtemp\Battle Athletes\ep7\New Folder\d2v\intro.D2V"),mpeg2source("C:\Documents and Settings\BadAss\Desktop\dvdtemp\Battle Athletes\ep7\New Folder\d2v\ep7.D2V"),mpeg2source("C:\Documents and Settings\BadAss\Desktop\dvdtemp\Battle Athletes\ep7\New Folder\d2v\ending.D2V"),mpeg2source("C:\Documents and Settings\BadAss\Desktop\dvdtemp\Battle Athletes\ep7\New Folder\d2v\preview.D2V"))
LoadPlugin("C:\Program Files\DVD2SVCD\InverseTelecine\Donald Graft\Decomb.dll")
Telecide(guide=1,gthresh=50,chroma=true,threshold=30)
Decimate(cycle=5)
mfToon()
BicubicResize(480,480,0.0,0.6)
ResampleAudio(44100)

[lamer_de]

aWarpSharp - a WarpSharpenning filter for Avisynth 2.5 - beta 1
Update to avisynth 2.5.x and it will work.

CU,
lamer_de

[mf]

Originally posted by onisama
LoadPlugin("C:\Program Files\DVD2SVCD\InverseTelecine\Donald Graft\Decomb.dll")Eeks! And there I was thinking I was typing much with my LoadPlugin("D:\Video\AVISynth\AVISynth\Filter.dll") lines - you sort them to category and author! ~__~

[kxy]

I found a problem.
The difference in the ascript is just mftoon()

Note, the screenshot is cropped.


Original (http://web.ics.purdue.edu/~yaok/ori.PNG)

MfToon (http://web.ics.purdue.edu/~yaok/mftoon.PNG)

[Kurosu]

Play a bit with the thresholds, or rather, clean your source *first* with a spatial denoiser (deen, convolution3D and mipsmooth come to mind), in order not to sharpen/filter the compression noise. I see as first purpose for mftoon to sharpen edges that are already sharp. Furthermore, if you don't see it when it's played at (was: in) realtime speed, then it's not there (unless you want to squeeze some more compressibility).

Note: it may be a filter bug (not really in mftoon), but still, I think the result would more representative if you applied the much needed spatial smoother.

[mf]

1) This source doesn't need mfToon. The lines are dark enough. If you want to sharpen anyway, try this:
ssxshref = last
ssxshref.LanczosResize(ssxshref.width*4, ssxshref.height*4)
XSharpen(255, 255)
LanczosResize(ssxshref.width, ssxshref.height)
aWarpSharp()


2) Kurosu is right. The artifacts you are seeing are a known issue on noisy sources.

[mf]

I have a new version.. I actually wanted to make a "proper" version, but as I'm leaving for holidays tomorrow (and don't have time to script avisynth in the mean time[then why the hell do I have time to post on doom9]), I can't create that version. Instead, I have a intermediate version for all you experienced anime encoders that are breaking their heads over the "emboss artifact" of mfToon v0.3x :D.

Get it at mf's AVS dumpster (http://nabeshin.ddo.jp/avisynth/) :D. As you can see there's also some other crap in there, so leech at your pleasure.

Now why is this an intermediate version? Cause it's locked at 640x480 (or 768x576), unless you use your l33t AVS scripting skills on it and adapt it for another res (don't forget to update ssw and ssh if you do!). I was getting errors when making it dynamic, (and there was an important encode deadline I had to keep :D) so I just left it at this.
When I get back I hope to create a better version.

Happy summer everyone! :)

[Duncan_07]

Hi all.

I tried mfToon v0.32, but it crash my system during te first pass.

"Out of system ressource".

I haven't try others version of mftoons because the resolution of the video is not 640x480.

Someone have tips on that ?

Thinks.

[Jedd]

Hi,
I was playing around with mfToon and I have to say it works great when it comes to edging:
source (http://webpages.charter.net/jedd/ORIG.png)
no mfToon (Dean, TemporalCleaner): (http://webpages.charter.net/jedd/NO_MF.png)
mfToon (Dean, TemporalCleaner) (http://webpages.charter.net/jedd/MF.png)

As you can see without mfToon original source and compressed one look almost identical (~1000kbps here). After mfToon get involved picture became much more alive and warm, BUT - check all those blocks on background and artifacts around bed edges. :scared:
Now, I wonder, is it possible to use mfToon and keep blocks and artifacts at normal level?
:stupid:

P.S. I was using 0.32, 0.4 complains "Unrecognized Exception! line 106"( BlankClip(greymask, color=$FFFFFF) ??) even with avisynth-that-works-with-mftoon.dll renamed and put into system32.

[mf]

Originally posted by Jedd
P.S. I was using 0.32, 0.4 complains "Unrecognized Exception! line 106"( BlankClip(greymask, color=$FFFFFF) ??) even with avisynth-that-works-with-mftoon.dll renamed and put into system32.
Please make sure the input mfToon is getting is YV12, and your script is fault-free, and try removing Temporalcleaner. I have found that whenever I get an unrecognized exception with a release mfToon (that means not a new version I'm still working on), it is one of those. For some evil reason, mfToon often conflicts with filters like Convolution3D. I've been thinking of getting an avisynth dev to look into that, since mfToon is 100% scripted, so any scripting errors should come out as parse errors, and the rest of the faults can only be caused by avisynth itself, its filters, or your environment (I've had cases where something would work on one system and the same script would give some error on another).

Hope this helps, and thanks for using mfToon :).