New anime filter: mfToon

[Jedd]

mf
I found the problem, I had a crop to 698. When I have changed it to 696 I managed to load 0.4 (doesnt really explain why 0.32 works fine with 698)
And I have to say .4 looks like a great improvement over .32
Check this out:
mfToon.32 + Deen().TC() (http://webpages.charter.net/jedd/f2_mf_32_dt.avi)
mfToon.4 + Deen().TC() (http://webpages.charter.net/jedd/f2_mf_4_dt.avi)
But Deen().TC() is a must here to get rid of blocks and warps.
Plain mfToon.4 (http://webpages.charter.net/jedd/f2_mf_4.avi)

Right now my script is as simple as this:

LoadPlugin("C:\Media\Plugins\warpsharp.dll")
LoadPlugin("C:\Media\Plugins\aWarpSharp.dll")
LoadPlugin("C:\Media\Plugins\MaskTools.dll")
LoadPlugin("C:\Media\Plugins\deen.dll")
LoadPlugin("C:\Media\Plugins\TemporalCleaner.dll")
LoadPlugin("mpeg2dec3.dll")
LoadPlugin("decomb.dll")
Import("C:\Media\mfToon-v0.4.avs")

mpeg2source("C:\Media\DVD\slt_a.d2v")
Telecide(guide=1).Decimate(5)
crop(12,0,696,480)
mfToon()
Deen("a2d",5,5,10).TemporalCleaner(4,6)

What else can be done here? Maybe some dup()? And I can't really decide - should I resize 696x480 to 640x480 or not...

[mf]

Originally posted by Jedd
What else can be done here? Maybe some dup()? And I can't really decide - should I resize 696x480 to 640x480 or not...
Your source seems pretty clean, try raising deen's temporal influence with steps of 10 at a time until you're satisfied. That should also remove the need for temporalcleaner (it has a tendency to ghost, so I would disrecommend it). Avoid Dup at all costs.

[Jedd]

Originally posted by mf
Your source seems pretty clean, try raising deen's temporal influence with steps of 10 at a time until you're satisfied. That should also remove the need for temporalcleaner (it has a tendency to ghost, so I would disrecommend it).
You mean like Deen("a2d",7,5,10,10), Deen("a2d",7,5,10,20), Deen("a2d",7,5,10,30) etc.? Don't really see any difference between 10 and 50. :confused:
I have tried almost all combinations, but the only way to reduce blockiness is to use TemporalCleaner.

[mf]

Originally posted by Jedd
You mean like Deen("a2d",7,5,10,10), Deen("a2d",7,5,10,20), Deen("a2d",7,5,10,30) etc.? Don't really see any difference between 10 and 50. :confused:
I have tried almost all combinations, but the only way to reduce blockiness is to use TemporalCleaner.
Err, a2d doesn't have temporal influence. Try a3d.

[killerhis]

yo mf.... I really like these new versions of mftoon (0.4 and 0.4-768) but I still get some artifacts on all my encodes.... The collors looks like to be darken and then light again and changing all the time. When I remove mftoon the artifacts disapere. If I use version 0.32 I also lose the artifacts.. but it ain't that good as the new versions.

Here I got a previeuw.. compare them and look especally at the monitor.. you'll see that the collor changes a bit from dark to light and again.

So anyone know how to fix it or mf... when will the new version be done :D

http://home.hccnet.nl/h.chourak/clip(original).avi
http://home.hccnet.nl/h.chourak/clip(remasterd).avi

[killingspree]

hi @ll
since i just had a bit of a hard time to find masktools, here's a new link to kurosu's download page:
http://kurosu.inforezo.org/avs/MaskTools/index.html


furthermore i've asked kurosu if it's ok for him to put masktools on warpenterprises filter collection and since he has given his ok, i'm going to PM warpenterprises now and ask him to upload the piece so hopefully you can soon find all filters there :)

steVe

[mf]

@killerhis

The only source of your problem that I can see, is the anime being analogue which always has some annoying fluctuations.. It might be mfToon enhancing them. Some further denoising might fix it. And about the new version: I'm off-schedule with anime encoding and I really need to catch up with some other things, so I'm a bit busy right now.

[killerhis]

@mf

I've been jusing the values: mftoon(sharpen=false).awarpsharp(cm=1)

I get the same result on clean sources, like blue submarine, but I'll try the heavier denoising.. and it's to bad to hear new mftoon is delayed for a while
:mad:

/[edit] heavy denoising, not much effect: VegueDenoiser(threshold=32,method=1,nsteps=7,chroma=true)

[Kurosu]

Originally posted by killingspree
hi @ll
since i just had a bit of a hard time to find masktools, here's a new link to kurosu's download page:
http://kurosu.inforezo.org/avs/MaskTools/index.html

That's the page that was not adapted in a long time. MaskTools is one of the filter for which I've pretty much dropped support, and only update for my own purpose.

An actual link would be http://kurosu.inforezo.org/avs/MaskTools/MaskTools1.4.zip

[killingspree]

thanks for the info

hmm i guess i must have gotten the oldest version possible :) -0.1

no wonder mftoon then is soooo slow... it took me 12 hours to do the first pass of a 40 min anime :D

steVe

[mf]

Originally posted by killerhis
/[edit] heavy denoising, not much effect: VegueDenoiser(threshold=32,method=1,nsteps=7,chroma=true)
Isn't VagueDenoiser 2D ? Luma fluctuations are temporal, so you need at least a 3D denoiser.

[Jedd]

Originally posted by killerhis
The collors looks like to be darken and then light again and changing all the time. When I remove mftoon the artifacts disapere. If I use version 0.32 I also lose the artifacts..

killerhis,
I had absolutely the same problem after using mfToon 0.4. So I made 40 frames span and started to apply Deen() filter with all kind of parameters. After making about two dozen clips I opened em up in the VirtualDub and sorted out all with too much blocks, warps and blinks.
Deen("a3d",3,10,12,20) have worked best for me.
The only downside - expect another 12 hours of encoding. :sly:

[Kurosu]

Originally posted by killingspree
no wonder mftoon then is soooo slow... it took me 12 hours to do the first pass of a 40 min anime :D
The filter is really slow, particularly because of upsampled operations. Then there is various colorspaces conversions and the like. MaskTools1.4 has a rewritten mfToon version that avoid most of them. AFAIK, it was around 3 times faster without upsampling, but was outputting images a bit different. Anyway, the purpose wasn't to produce a faster mfToon, but rather to show how it could be rewritten for a close result.

[killingspree]

first of allm, the result of the last 24 hours of encoding are simply amazing... the quality is simply soo much better :)

anyway, as i was still using masktools 0.1 i decided to try again with masktools 1.4 from the zip file kurosu linked to just above.

unfortunately, when loading exactly the same script into vdubmod as before i get the following error:
Script error: invalid arguments to function "EdgeMask"
(h:\folder\mftoon-0.4.avs,line 91)
(h:\folder\main.avs, line 13)


here's the original script which worked with 0.1 but didn't work with 1.4! i guess it could use some major tweaking too :)
btw, i'm dealing with a relative clean PAL source which is already encoded at a fairly low bitrate though. the vobs just span 1.3 Gigs. my encoding bitrate is fairly high too as i do not have anything else to put on the cd. so i'm going for 2410 kbps and a 192 kbit mp3 soundtrack...


Loadplugin("c:\programme\avisynth 2.5\plugins\awarpsharp.dll")
loadplugin("h:\f102\video_ts\warpsharp.dll")
Loadplugin("c:\programme\avisynth 2.5\plugins\masktools.dll")
import("h:\f102\video_ts\mftoon-v0.4.avs")

mpeg2source("H:\F102\VIDEO_TS\main.d2v")

FieldDeinterlace(blend=false)

crop(14,8,694,568)
BicubicResize(608,464,0,0.5)

MfToon()
Deen("a2d",5,5,10)
TemporalCleaner(4,6)

BicubicResize(608,464,0,0.5)


thanks for any help
steVe

[Kurosu]

NoOoOo, don't force me to look for a bug / compare old and new syntaxes >.<
Besides, webhost is done *sigh*

[mf]

Originally posted by Kurosu
NoOoOo, don't force me to look for a bug / compare old and new syntaxes >.<
Using fixed params pays off? :D

[Kurosu]

Originally posted by mf
Using fixed params pays off? :D
It would but EdgeMask has changed too much. It should now read:
orig.EdgeMask(thY1=0, thY2=255, type="sobel")
(maybe add U=-128, V=-128, but chroma shouldN'T matter) [EDIT]
instead of:
orig.EdgeMask(0, "sobel")


1) EdgeMask(th1=[0..255],th2=[0..255],string):
Component is either Y, U or V; derivative is the result of the convolution
by the differential kernel. MMX code is only used with MOD8 width (might
not be necessary).
- string: which differential convolution to use:
. "line" will try to spot darker thin (1 pixel-wide) lines in the
picture (it has an incorporated anti-noise floor which makes it ignore
any variation lower than 3)
. "roberts" will apply a pseudo-Roberts 2x2 kernel:
2 -1
-1 0
. "sobel" will apply a pseudo-Sobel 3x3 kernel:
0 -1 0
-1 0 1
0 1 0
. "hq" uses a Laplacian kernel:
-1 -1 -1
-1 8 -1
-1 -1 -1
. "special" uses this kernel:
. "cartoon" uses the pseudo-Sobel kernel but only keeps components with
lower value than their neighbours (i.e. negative values of the
derivative)
- th1: threshold under which component value is set to 0
- th2: threshold under which component value is set to the derivative
value, and over which it's set to 255

(notice how it was edited in a hurry and almost left out as is)

[mf]

The cartoon mode is interesting, from what I read of it it only catches two-sided edges (lines)?

[Kurosu]

Well, in a cartoon/anime line, there are 2 edges; consider this line of pixels:
... 167 167 167 34 34 34 167 167 167...
a basic derivative line is (kernel -1 2 -1):
... ? 0 33 -33 0 -33 33 0 ? ...
2 edges are therefore seen.

The cartoon mode simply returns:
... ? 0 0 33 0 33 0 0 ? ...
and now the edges are located on the pixels that are the "darker" (only valid in luma plane, you can produce other masks on other plane with other derivative types). It therefore produces less "false" positives regarding the lines in a cartoon image. But it can have its drawback, and the fact that you Inflate() the edgemask should mean you aren't looking for that effect.

Another possibility could be to only mark as edges pixels that are dark (not darker, like 224 compared to 240, which are both bright), but it would mainly produce artifacts due to thresholding (that's why I introduced 2 thresholds to allow direct manipulation without tweaking with levels()).

One last point: there are now 2 means to increase/decrease areas. I don't remember which is which, but:
- one takes the maximum/minimum value in the 3x3 neighbourhood
- another calculates the average (center pixel not included) of such neighboorhoud, and replace it when it's higher/lower than the original value

Most of them would benefit a lot from iSSE optimizations (over simple MMX as of now) but they weren't worth the developping time needed for my sole purpose.

[mf]

Originally posted by Kurosu
The cartoon mode simply returns:
... ? 0 0 33 0 33 0 0 ? ...
and now the edges are located on the pixels that are the "darker" (only valid in luma plane, you can produce other masks on other plane with other derivative types). It therefore produces less "false" positives regarding the lines in a cartoon image. But it can have its drawback, and the fact that you Inflate() the edgemask should mean you aren't looking for that effect.
I inflate because the edgemask I get is very dim and small values are 1) easily lost and 2) will not give very much effect when applying the actual line darkening effect. mfToon is "safe" in the sense that applying it to a greater area than only the line itself will not be seen. So with my inflated and amplified mask, I'm sure that all lines will get caught and get a good line darkening effect (slight lines deserve just as much darkening as thick lines). The only trouble is that EdgeMask also catches "single" edges, like the edge of cel shading (a line has edges on both sides of it, shading has only a single edge). That is the only real drawback it has (and it darkens black-outlined OP credits, but that's barely noticable and perfectly within its function specification. It darkens black lines :D).

[alucard83]

Damn, I need aWarpsharp but the link is broken I believe and it links me to some dating(?) site. Is there anywhere else that hosts it?

[MajinMarc]

Error in script?
I recently tried using the params for mftoon in my script. it was always unable to load the preview if I used params and if I tried to encode vdubmod's status window never popped up and my computer lagged horribly. I was wondering if maybe this is a result of my using params that are too high, or did I make an error in my script.

SetWorkingDir("C:\Documents and Settings\Marc\My Documents\My Downloads\Encoder\filters")
LoadPlugin("Warpsharp.dll")
LoadPlugin("awarpsharp.dll")
LoadPlugin("MaskTools.dll")
AVIsource("C:\PICCOLO_JR_SAGA2_DISC2\VIDEO_TS\english2.avi")
Import("mfToon-v0.32.avs")
mfToon(strength=100,wdepth=25.0,wblur=10,wthresh=0.7,ssw=25,ssh=25)

Thank you for your time.

[pcsoul]

it is so slooooooooooooooow...

2 fps @ P IV 2.4C + Asus 865PE + DDR 400 (Dual Channel) With mfToon 0.32

cry...

who can help me?

[alucard83]

k, I loaded my script into Vdub and I immediately got an error:

AviSynth open failure:
Script Error: there is no function named "EdgeMask"
(G:\mfToon-0.32.avs, line 75)
(G:\Rurouni Kenshin(DVD1)\ep5.avs, line 23)


It seems it cannot recognize the function in the mfToon script itself
Here's my script

SetWorkingDir("C:\PROGRA~1\GORDIA~1\")
LoadPlugin("mpeg2dec3.dll")
LoadPlugin("decomb.dll")
LoadPlugin("UnDot.dll")
LoadPlugin("WarpSharp.dll")
LoadPlugin("Convolution3DYV12.dll")
LoadPlugin("SmoothDeinterlacer.dll")
LoadPlugin("UnFilter.dll")
Import("G:\mfToon-v0.32.avs")

mpeg2source("G:\Rurouni Kenshin(DVD1)\ep4.d2v")


#IVTC
Telecide(guide=1,gthresh=50,chroma=true,threshold=30)
Decimate(cycle=5)

#DEINTERLACE
ConvertToYUY2().SmoothDeinterlace()

#FILTERS
mfToon()
Convolution3d("AnimeLQ").Temporalsoften(4,4,8,mode=2,scenechange=8)
UnFilter(0,0)
UnDot()

Can someone tell me what I'm doing wrong? I've tried to open it in vdub v1.5.4 as well as the latest version of vdubmod and I get the same error
I'm fairly new to AviSynth so plz bear with me:)

[pcsoul]

Originally posted by alucard83
k, I loaded my script into Vdub and I immediately got an error:

AviSynth open failure:
Script Error: there is no function named "EdgeMask"
(G:\mfToon-0.32.avs, line 75)
(G:\Rurouni Kenshin(DVD1)\ep5.avs, line 23)


It seems it cannot recognize the function in the mfToon script itself
Here's my script

SetWorkingDir("C:\PROGRA~1\GORDIA~1\")
LoadPlugin("mpeg2dec3.dll")
LoadPlugin("decomb.dll")
LoadPlugin("UnDot.dll")
LoadPlugin("WarpSharp.dll")
LoadPlugin("Convolution3DYV12.dll")
LoadPlugin("SmoothDeinterlacer.dll")
LoadPlugin("UnFilter.dll")
Import("G:\mfToon-v0.32.avs")

mpeg2source("G:\Rurouni Kenshin(DVD1)\ep4.d2v")


#IVTC
Telecide(guide=1,gthresh=50,chroma=true,threshold=30)
Decimate(cycle=5)

#DEINTERLACE
ConvertToYUY2().SmoothDeinterlace()

#FILTERS
mfToon()
Convolution3d("AnimeLQ").Temporalsoften(4,4,8,mode=2,scenechange=8)
UnFilter(0,0)
UnDot()

Can someone tell me what I'm doing wrong? I've tried to open it in vdub v1.5.4 as well as the latest version of vdubmod and I get the same error
I'm fairly new to AviSynth so plz bear with me:)

try to edit mfToon-0.32.avs,find orig.EdgeMask(0, 255, "sobel"),use orig.EdgeMask(thY1=0, thY2=255, type="sobel")

[mf]

I'll put up a new AVS dumpster with all necessary filters soon. Hang on.

[alucard83]

Thnx pcsoul I'll try that



Update*: It's a no go:( :mad:

[pcsoul]

Originally posted by mf
I'll put up a new AVS dumpster with all necessary filters soon. Hang on.

good news!!!

[mf]

Originally posted by mf
I'll put up a new AVS dumpster with all necessary filters soon. Hang on.
And here it is (http://mf.creations.nl/avs/). Cheers! :)

[alucard83]

Schwanky!

[mf]

OMG, I am such a 馬鹿!! (baka)

I was defining "width" twice, instead of defining "width", and "height", once. So that's why it wasn't working. Trivial, trivial, trivial. mfToon v0.41 (http://mf.creations.nl/avs/functions/mfToon-v0.41.avs) is out, and it finally supports target width and height properly. Thanks to superdump for motivating me to look at it again, so I could finally find the flaw. Anyway, have fun. :)

[lamer_de]

Hi all.
I adapted the script kurosu provided with his MaskTools to mfToon 0.4.1. It runs around 10% faster on my system and with my usual scripts. I don't see any difference to the original mfToon 0.4.1 besides the speed increase, but kurosu stated that there's up to 5% difference in the edgde mask, so use at your own risk ;)

EDIT:
hmm, somewhere I f*cked it up, but nobody complained, so I guess nobody usedit anyway :P

CU,
lamer_de

[NightMare]

hey mf
i found mfToon-v0.42.avs at your site
whats new in that

[mf]

Sorry lamer_de, I haven't switched to the latest MaskTools yet, BUT I do have a new version, with some lame new features.

Here they are:
Horizontal/vertical sharpening: If you need less sharpening in either the horizontal or vertical direction, you can use these two parameters. Minimal value is 1 (meaning 100% sharpening), maximum is the same as you set your ssw/ssh to, so if you set it to 4 while ssw/ssh is 4, you will have 0% sharpening (example: mfToon(hsharp=1, vsharp=4) will sharpen only horizontally).
Show: Darkened lines are emphasized in purple, or any other color you choose with scolor.
Debug: Show some lame text, and optionally show any of the processing stages (example: mfToon(debug=true, dclip="sharp2") will show the unsharp masked image).

It's here (http://mf.creations.nl/avs/functions/mfToon-v0.42.avs).

Edit:
Originally posted by NightMare
hey mf
i found mfToon-v0.42.avs at your site
whats new in that
Damn you're too fast, I was busy with updating :p.

[Chainmax]

Is there any chance that mfToon becomes an actual DLL-based filter? Maybe you could grab the source of all the filters it uses and (with the permission of the original filter authors, of course) implement all the individual things you use from those filters in a single DLL. That would probably make it easier to upgrade/update mfToon, right?

Not that I'm complanining, though. mfToon is perfect right now, I'd just prefer to handle a DLL rather than a script.

[OBcecado]

Hi, just to warn that awarpsharp's link (http://ziquash.chez.tiscali.fr/aWarpSharp%20beta%201.zip) is broken.



All the best.

[Zarxrax]

I just tried mfToon, but it crashes vdubmod whenever I load the avs file using it! :confused:
I'm only interested in seeing the line darkening...

[mf]

Originally posted by Chainmax
Is there any chance that mfToon becomes an actual DLL-based filter? Maybe you could grab the source of all the filters it uses and (with the permission of the original filter authors, of course) implement all the individual things you use from those filters in a single DLL. That would probably make it easier to upgrade/update mfToon, right?

Not that I'm complanining, though. mfToon is perfect right now, I'd just prefer to handle a DLL rather than a script.
I would, if I could code...

Originally posted by Zarxrax
I just tried mfToon, but it crashes vdubmod whenever I load the avs file using it! :confused:
I'm only interested in seeing the line darkening...
See this thread (http://forum.doom9.org/showthread.php?s=&threadid=61689). The problem seems to come from mfToon's complexity, and maybe accumulating bugs or something. Afaik my script is errorfree, so the problem has to lie in AVISynth.

[Chainmax]

Originally posted by mf:
quote:
--------------------------------------------------------------------------------
Originally posted by Chainmax
Is there any chance that mfToon becomes an actual DLL-based filter? Maybe you could grab the source of all the filters it uses and (with the permission of the original filter authors, of course) implement all the individual things you use from those filters in a single DLL. That would probably make it easier to upgrade/update mfToon, right?

Not that I'm complanining, though. mfToon is perfect right now, I'd just prefer to handle a DLL rather than a script.
--------------------------------------------------------------------------------


I would, if I could code...

Maybe you could ask some of the other pros here to help you. You could always make new releases as avs scripts and let other people make DLLs out of them.

[mf]

Originally posted by Chainmax
Maybe you could ask some of the other pros here to help you. You could always make new releases as avs scripts and let other people make DLLs out of them.
Coding bitches are hard to find. Nobody has responded to my call for a coder for the best media player ever, either (it's in my sig).

[qwerpoi]

Actually, I don't see what's the problem in keeping it a function. Functions can autoload just as easily as dlls, just keep a text file in your plugins directory called "functions.avsi" and paste any number of functions in there (just be sure none of your functions have any syntax errors :) ). Unless there is some kind of significant speed increase to talk about when converting a function into a dll, I think functions are more appropriate for things like mftoon which exclusively uses existing filters.

[mf]

1: There is significant speed to be gained if everything was in YV12.
2: Parsing mfToon causes great instability to AVISynth
3: There might be some nice things optimizable.

[Zarxrax]

mf: I believe I may have found some insight on the crashing of mftoon. Ever since I installed the warpsharp.dll that mftoon requires, avisynth has been acting EXTREMELY unstable in all areas. Just removing this filter from my plugins folder, avisynth is back to normal and functioning great. What is this warpsharp.dll used for in mftoon? Is it possible to use some other filter in place of it, or not use it at all?

[stickboy]

Which version of warpsharp.dll are you using? Where did you get it from?

[Zarxrax]

I downloaded the one at the beginning of this thread.

[qwerpoi]

Unless I'm mistaken, this is a known problem (see the sticky YV12 faq in this forum, question 15) with warpsharp.dll. If you want to use it without crashing, all you have to do is put the dll somewhere other than your autoloading plugins directory and load it manually. Warpsharp.dll is necessary for the XSharpen() function (I believe mftoon uses awarpsharp() to do any warp sharpening).

[Zarxrax]

Ah I see, mftoon actually works now ^_^

[mf]

Originally posted by Zarxrax
mf: I believe I may have found some insight on the crashing of mftoon. Ever since I installed the warpsharp.dll that mftoon requires, avisynth has been acting EXTREMELY unstable in all areas. Just removing this filter from my plugins folder, avisynth is back to normal and functioning great. What is this warpsharp.dll used for in mftoon? Is it possible to use some other filter in place of it, or not use it at all?
There are two versions of warpsharp. Both contain XSharpen() and WarpSharp() functions. One used to be available from the AVS 2.5 alpha download page, and the other one from the beginning of this thread. The problem with the one from the AVS 2.5 alpha page, is that it causes jagged edges on full strength (http://forum.doom9.org/showthread.php?s=&threadid=46612&highlight=xsharpen). The one I use doesn't. Oh and btw, I never autoload plugins.

[Chainmax]

Originally posted by mf:
Coding bitches are hard to find.
They'll surely be if you call them like that :sly:. Plus, it's not like they'd have to code everything from scratch, but rather sew code bits together. I'd guess the workload would be lower that way.

[mf]

Originally posted by Chainmax
They'll surely be if you call them like that :sly:. Plus, it's not like they'd have to code everything from scratch, but rather sew code bits together. I'd guess the workload would be lower that way.
It's just a joke to call them like that ;). It's usually very fun for both sides to develop something with me (at least that's what they told me :D). I actually miss developing with a coder :(. But the pain of finding someone willing demotivated me so I don't usually go looking for them anymore.