New anime filter: mfToon

[skizm]

Thanks for the help, that solved that problem. But now I'm back to where I started:

Evaluate: Unrecognized exception!
(mfToon-v0.5.avs, line 112)

.XSharpen(xstren, xthresh).BicubicResize(twidth, theight, 0, 0.75)

That's the line. I dunno why :\

[Duncan_07]

So i test the new version and it'is two time faster (4 img/s :p) on my athlon 2400+ without other filters.
Very good work Mf ^^
I will do some other tests after christmas to see if there is a qualitee improvement. But i think it can't be wost than the 1.4 ;)

[MrBunny]

Hey mf, love your work!

I am having the same problems as skizm with the unrecognized exception on line 112. It seems to me to have something to do with the height in the BicubicResize:
semifinal = dark.BicubicResize(ssw2, ssh2, 0, 0.75)

Even if I remove the mergeluma(semifinal) in the final= line, it will cause the exception. The only way I've found to bypass the exception is to replace ssh2 with the original height (though that will cause an avisynth memory access error). I tried it with just mftoon() and mftoon(512,384).

I hope this helps a bit and I look forward to any response.
Thanks again mf,

Mr. B

[clima]

Originally posted by skizm
Thanks for the help, that solved that problem. But now I'm back to where I started:

Evaluate: Unrecognized exception!
(mfToon-v0.5.avs, line 112)

.XSharpen(xstren, xthresh).BicubicResize(twidth, theight, 0, 0.75)

That's the line. I dunno why :\

I have got the same issue.

I have got AviSynth 2.52, build: Jun 1 2003, I am going to try a newer one. (I know it is supposed to work under avisynth 2.52)

The test script is :


Top=6
Left=4
Right=2
Bottom=10

Import("C:\Program Files\AviSynth 2.5\plugins\mfToon-v0.5.avs")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\WarpSharp\WarpSharp.dll")

Mpeg2Source("I:\Animes\test.d2v")
IT()

LanczosResize(640,480)

mfToon()


Edit : Latest avisynth version doesn't change anything.

[clima]

Originally posted by MrBunny
[B]The only way I've found to bypass the exception is to replace ssh2 with the original height (though that will cause an avisynth memory access error). I tried it with just mftoon() and mftoon(512,384).


I have changed the "ssh2 = 480" for the previous script and status bar of vdubmod indicates : Avisynth read error: "Avisynth: illegal instruction at 0x03a21be3"
Changing "semifinal = dark.BicubicResize(ssw2, 480, 0, 0.75) \ " and "dstring3 = "Bicubicsized: " + String(ssw2) + "x" + String(480)" leads to the same line 112 error.

[mf]

Originally posted by clima
I have changed the "ssh2 = 480" for the previous script and status bar of vdubmod indicates : Avisynth read error: "Avisynth: illegal instruction at 0x03a21be3"
Changing "semifinal = dark.BicubicResize(ssw2, 480, 0, 0.75) \ " and "dstring3 = "Bicubicsized: " + String(ssw2) + "x" + String(480)" leads to the same line 112 error.
Don't even bother. That won't do what you want it to do.

My testing:
AVS 2.5.0 works (http://prdownloads.sourceforge.net/avisynth2/AviSynth_250.exe?download)

AVS 2.5.1 works (http://prdownloads.sourceforge.net/avisynth2/AviSynth_251.exe?download)

AVS 2.5.2 works (http://prdownloads.sourceforge.net/avisynth2/AviSynth_252.exe?download)

AVS 2.5.3 works (http://prdownloads.sourceforge.net/avisynth2/AviSynth_253.exe?download)

I'll try to debug some more.

[Chainmax]

I'm going to rip my Simpsons DVDs soon. What do you recommend me for sharpening, SSXSharpen() or mfToon()? I'm running on an Athlon 850Mhz with 160Mb Ram; what speed do you think I should expect when using either? Maybe 1 fps?

[mf]

Originally posted by Chainmax
I'm going to rip my Simpsons DVDs soon. What do you recommend me for sharpening, SSXSharpen() or mfToon()? I'm running on an Athlon 850Mhz with 160Mb Ram; what speed do you think I should expect when using either? Maybe 1 fps?
Simpsons has pretty thick lines, so I'd say ssxsh. No need to darken anything from what I've seen.

[Chainmax]

Cool, thanks :).

[Popovk]

Originally posted by skizm
Thanks for the help, that solved that problem. But now I'm back to where I started:

Evaluate: Unrecognized exception!
(mfToon-v0.5.avs, line 112)

.XSharpen(xstren, xthresh).BicubicResize(twidth, theight, 0, 0.75)

That's the line. I dunno why :\
Having the same error, and it seems to be related to WarpSharp. I mean, as for each script, plugin are not in default avisynth folder and so manually loaded, whatever warpsharp is loaded or not, the error is the same, and vdub/avisynth didn't complain about the missing plugin
Already tried the one from the package in 1st page of the thread and the other in the post a little upper, same results. Strange, as it worked with 0.4x of mftoon
Too late to continue on it, and I won't be able until end of week ... ;)

[MrBunny]

For those of you like me having trouble, does commenting out the line of that code

.XSharpen(xstren, xthresh).BicubicResize(twidth, theight, 0, 0.75)

with or without deleting the \ at the end of the previous line prevent the error from occuring? It doesn't for me :(
Thanks for trying to debug mf, I'll be playing with it a bit more later.

Mr. B

[skizm]

#semifinala = dark.BicubicResize(ssw2, ssh2, 0, 0.75)
semifinal = dark.XSharpen(xstren, xthresh).BicubicResize(twidth, theight, 0, 0.75)

I changed lines 111 and 112 to be like that. While that won't do what mfToon is made to do, it seems to have gotten the error outside of the mfToon script. VirtualDubMod then gave me an error in MY script on line 33, which was, surprise, a BicubicResize. I commented that out and now I get the memory access error that has already been described here. I have no idea what the problem is ;/

[MrBunny]

This is what it seems to boil down to for me right now. Having it the way it is already written will give me an the line 112 error. Changing the lines to :

semifinal = dark.BicubicResize(ssw2, theight, 0, 0.75) \
.XSharpen(xstren, xthresh).BicubicResize(twidth, theight, 0, 0.75)

final=orig.BicubicResize(twidth, theight, 0, 0.75).MergeLuma(semifinal).aWarpSharp(cm=cwarp2, depth=wdepth, \
blurlevel=wblur, thresh=wthresh)
will get rid of that error, but there will be an avisynth read error when the avs is loaded.

Changing the code to (removing the mergeluma):
semifinal = dark.BicubicResize(ssw2, theight, 0, 0.75) \
.XSharpen(xstren, xthresh).BicubicResize(twidth, theight, 0, 0.75)

final=orig.BicubicResize(twidth, theight, 0, 0.75).aWarpSharp(cm=cwarp2, depth=wdepth, \
blurlevel=wblur, thresh=wthresh)
will allow the script to function (though of course it's not working as it was designed without the mergeluma)

This is all with mftoon() after resize. If I use it before resize then as skizm just said, avisynth will give an exception on the resize line in the main script and if there is no resize before or after, it just gives me an avisynth read error.

I hope this might help the debugging...

Mr B.

[mf]

The only fixes I can see:

Have the same filter versions as on http://mf.creations.nl/avs/filters/

Don't use plugin autoloading

Don't load more than one version of MaskTools at a time (Autoloading might give problems?)

If that doesn't work I really wouldn't know of any fixes anymore..

[mf]

Just in news from my lovely debugger Akirasuto, the problem is probably related to MergeLuma. mfToon v0.51 (http://mf.creations.nl/avs/functions/mfToon-v0.51.avs) doesn't use MergeLuma, so I hope this fixes all your problems (except the love-life ones ;)). Merry Christmas/hanukkah/whateveryoucelebrate!

[Dali Lama]

I hate to be the bearer of bad news, but mftoon v5.1 gives an unrecognized exception on line 115 now.

I hope others have more success. Thanks for helping mf.

edit2: I was wrong. The exception in mftoon v5.1 comes from:

final = dark.BicubicResize(ssw2, ssh2, 0, 0.75).XSharpen(xstren, xthresh).BicubicResize(twidth, theight, 0, 0.75).MergeChroma(semifinal).aWarpSharp(cm=cwarp2, depth=wdepth, blurlevel=wblur, thresh=wthresh)

edit3: when I comment out everything after bicubicresize the script loads but gives an access violation error. If I leave bicubiresize in it won't load. Could it be the resize?

Now, if only I could fix it ;)

-Dali

[mf]

Originally posted by Dali Lama
edit3: when I comment out everything after bicubicresize the script loads but gives an access violation error. If I leave bicubiresize in it won't load. Could it be the resize?

Now, if only I could fix it ;)

-Dali
And my above fixes (autoload, etc)?

[mf]

I've just noticed at every scenechange one of the masks comes through to the return clip. Really, something should be fixed in AVISynth, as I really don't see another reason why it would happen.

[Didée]

Originally posted by mf
I've just noticed at every scenechange one of the masks comes through to the return clip. Really, something should be fixed in AVISynth, as I really don't see another reason why it would happen.
Some time ago, I have seen exactly that when I was trying some mask operations on very big framesizes. The solution was to manually force SetMemoryMax() to high values, like 128 or 160.
I'm not sure if this is really AviSynth's fault, rather than MaskTools' one.
Generally, the SetMemoryMax-trick *sometimes* is able to solve unexplainable errors in scripts that are technically correct. It never hurts to give it a try in complex scripts ;)

- Didée

[mf]

Well, I've got 1GB of memory so that won't be a problem then :D. I'll do SetMemoryMax(768) and see if that fixes it :p.

[MrBunny]

Thanks for looking into this mf, but it still isn't working :(
This is my current script:

LoadPlugin("...path...\mpeg2dec3dg.dll")
LoadPlugin("...path...\masktools142.dll")
LoadPlugin("...path...\warpsharp.dll")
LoadPlugin("...path...\awarpsharp.dll")
import("...path...\mfToon-v0.51.avs")

SetMemoryMax(512)
mpeg2source("...path...\2.d2v",cpu=0,ipp=true)
crop(8,2,704,476)
LanczosResize(512,384)
mftoon()


This gives the exception at line 115.
I just redownloaded all 3 .dlls from mf's website too, so it shouldn't be that and there should be no dlls being autoloaded.
Happy Holidays to all

Thanks,
Mr. B

[clima]

I have got the exception at line 115 too :(

[K-Dash]

Impressive, thx for the good work. :D

[mf]

Originally posted by K-Dash
Impressive, thx for the good work. :D
Thanks :).



Now, before I release v0.52 (it's not ready anyway), I want to put up a small poll.
The question is: Do you have a PC faster than me?
Answers are: Yes/No
Please state your processor speed and RAM capacity and speed. You may also guess why I'm asking this :D.
I have an Athlon 2500+ (Barton), with 1GB dual-channel DDR RAM.

This poll is related to something new in mfToon v0.52 :p.

[skizm]

I've got a Pentium 4 3.2ghz HT with 1GB of PC 3200.

[acrespo]

Answer: No
Athlon XP 1800, 1Gb RAM.

[Chainmax]

No.
Athlon T-Bird 850Mhz + 160Mb PC100 RAM (used to be 256Mb PC133).

If you're worried about wether this will be to slow for some of us, what about making it optional?

[skizm]

Originally posted by Chainmax
If you're worried about wether this will be to slow for some of us, what about making it optional?

Everything is optional anyway ;p

[mf]

Originally posted by Chainmax
No.
Athlon T-Bird 850Mhz + 160Mb PC100 RAM (used to be 256Mb PC133).

If you're worried about wether this will be to slow for some of us, what about making it optional?
I'm just interested, and of course it's optional. Nobody wanna take guesses at what it is ? :D

[clima]

PIV 2,6 GHz with 512 MB of DDR Ram

@ mf : Do you have any other idea about what is causing mfToon 0.5 not to work for about half of us ?


Nobody wanna take guesses at what it is ?


Humm, you are willing to slow down under 1 fps for such high config's PCs ? :p

[mf]

Originally posted by clima
@ mf : Do you have any other idea about what is causing mfToon 0.5 not to work for about half of us ?Sadly, nope :(.
Humm, you are willing to slow down under 1 fps for such high config's PCs ? :p Wrong guess. Try again :D.

[Dreassica]

Might it be the HQdering u are talking about in teh other thread? ;)

[acrespo]

mftoon 0.51 is working here, but when I change the strength parameter to anything different than 255 (default value), I receive a error in line 107 ('dark' is not recognized).

[mf]

Originally posted by acrespo
mftoon 0.51 is working here, but when I change the strength parameter to anything different than 255 (default value), I receive a error in line 107 ('dark' is not recognized).
Yeah. I noticed that too. Kurosu interpreted my script wrong and put it like that in his K-mfToon. When I was looking for things i could steal (:devil:) from his version I simply copywasted that line. Expect a fix in 0.52. And Dreassica: Nope :D.

[Popovk]

Originally posted by mf
And my above fixes (autoload, etc)?
none of them.
On the other hand, I think I have found why it doesn't work on some computer (thanks to the 3 differents I have at home :D )

Mf, I assume you have an athlon, right ?
CPU type resume of those who have the error :
MrBunny : ?
skizm : Pentium 4 3.2ghz HT with 1GB of PC 3200
Dali Lama : ?
clima : PIV 2,6 GHz with 512 MB of DDR Ram
And myself : P4 3.0Hz with 1Gb ram in dual mode, tested with and without hyperthreading.

Now the final answer from the final test : I did a copy/paste of the directories to my athlon 3ghz, lanched the avs script with mftoons, and it worked like a charm. As none who have an AMD processor did complaign about the error, easy to understand where the error come from.

mf, for purpose testing sake, can you do the same script without using any external plugin ? It may be usable to found where the error is coming from, as separately, the external plugin are working fine (also, if someone could test them to confirm)

[mf]

Originally posted by Popovk
mf, for purpose testing sake, can you do the same script without using any external plugin ? It may be usable to found where the error is coming from, as separately, the external plugin are working fine (also, if someone could test them to confirm)
Of course I can't. mfToon v0.5 barely uses any internal functions anymore. Used ones are:
Unsharpmask - WarpSharp.dll
Edgemask - MaskTools142.dll
YV12Layer - MaskTools142.dll
Inflate - MaskTools142.dll
MaskedMerge - MaskTools142.dll
XSharpen - WarpSharp.dll
aWarpSharp - aWarpSharp.dll

In that sense, it's easy to put the problem at MaskTools. Earlier mfToons used the old MaskTools, and alot of internal AVS functions (mainly Mask() and Layer()). Now, does MaskTools use any SSE2? If so, it might be interesting to disable it and create a new build.

Also: AllTimeSToneD has a dual P4 and can use mfToon without too many problems (none are really reproducible). That makes it a bit more mysterious.

[MrBunny]

Nice catch Popovk.

I'm using a P4 2.4c with HT enabled. I might be able to try it on a athlon cpu in the next couple days.

Mr. B

[Kurosu]

There are some 3DNow opcodes used instead of iSSE around the asm parts (can't remember where). Have Manao or a willingful asm coder check for paddusb instead of paddb, prefetch/prefetchw instead of prefetchnta.

I asked someone to send mf a fixed version of masktools 3 weeks ago, but it seems it wasn't done. bah.

[mf]

Kurosu! :) Haven't seen you in a while. Indeed, I never got a "fixed" version of MaskTools, other than 142 that Manao compiled. You'll be glad to see that I'm putting your "cartoon" edge detection to use in 0.52. Best wishes for the new year!

[Dali Lama]

Originally posted by Popovk

Dali Lama : ?


Dali Lama = P4

I think your're right Popovk, we are all running P4's that are having problems. Hopefully Kuroso has spotted the problem and someone can fix it.

-Dali

[Aiorus]

Well, for what it's worth, I tried mftoon v 0.50 on a PIII 700 MHz and it also has the error (Line 112 on 0.50, line 115 on 0.51+), and the same error appeared on a celeron computer. Apparently it's an Intel thingie.

Edit: I forgot to mention, it did work on an Athlon XP 2700

[WyldeF1r3]

kurosu coded the latest masktools in amd assembler. i have an intel-fixed version of masktools 1.4.1 if someone knows how i can attatch a file to a post. or you could just wait until kurosu rewrites it now that he seems back from vacation ^_^

[mf]

Originally posted by WyldeF1r3
kurosu coded the latest masktools in amd assembler. i have an intel-fixed version of masktools 1.4.1 if someone knows how i can attatch a file to a post. or you could just wait until kurosu rewrites it now that he seems back from vacation ^_^
Well that would be useless because Inflate is broken in 141. So unless you have the source too it won't help anyone.

[Manao]

I updated the MaskTools to v1.4.3, it should work with P4 ( not tested though ). I only removed the prefetch instructions. I corrected some bugs with optimizations ( which means I disabled some optimizations ), and I optimized two filters ( Invert and Binarize, which were very simple to optimize ).

Please report bugs on this thread (http://forum.doom9.org/showthread.php?s=&threadid=67232). You'll find the new version here (http://www.geocities.com/manao47/Filters/masktools-v1.4.3.zip). Copy & paste the link in your adress bar

[skizm]

The Geocities page doesn't work :\

If you want you can e-mail me the file and I'll post it on my hosting. Let me know if that's the case.

[Manao]

I forgot the usual disclaimer : copy & paste the link in your adress bar. I edited my previous post.

[NightMare]

hey mf wazzup

i'v been away for a while and when i come back i see mftoon v0.51

great work mf congrats

sadly i dont have any anime to encode right now but i'm gona try it on naruto and see hope nuthin's wrong ;)

[SSJGesus]

yep, it works fine on my P4 with the new masktools

pity i only get 1 or 2fps, and divx 5.1.1 running on slowest ontop of that :P

just imagine if i was doing multi-pass :P

[Popovk]

working fine now, and mf, you're crazy as hell, as usual.
Without mftoon but some plugins instead like msharpen : 26 fps
only mftoon : 4 fps :rolleyes:

[mrfil13]

im using the .51 mftoon and did a test sample of 1.2min at a filesize of 5mb but the quality was poor, here are my settings and script
oh and im trying to compress cowboy bebop the movie

newest koepi's xvid

2 pass
AS @ 5
h.263
quartpel
GMC
Bvop, max consec bvop 2, qaut ratio 1.5, offset 1, packed bitstream, closed gov

MSP - 6 ultra high
VHQ - 4
Chroma motion
trellis

script
# PLUGINS
LoadPlugin("E:\Rip\GordianKnot\mpeg2dec3.dll")
LoadPlugin("E:\Rip\GordianKnot\undot.dll")
LoadPlugin("E:\Rip\GordianKnot\masktools\MaskTools142.dll")
LoadPlugin("E:\Rip\GordianKnot\aWarpSharp\aWarpSharp.dll")
LoadPlugin("E:\Rip\GordianKnot\WarpSharp.dll")
# SOURCE
mpeg2source("E:\Fast\cowboy009.d2v")
# CROPPING
crop(0,14,718,548)
# RESIZING
lanczosResize(640,336)
#
Import("mfToon.avs")
mfToon()

im trying to get this onto a 700mb cd