mftoon 0.52 error

[IceManTX]

I have been trying to use this script to help with some anime stuff I have been recording. The 0.44 version seems to open fine in vdub but when I try to use the newer (and I assume better?) version 0.52 then I get the following error.

Avisynth open failure:
Script error: Invalid arguments to function "EdgeMask"
([path to mftoon52.avs file], line 89)

Does anyone know the reason for this? Also, this tool is dated around the 2004 mark as best I can tell. Is this still the best thing to be using for the line darkening/sharpening or has this gone by the wayside and something better has come along?

thanks!

[unskinnyboy]

:script:

[IceManTX]

OK, My script looks like this:

# PLUGINS
LoadPlugin("C:\Encoding\GORDIA~1\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Encoding\GordianKnot\AviSynthPlugins\TomsMoComp.dll")
#LoadPlugin("C:\Encoding\AviSynth 2.5\plugins\fluxsmooth.dll")
LoadPlugin("C:\Encoding\AviSynth 2.5\plugins\TIVTC.dll")
#LoadPlugin("C:\Encoding\AviSynth 2.5\plugins\AGC.dll")
LoadPlugin("C:\Encoding\AviSynth 2.5\plugins\degrainmedian.dll")
LoadPlugin("C:\Encoding\AviSynth 2.5\plugins\ColorMatrix.dll")
LoadPlugin("C:\Encoding\AviSynth 2.5\WarpSharp.dll")
LoadPlugin("C:\Encoding\AviSynth 2.5\MaskTools.dll")
LoadPlugin("C:\Encoding\AviSynth 2.5\aWarpSharp.dll")

Import("E:\Temp\OUTPUT\mfToon-v0.52.avs")
#Import("E:\Temp\OUTPUT\mfsharp.avs")

# SOURCE
mpeg2source("E:\Temp\OUTPUT\2x12 & 2x13 - The Serpent's Pass & Secret Of The Fire Nation.d2v",info=3) .ColorMatrix(hints=true,interlaced=true)

# IVTC/Deinterlacing
TFM(order=1,mode=3,PP=1)
TomsMoComp(1,5,1)
tdecimate(hybrid=1)

# CROPPING
crop(8,0,704,474)

# RESIZING
LanczosResize(640,480)

# DENOISING
DeGrainMedian()
#fluxsmoothst(7,7)

# Brightening
#mergeluma(HDRAGC(coef_gain=1.5,min_gain=1.0,corrector=.8),.20)
#HDRAGC(coef_gain=1.0,min_gain=1.0,corrector=.8)

mfToon()
#ssxsharpen()

=====================================


If I use the script at http://mf.creations.nl/avs/functions/mfToon-v0.44.avs then all is well.

But if I use the script at http://mf.creations.nl/avs/functions/mfToon-v0.52.avs then I get the error stated above.


Thanks!

[unskinnyboy]

http://forum.doom9.org/showthread.php?t=96984

Next time, :search:

[IceManTX]

Thanks!

Looks like there is a vmtoon now.

I understand if you might be frustrated with me not finding my answer using the search but I honestly have searched until my eyes are crossed. The search feature here is not very helpful for me most of the time. I just hope you understand that there is soooooooooo much information out there for a complete noob like me and only so many hours in a day to read and follow wild goose chases looking for a tiny tidbit of data that is current and helpful and pointing in the right direction.

It would be a nice thing to have some sort of list that shows all the outdated scripts/plugs so that someone just coming in has an inkling of a clue where to start and doesn't have to wonder endless paths to neverland. (I'm not suggesting you should compile such a list.....just observing that one would be waaaaaaay beneficial)

Even with following the link you provided I have spent almost 2 hours reading before finding all the vmtoon stuff and I still haven't even begun to implement it.

I very much do appreciate your quick response, it helped me immensely. I apologize if I troubled you. :(

[foxyshadis]

The line-darkening is all I use vmtoon for anymore. It's just too slow, when limitedsharpenfaster and warpsharp do sharpening and linethinning even better and faster; though they do need some tweaking to get the best out of them. mftoon and ssxsharpen were good ideas, but Didée gave us great ones. :p

I should add a redirect from mftoon to vmtoon in the wiki.

[IceManTX]

foxy,

All I really need it for is for line darkening as well. My picture is already pretty sharp and clean (I think). Is limitedsharpenfaster and warpsharp mostly for source material that is not so good or do you use those even when starting with great source?

I was just using vmtoon() syntax in its most generic form.

How would I implement it to just do line darkening?

thanks.

[unskinnyboy]

How would I implement it to just do line darkening?
Have you tried FastLineDarken() (http://forum.doom9.org/showthread.php?s=&threadid=82125)?

[foxyshadis]

FastLineDarken is older (and MaskTools 1, ick! :p), what's in vmtoon is a little better. It still needs more improvement though, it tends to "thin" way too much.

vmtoon(sharpen=false,thinning=0)

[IceManTX]

Ahhh...thank you both.

I had looked at the FastLineDarken but it appears it is older now as well! Is there no end to this damned constant progress!? (sarcasm)

What about the other two that you mentioned, foxy. Should I toy with those or are they for not so good source material?

I'm using these to do Avatar & Oban cartoon eps that my kid loves so it's a little different than the scifi eps I was learning on.

[foxyshadis]

LSF is an excellent all-purpose sharpener that works on pretty much anything, without making halos. See http://www.avisynth.org/mediawiki/wiki/LimitedSharpenFaster

Warpsharp has a unique flavor that usually works best at high resolution, especially with anime, since it thins out thick lines, but it can take a bit of tweaking to look the way you want. Tends to cause a bit of a stained glass effect near edges if it's too strong.

[IceManTX]

Is that web page down? I haven't been able to view it all day.

Do both LSF & Warp do the same thing? I mean, would I use one or the other or do you usually use them both simoutaneously?

You were right about vmtoon being s-l-o-o-o-o-w though. I left it running last night and when I got home I noticed that what had been taking me about 3 1/2 hours to encode had skyrocketed up to 11 hours!!

[foxyshadis]

Oh yeah, sorry, I only noticed avisynth.org was down later. The hosting company's having issues, but it should be back up in a day or two, apparently.

I usually use both. LSF is good for enhansing contrast and bringing out subtle detail, while warpsharp is good for line thinning and crispening (I'm a fan of lineless animation, too bad it's so rare), an msmooth-type solid area smoothing, and make for some extremely trippy effects when depth and bump are set too high. :p Warpsharp is especially good for animation though; try each separately, and then combined, to see what you like.

In case you're not familiar with MSmooth, it's very similar to the photoshop plugin Smart Smoother (that isn't that smart...).

[IceManTX]

foxy, you are a godsend and have provided me with great help. But I think I would need a PHD to understand how to use LSF & Warpsharp. I don't know if the wiki info is easier to follow than the actual threads on those items but I can tell you after reading them multiple times and even trying deperately to make heads or tails out of JeremyDuncan's newbie thread I don't have an inkling of a clue which files are actually needed, what syntax to use, or what god to pray to.

All I've actually succeeded in doing is confusing the hell out of myself and stumble upon yet another solution (seesaw) that may or may not be even close to what I need and which I'm sure will have me pulling out what little hair I have left on my bald head.

Here is what I have got, if you can tell me if this is miraculously correct or what I need to do to change it I would be indebted to you yet again.

A whole folder of plugs from DenoiseSharpen, MaskTools, warpsharp to multiple versions of RemoveGrain, Repair, Rsharpen and SSETools. I also have a LimitedSharpenFaster.avs file.

I have imported the .avs file and loaded several of the plugs and used this line of code:

LimitedSharpenFaster(ss_x=1.0, ss_y=1.0, Smode=3, strength=80, overshoot=1)

But I don't know if that line is correct (I just found it somewhere along the way) or if all the .dlls I'm using are needed and I don't want to load ones that I don't need. I have a P4 HT 3.2 GHz processor if that helps.

I haven't even begun to try to figure out warpsharp yet as I'm still dizzy from screwing with LSF for the last few hours.

Thanks yet again.

[unskinnyboy]

Looks like you got all the needed components of LimitedSharpenFaster. What is the issue now? Any errors? If not, that call should work fine (to know more about the values, just check the first post in the thread on LimitedSharpen). Did you ever got around to trying FastLineDarken yet? Even though it uses an older version of Masktools, for your purpose (line darkening), I suspect you will find it useful. You will never know unless you try it.

[IceManTX]

I am not getting any errors. Well except when I try to do the MT command but I suspect I need the MT_0.5 or some such that I read about. I don't think I can get that file though until the avisynth.org comes back up. I don't see links to it anywhere else.

From remming lines out one at a time I think all I need is the RemoveGrainSSE3.dll, masktools.dll, and the limitedsharpenfaster.avs script. Is the sse3 version the correct one for me? Am I right about just those files?

Yes, I did try the FLD the other day and it looked just as good (at least to me) for the line darkening. Now I'm off on another avenue exploring the world of sharpeners (damn that foxy :p) and stuck wondering if I ever should have started this journey.

Sooner or later I'm gonna have to actually start encoding these darn things and stop obsessively toying with the script. I've got DTV crap stacking up on my hard drive left and right. I know my 5 year old won't know if I used sharpenfast or sharpenslow or sharpensideways but dammit I'll know!!! And it'll drive me nuts.

Thanks UB.

[foxyshadis]

I stopped encoding in favor of relentlessly tinkering ages ago. :p

But it's worthwhile to spend a few hours up front on different topics, especially ones as important noise removal and sharpening, in order to learn how to quickly improve your work, so that later you can tweak each movie in only a couple minutes.

The only comment on the call is that ss_x and ss_y don't need to be used for encoding; they're lower there because realtime playback has stricter requirements. MT won't be useful for you. Smode 4 is also nice, but really tends to bring noise out badly.

The wiki really does help, I wrote descriptions of all parameters, and sample uses, along with links to requirements (masktools 2, removegrain 0.9 and pre-1.0) but I forgot a lot of it now. ^^

Warpsharp is easy though, the defaults are basically enough. bump=strength (128 default), bump=warping (128 again) if you need to tweak.

[IceManTX]

OK, I'm done. I can test no more. I guess my only questions left are the order of items in my script and the necessity of items.

I am using in this order:

colormatrix
tfm
tomsmocomp
tdecimate
crop
resize
vmtoon
LSF
warp
brightening (AGC)

Do I not need DegrainMedian or Fluxsmooth any longer with this current lineup? Am I doing too much? The video looks bright and clear but I don't want to over process it.

thanks.

(and don't you dare bring up another plugin or script, foxy, I need my life back)

[foxyshadis]

The way to tell if you're overprocessing is to encode a random couple minutes of it, then watch some other stuff for a while, before coming back and watch it. If it makes you go WOW you're on the right track, if you suddenly notice faces look like airbrushed models you need to turn smoothing and warpsharp down a bit.

Without any denoising, though, that sharpening would turn everything into a noisefest. :p

[IceManTX]

OK, just noticed that as I was watching some toons with my boy that I encoded there was a slight problem.

All the outlines are jagged. Well not so much jagged as 'dashed'....like this ---------------------

What would cause this effect and what would be a good way to remedy the problem?

thanks.

If you need a short clip as an example let me know and I'll post it.

[foxyshadis]

At least a screenshot would help.

[ChronoCross]

that sounds like poor deinterlacing to me. It's very common in anime.

[IceManTX]

I was wondering if it was the deinterlacing but it had worked so well for me on the scifi eps I had encoded that I just didn't know. Maybe I should do away with the TFM and just do the Toms by itself when I do the anime.

OK, I'll put something up when I get home from work tonight.

Either way, I'll have to live with it on that ep as I already deleted the source. But I definately need to fix it going forward.

thanks!

[IceManTX]

OK, here's 19 seconds of it.

TestStrip.avi (http://71.164.186.136:81/moviez/teststrip.avi)

Thanks for taking a look.

[unskinnyboy]

Can you post the unprocessed sample and also the script you used when you got this?

[IceManTX]

Unfortunately, I already deleted the source files before I noticed this problem. Upon first glance I thought all was well. It wasn't until I sat down and watched the whole ep that actually saw it.

Here's the script though:

# Script Calls
Import ("C:\Encoding\AviSynth 2.5\LSF\LimitedSharpenFaster.avs")
Import("C:\Encoding\AviSynth 2.5\vmtoon0.7.avs")

# SOURCE
mpeg2source("Avatar.d2v",info=3) .ColorMatrix(hints=true,interlaced=true)

# IVTC/Deinterlacing
TFM(order=1,mode=3,PP=1)
TomsMoComp(1,5,1)
tdecimate(hybrid=1)

# CROPPING
crop(8,4,700,472)

# RESIZING
LanczosResize(640,480)

# DENOISING
fluxsmoothst(7,7)

# Sharpening, Line Darken
vmToon(sharpen=false,thinning=0)
LimitedSharpenFaster(Smode=3,strength=80, overshoot=1)
warpsharp()

[foxyshadis]

You mean the cruel and unusual blending? Yeoutch. That's something only a source sample could give a hope of fixing. If I had to guess, I'd say it's the verticalfilter in tomsmocomp doing it - if whole frames are coming out hugely combed with it off, something is wrong in the chain. Maybe field order is incorrect; since mpeg2source sets it there's no reason to specify it normally.

Also, a way to more effectively use TFM with a deinterlacer is something like TFM(clip2=TomsMoComp(1,5,1),PP=5). When combed areas are detected it'll replace them with that. PP=1 is just wasting time if you don't use tdeint as a post processor.

[IceManTX]

Thanks!

OK, This Friday I will have a new ep of Avatar and I will get to try this all over again. I'll read over these filters again to try and understand them better before then. I'll also verify field order then but I'm 99% sure I got that right.

But let me know if I understand you correctly.

1) In the TFM line I don't need the 'order=1' part.

2) I need to combine my TFM/Toms line as you suggested instead of having them seperate.

3) I need to change to PP=5 because 1 is doing absolutely nothing for me since I'm using an external deinterlacer. (If this is the case then this may very well be why it looks like doo doo.)

Which brings me to 4 very important questions.

1> Is #1 above because my .d2v file already has that info when it is imported with mpeg2source or because mpeg2source assumes TFF?

2> If I am not working on hybrid material (like Stargate) should I even be using TFM at all or should I go straight to Toms?

3> If I did do away with TFM, could I still use Tdecimate without it? Every ep I have ever downloaded is always at 23.976 so naturally I do it to. But I never understood why this was done instead of leaving it at 29.97.

4> If I stick with TFM do I need this line: d2vpath = "c:\mysource.d2v"? I've read the documentation several times but I just don't understand the purpose of this line.

[foxyshadis]

1. The d2v has the field order, yes.

2, 3. It depends on what you have. Most films and most cartoons are still entirely or mostly telecined. TFM undoes the telecine without cutting the stream down, TDecimate does that. Deinterlacing a telecined source should only be done as a last resort to work with hybrid material, because it will look jerky, or as a post-processor like you used it. If every field is different, like most other television shows, sure, go ahead and Deinterlace.

4. Do you mean the TFM d2v option? You can use it, it might make processing faster and slightly more accurate if you do. If you just mean as a variable to pass to mpeg2source, it's no different from using the string directly, just a convenience.

I also mentioned that I think having the last parameter in Toms a 1 insted of 0 might be causing all that blending. In that case it's just hiding the symptoms of the real problem.

[IceManTX]

I wish this stuff would sink deeper than my short term memory. I feel I have read the ReadMe's to these filters about 30 times now trying to remember what each param means.

OK, I was referring to using TFM with d2v but your answer let me in on something I hadn't played with before. Variables!

Like this?
=================================
Myd2v="e:\temp\output\avatar.d2v"

mpeg2source(Myd2v)
=================================

Using it with TFM is more complicated for me but I love the thought of 'faster and more accurate'. At least as far as where to put the clip2 and PP=5 part I am struggling.

Would I do it like this or did I butcher the hell out of it?
=================================
mpeg2source(Myd2v)
tfm((d2v=Myd2v),clip2=TomsMoComp(1,5,1),PP=5)
tdecimate(mode=1,hybrid=1)
=================================

I also read about the vertical filter in Toms. I guess the only way for me to tell on that one is to try it both ways and see what happens. Also, should I set the first number to -1 for autodetect? Such as TomsMoComp(-1,5,0)?

[foxyshadis]

The TFM would be like:
TFM(mode=3,d2v=Myd2v,pp=5,clip2=TomsMoComp(-1,5,1))

I wonder if it might be TDecimate causing the blending, I just noticed the hybrid=1 argument... I'd check it without, unless it's known to have 60i or 30p at points. It'll certainly work, blending just requires a higher bitrate (or lowers quality at the same bitrate).

[IceManTX]

OK.

To better understand, by setting the hybrid=1 argument that is like when I encode Stargate where it is interlaced and progressive. But on the Avatar it is most likely pure interlaced so I would leave hybrid to default=0.

Is my logic correct? Or should I have tried hybrid=0 on Stargate as well?

(I can't wait until I start encoding my home movies and I have to develop yet another script)