MaskTools version to use with mfToon on a P4

[mg262]

Okay, just before I was about to post this I found a version that worked... but given how much of a headache it was I think this is worth posting for reference:
____________________________________________

I'm having trouble finding a version of MaskTools that will work with mftoon 0.52 on a P4 without hyperthreading. I'm using this as the reference post:

http://forum.doom9.org/showthread.php?p=429659#post429659

I'm using the versions of warpsharp and awarpsharp from there, and loading all three plug-ins using LoadPlugin. I've tried various versions of MaskTools and I get the following errors:

No function named YV12Layer
-- versions 1.5.6, 1.5.5, P4

Invalid arguments to function "Edge mask"
-- version with original mftoon

Illegal instruction
-- version 1.4.2, 1.4.1

The 1.5.5 and 1.5.6 are from @Manao's web site; the 1.4.2 from the thread I linked to, and the P4 version was on my PC (and flags as 1.5.1.0).

Version 1.4.9 works.

[Pookie]

Hey, thanks for posting that info. You saved me lots of troubleshooting time :)

[signal]

I'm having the exact same issues.

I've looked at the three versions of MaskTools on the avisynth filter collection site and don't see 1.4.9.

1.4.16, 1.5.6, and 1.5.8 are there.

Manao's site doesn't have any 1.4 versions on it.

Find the 1.4.9 version here: http://mf.creations.nl/avs/filters/

[Manao]

http://forum.doom9.org/showthread.php?p=764490#post764490

Enjoy

[signal]

Many Thanks Manao, I did try http://manao4.free.fr/ and the link to the 2.0a version, but the site appears to be down.

I've been looking at http://jourdan.madism.org/~manao (originally thought this was "the" site for masktools), but looks like the 2.0 versions aren't there.

[Didée]

Temporarily mirrored: MaskTools 2.0a17 (http://home.arcor.de/dhanselmann/_stuff/masktools-v2.0a17.zip)

[signal]

Cheers Didee.

[Manao]

OK, mf hinted strongly on IRC that the mfToon script wasn't as good as the vmToon script ( written by Vectrangle and him ).

So here is the modified vmToon script for the MaskTools v2 : http://manao4.free.fr/vmtoon-v0.74.avs

# pre-release vmToon v0.7
# used to be known as mfToon
# pre-release because vectrangle is in the opposite timezone of me ;)
# if you're wondering why 0.7 and not 0.6, mftoon 0.6 was never released

function mf_min(string a, string b)
{
return a + " " + b + " < " + a + " " + b + " ?"
}

function mf_max(string a, string b)
{
return a + " " + b + " > " + a + " " + b + " ?"
}

function mf_str_level(string x, int in_low, int in_high, int out_low, int out_high)
{
return mf_max(mf_min(x + " 16 - 255 * 219 / " + string(in_low) + " - " + string(in_high-in_low) + " / " + string(out_high - out_low) + " * " + string(out_low) + " + 219 * 255 / 16 +", "235"), "16")
}

function vmToon(clip input, int "strength", int "luma_cap", int "threshold", int "thinning", \
bool "sharpen", bool "mask", bool "show", string "showclip", int "ssw", int "ssh", int "xstren", \
int "xthresh") {

# vectrangle's stuff

str = String(Default(strength, 48) /128.) # line darkening amount, 0-255
lum = String(Default(luma_cap, 191)) # bright limit for line detection, 0-255 (255 = no limit)
thr = String(Default(threshold, 4)) # threshold to disable slight darkening (of noise) 0-255
thinning = Default(thinning,24) # line thinning amount, 0-255
thn = String(thinning /16.)

# mf's stuff

sharpen = Default(sharpen, true) # sharpening on/off
mask = Default(mask, true) # masking on/off
show = Default(show, false) # show the showclip or the output, true/false
showclip = Default(showclip, "lines") # show the lines or something else
ssw = Default(ssw, 4) # supersample factor horizontally, 0-inf
ssh = Default(ssh, 4) # supersample factor vertically, 0-inf
xstren = Default(xstren, 255) # xsharpening strength, 0-255
xthresh = Default(xthresh, 255) # xsharpening threshold, 0-255

mt_lutxy(input, input.blur(1.0), mf_str_level("x y - abs 128 +", 132, 145, 0, 255), y=3, u=1, v=1).blur(1.0).mt_lut(mf_str_level("x", 0, 64, 0, 255), y=3)

##
edgemask = last

# vectrangle's stuff

exin = input.mt_Expand().mt_Inpand()
diff = mt_lutxy(input, exin, YExpr= "y "+lum+" < y "+lum+" ? x "+thr+" + > x y " \
+lum+" < y "+lum+" ? - 0 ? 127 +")
linemask = mt_lutxy(diff.mt_Inpand(), "x 127 - "+thn+" * 255 +") \
.mt_Convolution("1 1 1", "1 1 1", Y=3)
thick = mt_lutxy(input, exin, YExpr= "y "+lum+" < y "+lum+" ? x "+thr+" + > x y " \
+lum+" < y "+lum+" ? - 0 ? "+str+" * x +")
thin = mt_lutxy(input.mt_Expand(), diff, YExpr="x y 127 - "+str+" 1 + * +")
(thinning == 0) ? thick : mt_merge(thin, thick, linemask, Y=3, U=1, V=1)
darkened = last

# mf's stuff

MergeChroma(Overlay(input, BlankClip(input, color=$00FF00), mask=edgemask))
lines = last

mt_merge(input, darkened, edgemask, Y=3, U=2, V=2)
masked = last

mask ? masked : mergechroma(darkened, input)
LanczosResize(width*ssw, height*ssh)
XSharpen(xstren, xthresh)
LanczosResize(width/ssw, height/ssh)
sharpened = last

showclip2 = Eval(showclip)
sharpen ? sharpened : mask ? masked : darkened
show ? showclip2 : last

}As a sidenote, some might be interested by the mf_min, mf_max and mf_str_level functions, when using luts

Speed gain will only come from the faster expand / inpand, and the removal of the useless chroma operations that were done.

[Didée]

Huhuuh, mf still testing people's patience with using 4-times-supersampled XSharpen() ... /*shrugs shoulders*/

BTW, it might be beneficial to replace "blur(1)" with "RemoveGrain(11)".
(Try yv12subtract(blur(1),RemoveGrain(11)).coloryuv(analyze=true) ;) )

[Manao]

I don't test scripts pertinency, I only blindly convert scripts for people to switch to v2 of the masktools :p

Btw, I'm amazed, you haven't found a bug for a long time now. What's happening ? I hope you're not sick :p