MaskTools 2.0a48

[Didée]

You go here (http://avisynth.org/warpenterprises/#warpsharp), download the ZIP for "warpsharp package". From that archive, you extract *only* the warpsharp.dll file to your plugins folder. (Do not extract any other files, unless you're asking for problems!);)

I'm not quite sure if that dll is compatible with auto-loading. In case it's not, load it manually with LoadPlugin().

[Amateur]

i have warpsharp.dll in the proper folder and still no luck. even tried adding a manual load plugin but i get the same error.

it says it's from line 78 of mftoon. do i have to actually change the mftoon script?

[Amateur]

any more advice? i still haven't found a fix for this...

[Didée]

Hmh. Maybe you get trapped by the plugin overloading feature. Empty your plugin folder from ALL plugins that smell like "warpsharp". Then manually load the warpsharp.dll from the linked package. (Here, locally, I've renamed the DLL to "warpsharp_package.dll" to avoid confusion with other warpsharp DLLs.)

[Amateur]

so i got your warpsharp package and now it's saying there is no maskedmerge. guess i'll try adding back things one by one to see if i can ever get this to work

edit: after adding everything back to my warpsharp package it now says that there is no function named YV12Layer from mftoon line 93.

edit 2: well replaing my previous warpsharp with the one you linked me to seemed to fix my problem with vmtoon (used to give me lots of white spots in all of my lines). maybe i'll see how that helps me. would still like to fix my mftoon problem though if possible

[Didée]

Well, yeah ... YV12Layer belongs to an outdated revision of the outdated v1.x branch of MaskTools (YV12Layer was in v1.4, and has been removed in v1.5 of MaskTools ... while the whole v1.x branch usually isn't used anymore.)

Short of saying mftoon is outdated. Get a time machine and travel 7 years back.

[Amateur]

oh so i should just stick with vmtoon? is that the improved mftoon? the guides i have found are probably old in that case. maybe you could pm me a list of your usual filters so i know what to read up on? that would be awesome

[jmac698]

I have a request that would make my life a lot easier. I'd like logic arithmetic operators available in the polish notiation, and I'd like to set the rounding mode to also truncate.
Try adding two 16 bit numbers with mt_lut, it's really a pain! I can't get a high byte because /256 will round, and there's no AND, so my only choice is to subtract expr 256 % 256 / to do it. Now try testing for carry from adding two 16bit numbers. That's a huge expression.
It seems like you could just not limit &. ie 255 15 & (should) gives 15 (but now 1) and still 1 1 == 0 0 == ^ gives 1, 1 1 == gives 1, so why limit & to 1 bit?

[Gavino]

It seems like you could just not limit &. ie 255 15 & (should) gives 15 (but now 1) and still 1 1 == 0 0 == ^ gives 1, 1 1 == gives 1, so why limit & to 1 bit?
The issue here is that the arithmetic is all done on floating point, so how would you interpret 246.1 & 14.8 ? There is also the question of backwards compatibility.

One possibility would be to introduce a separate integer mode, controlled by a new boolean parameter, in which internal values are manipulated as integers rather than float.

[jmac698]

That's easy, & only takes integers so cast it first, with truncate. There is perfect backward compatibility, the inputs to boolean are only 0 and 1 and a full range & handles this case. I see no problem.

[Manao]

Done (and a meaningless text for the 5 chars rule) (http://manao4.free.fr/masktools-v2.0a47.zip)

[Gavino]

That's easy, & only takes integers so cast it first, with truncate. There is perfect backward compatibility, the inputs to boolean are only 0 and 1 and a full range & handles this case. I see no problem.
But currently "1 2 &" produces 1, whereas a binary 'and' will give 0. Fortunately, Manao has solved that by using a different symbol for the arithmetic operators (&u, &s, etc).
Done
Impressively fast response (and not for the first time)! :)

By the way, how do you type the symbol for xor?
In the docs, it looks like "°" - I've always wondered if that was a typo. :confused:

[Manao]

It's not really a typo. I just never noticed that while the html file opened as a text file shows °, it appears as ° once displayed by browser. I guess I'll come around fixing that with the next release.

[Gavino]

If I open mt_masktools.html in Notepad, I still see "°".
And I don't have a "°" on my keyboard either. :(

[jmac698]

First of all, thanks Manao!! This just made 3 of my scripts at lot easier to finish :) Second of all, arghh! I'll explain why in a minute.
The mathematical symbol for XOR is a circle with a + in it. I'd guess he chose the degree symbol as a close replica. The problem is, on some keyboards (even english ones) no one knows how to type it. The answer is, hold ALT, type 248 on the numeric keypad only, then release ALT.
Now the argh.. I can't even type the symbol, because the moment I touch another key after it, Firefox thinks I hit "Back". I don't know why. It's like the 4th time I had to type this :)
I would suggest another symbol with exactly the same function, so we can use either one. I can see XOR in my future because it has uses for finding edges and filling. How do people feel about ` (it vaguely reminds me of ~) Really, the only other choices on my keyboard are _ or @.
Again, do these symbols appear on your keyboards?

Btw for the documentation try ° because it doesn't work for me either. Must be a different codepage or whatever you call it.

Update: Wait!! Thanks for adding again, but this doesn't solve my problem. I need to x y ff00 &u 256 /, so this helped for only one line and in fact, I still have to try the old technique for all other cases :( You must think I'm crazy but I don't use this for making masks at all, I have many other uses which are mostly pure arithmetic on entire videos.

[Didée]

There are true per-bit binary operations available through mt_logic. Perhaps the operation could be split up in smaller steps, with a combination of mt_lutxy and mt_logic?

BTW, on a german keyboard the ° sign is very easy to type. Here's another: °there°. And °some°°more°. Y°i°p°p°i°e°h - a°y - e°y! :p

[jmac698]

Nope, that's not the point. I'm working with higher precision intermediate values. It can't be broken down by definition that it's not an 8bit value.
My value that I'm working with only exists temporarily in a subexpression.

[Manao]

Wait!! Thanks for adding again, but this doesn't solve my problem. I need to x y ff00 &u 256 /, so this helped for only one line and in factYour operation is lacking (at least) one operator. What exactly are you trying to do ? I can drop the (arbitrary) clipping of operands to [0..255] or [-128..127], and clip instead to Uint32, Int32 or even Uint64 / Int64 (going over that couldn't make sense, since Int64 has more precision than the integer part of a double)

[jmac698]

Thanks for being understanding manao. I agree the limiting was arbitrary, the other operators (+-*/) don't have such rule. 32bit would be fine for me. Another idea I had is to include bit shifting, like 512 8 >> equals 2. I'm not sure what is the best order here, maybe 8 512 >> makes more sense.
Btw, I'd really love to have 4 clip expressions, but I realize the table method would reach it's limits. I did something crazy and tried to compute a correlation completely in masktools, it's almost but not quite possible without 4 values. I wasn't a plugin programmer then. But, if you wanted to add a correlation function, that would be fantastic ;) "x y corr"
Like I said, I found 3 completely different uses to operate on video with large valued intermediate results. In these cases I need to return the high byte and low byte.
Here's what I wrote so far:

#Set up some numbers in a video
Global v1=blankclip(pixel_type="YV12").tweak(bright=-255,coring=false).tweak(bright=0,coring=false)#v1=0
Global v2=blankclip(pixel_type="YV12").tweak(bright=-255,coring=false).tweak(bright=1,coring=false)#v2=1
mt_lut(v2,yexpr=highbyte("x 255 +"))#long winded way to get int(x/256)
#messageclip(highbyte("x 255 +"))#Use AvsP to read pixel values for the answer as well.
# Change the custom status line to print YUV values.

function highbyte(string s) {
s+" "+s+" 256 % - 256 /"#Expression to get high byte, needed to avoid rounding of x/256.
#So I am doing (x-(x%256))/256. Try this to get the 3rd high byte - it's ridiculous!
#Also warning this doesn't work for values >65535
}

function var(int v, string s) {
string(v)+" "+s#Make expression by substituting one variable
}

function highvar(int v, string s) {
highbyte(var(v,s))#High byte of a one variable expression
}

function lowbyte(string s) {
s+" 256 %"#Low byte of an expression
}

function lowvar(int v, string s) {
lowbyte(var(v,s))#Low byte of one variable expression
}

A shift function or no limit boolean arthmetic, and correlation (I know that's a big request, but I have the formula for you as well) either feature would make me extremely happy right now.

[Manao]

Expression to get high byte, needed to avoid rounding of x/256What rounding ?

[jmac698]

v1=blankclip(pixel_type="YV12")
expr="128 256 /"#128/256=1 shows rounding. int(128/256)=0
mt_lut(v1,yexpr=expr)
subtitle(expr+"=")
ScriptClip("""
subtitle(string(AverageLuma,"%1.0f"),x=84)
""")

[Gavino]

Perhaps all you need is the addition of an 'int' operator ('floor' and 'ceil' might be useful too).

[jmac698]

Yes, that's one of the options I mentioned: rounding mode including also truncate, meaning floor.
Whichever is easier - floor, >>, unrestricted boolean.
>> becomes pointless if there's floor. And can you include an alternate ° in your next build?
Thanks.

[Manao]

I've added trunc, ceil, floor (round already existed), left & right shifts (signed and unsigned), and @ can be used instead of °. Finally, before binary operations, it doesn't clip to an 8-bits integer anymore, but to a 64-bits one.

[jmac698]

Fantastic! I will be testing, trust me :)
I haven't seen this level of uesr satisfaction since AvsP development, and what a great tool that is now...

[jmac698]

Is there a way to detect a minimum version 2.0a48 is loaded? Some way to get the version number?

[jmac698]

It's working. Are repeated subexpressions optimized? I have to repeat a long subexpression 4 times to do what I want.

250 x y 256 * + + 60160 > 60160 250 x y 256 * + + ? 4096 < 4096 250 x y 256 * + + 60160 > 60160 250 x y 256 * + + ? ? 255 &u
Isn't that crazy? :) There's two improvements to this; one is to allow variables "10 a = a 2 +" gives 12, or include a clamp command "250 16 235 clamp" gives 235 and "65535 64 60160 clamp 8 >>" gives 235.
In my example, I could dramatically reduce the code to:
250 x y 256 * + + 4096 60160 clamp 255 &u
Btw, I know my use case is weird but clamp is perfectly usable for 8 bit uses.

Also, being able to specify hex numbers would be nice too, 60160=0xff00

What do you think of these ideas;
210 255 235 softthresh#pixels above 210 are set to 235, untouched otherwise
16 23 20 softthreshn#posterized to 32 levels starting at 20, 28...
5 27 16 softthreshn#posterize to 8 levels
would need floating point for the softthreshn to get say 219 ranges divided evenly. With color you could do some cartoon recoloring or Sin City effects, like a range of red to pure red lipstick etc. You can't specify tint without two channels at once though... would make a really nice mask though. How would you do that now?
What about color swapping, you need at most to replace two colors at once, ie switch red with green in ranges, with or without setting to a single color. What about
40 48 138 144 44 142 recolor#make close to yellow into pure yellow ulow vlow uhigh vhigh ureplace vreplace
40 48 138 144 76 62 68 54 swapcolor#switch near yellow with near green ulow1 vlow1 uhigh1 vhigh1 ulow2 uhigh2 vlow2 vhigh2
and bluescreen effects:
208 110 216 118 x recolor#replace near blue with part of second clip
You could even replace specific shades with lutspa like functionality.. like if you had a sunset you could match all shades of it and replace with a new smoothly generated one based on y coordinate.

[kypec]

Though I don't use MaskTools in any other way than what is provided by more experienced users (-Vit-, LaTo, Dideé) I have to ask: what is wrong with ^ symbol for XOR operator? Seeing how many of you are struggling with ° and others are proposing @ for that purpose I just had to ask...

[jmac698]

It's used for exponentiation. The whole symbol set could be redone but that would break compatibility. @ is being used now in the latest alpha.

[jmac698]

Me again :) Rand would be really useful right about now, I'm working on dithering. I can work around it with another plugin, however.
I think low high RAND would be good. Clipping RAND is not the same; the probabilities of 16 and 235 would increase. Luckily I only need an 8bit random number today.

[Didée]

Luckily I only need an 8bit random number today.
At some point, the thing you really need is a compiler. ;)

[jmac698]

lol, true, true :)
It's a long term thing, steep learning curve. I find scripting more productive in general, because of the write/test cycle.
A compiler can't keep up with the rate I generate ideas - you're inspired for a week, and that's all you got to get something done.

[Manao]

If you want a rand, use mt_lutxy(clip, clip.addgrain).

[cretindesalpes]

I'm working on dithering.
Maybe you would be interested by my dithering package? See my signature below.

[IanB]

Remember when dealing with "random" elements in frame data the frame data must be the same every time you generate that frame.

Very early versions of addgrain() just grabbed a new random sequence for every GetFrame(N) call independant of N. This caused strange behaviour when accessing the same frames multiple times and only some were able to be cached. The fix was to remember the random key for each frame N, such that whenever frame N was rendered the exact same results could be returned.

Manao's solution, mt_lutxy(clip, clip.addgrain), is a good one because addgrain() is now well behaved and you get different randomness for each frame, but a given frame is always the same.

Remember all the mt_lux* are using static Look Up Tables that are created once in the filter constructor (compile time) and reused very fast in the GetFrame calls (run time). So having rand in the expression would still have the same LUT for every frame, which I suspect would be not quite what would be wanted. And as an added bonus for confusion repeated runs of the same script might produce different random results.

[Mini-Me]

Hey, can anyone point me to the documentation for mt_lutspa? It's not in the Readme with masktools-v2.0a48, and it's not at http://manao4.free.fr/MaskTools.htm or http://manao4.free.fr/mt_masktools.html. I've done a search on this forum, but the most I've found is a debate in this thread over placement of the "relative" parameter.

EDIT: I've pretty much figured it out now, but it'd still be nice to see the documentation so I can learn some more of the options. :)

[Manao]

There's an up to date mt_masktools.htm bundled in the zip package, in the directory masktools/documentation. I'll take care to place it in the root directory in the next releases.

[Mini-Me]

There's an up to date mt_masktools.htm bundled in the zip package, in the directory masktools/documentation. I'll take care to place it in the root directory in the next releases.

Thanks, Manao! :)

[Dogway]

with mt_edge, how can I shift one pixel without degrading chroma?

I use:
move = "
0 0 0
0 0 0
0 0 1 1"
mt_edge(move,0,255,0,255,U=3,V=3)

But chroma misaligns.

Also there is a possibility that mt_edge would leave black pixels when shifted?

[Manao]

If you want to do it with mt_edge, do move_luma = "
0 0 0
0 0 0
0 0 1 1"
move_chroma = "
0 0 0
0 1 1
0 1 1 4"
mt_edge(move_luma,0,255,0,255,Y=3,U=2,V=2).mt_edge(move_chroma,0,255,0,255,Y=2,U=3,V=3)You could also use one of the resize function of avisynth (they have crop parameters that ought to do what you want too).

[Dogway]

Thanks! it was worth asking as I see. Im mainly doing so to avoid resizing.
Now its aligned but I guess degrading can't be avoided.

What about the black border?


EDIT: upss! the chroma subsampling, got it.

[Nephilis]

İs it possible to add the laplacian-gaussian edge detection method in mt_edge function ?

[Didée]

blur(1).mt_edge("laplace")

[Gavino]

I've added trunc, ceil, floor (round already existed), left & right shifts (signed and unsigned), and @ can be used instead of °.
None of these new operators seem to be recognised by mt_polish().
trunc, ceil and floor are also missing from the body of the documentation (but are mentioned in the changelog).

[yup]

Hi all!

Please advice how create mask. If one pixel in horizontal line true (255) all pixel line could be true (255).

yup.

[Didée]

Does this work? (Can't try it right now...)

mask1 = whatever # this' the mask with some 255-white pixels in it

mask2 = mask1.mt_lut(Y=-255)
black = mask1.mt_lut(Y=0)

v1=interleave(mask1,black).assumefieldbased.assumetff.weave
v2=interleave(mask2,black).assumefieldbased.assumetff.weave

mt_hysteresis(v1,v2)
separatefields.selecteven.assumeframebased

[yup]

Does this work? (Can't try it right now...)

mask1 = whatever # this' the mask with some 255-white pixels in it

mask2 = mask1.mt_lut(Y=-255)
black = mask1.mt_lut(Y=0)

v1=interleave(mask1,black).assumefieldbased.assumetff.weave
v2=interleave(mask2,black).assumefieldbased.assumetff.weave

mt_hysteresis(v1,v2)
separatefields.selecteven.assumeframebased
Didée:thanks:
It is work.
I am not find full description mt_hysteresis.
Please advice.
yup.

[Didée]

Hehe, I never have used mt_hysteresis before. Now, by actually trying it, I get an all-white screen from the above script. Seems that mt_hysteresis forces full expansion for any pixel that's not exactly black. (?)

thresh = 254 # only pixels brighter than "thresh" will be considered
mask1 = whatever # this' the mask with some 255-white pixels in it
mask1a = mask1.mt_binarize(thresh)

mask2 = mask1.mt_lut(Y=-255)
black = mask1.mt_lut(Y=0)

v1=interleave(mask1a,black).assumefieldbased.assumetff.weave
v2=interleave(mask2,black).assumefieldbased.assumetff.weave

mt_hysteresis(v1,v2)
separatefields.selecteven.assumeframebased

Now it gives the expected result.

[yup]

Didée!
My mask contain 0 or 255 value after call
SDIad=mt_lutxy(SDI,SDIavg,"x "+THAVG+" y * > x 128 > & 255 0 ?",U=-128,V=-128)
Though I see gray pixel.
yup.

[Didée]

But my test gave correct result.

Wait - colorspace? Scanline masking smells like VHS capture. VHS capture smells like YUY2. My test was with YV12 video, in Avisynth 2.5.8.