MaskTools 2.0a48

[vanessam]

Hi,
Mt don't work with version after v20.a18.

[vanessam]

Mt don't work with version after v2.0a18 ...

My script:
Mt("hqdn3d(3)LimitedSharpenFaster(ss_x=1.4,ss_y=1.4,Smode=3,strength=50,overshoot=7)
Converttoyv12()")

[Manao]

At all those having issues with masktools and mt, please have a look here, and see if you can help :

http://forum.doom9.org/showthread.php?p=776215#post776215

[Boulder]

I got alpha 23 to crash with this function (Didée's YLevels adjusted to 2.0) if I don't feed YV12 data in it:

function Ylevels(clip clp,
\ int "input_low", float "gamma", int "input_high",
\ int "output_low", int "output_high", bool "show_function")
{
input_low = Default(input_low, 0)
gamma = Default(gamma, 1.0)
input_high = Default(input_high, 255)
output_low = Default(output_low, 0)
output_high = Default(output_high, 255)
show_function = Default(show_function, false)

wicked = "x " +string(input_low)+ " - " +string(input_high)+ " " +string(input_low)+ " - / 1 " +string(gamma)+
\ " / ^ " +string(output_high)+ " " +string(output_low)+ " - * " +string(output_low)+ " +"

return( show_function ? clp.subtitle(wicked) : clp.mt_lut(Yexpr = wicked, U=2,V=2) )
}

http://img206.imageshack.us/img206/3181/masktools9uj.th.jpg (http://img206.imageshack.us/my.php?image=masktools9uj.jpg)

[Manao]

Ok, here is a new release that should hopefully work with MT :

http://manao4.free.fr/masktools-v2.0a24.zip

Changelog from alpha 23 to alpha 24 : * fixed : issues with MT.dll ( thanks tsp, Boulder, vanessam and all those who suffered the bug )
* fixed : check for YV12 colorspace, and report an error if it isn't ( thanks Boulder )
* speed up : median mode for luts ( once again, thanks to tsp )

Boulder : if that one still crashes when closing the script, I'll try to cook up another debug version with pertinent logs.

[Boulder]

It still crashes VDubMod. Also, the script works fine for the first couple of frames, then begins to show the frames in a random order. It looks like every other frame is in correct order, every other isn't.

[Manao]

S**t...

I'll have to think of the information that i need to output before making a new debug build. Meanwhile, can you :
- tell me if you're still using the same script than the one you post
- whether "crashes vdubmod" means it crashes when closing or while processing
- send me another debug log from the mt_masktools2.dll debug build. I would need only a few frames of logs, starting before the frame's order is disrupted and ending after.

[Boulder]

1. The same script I wrote when I emailed the log.

2. VDubMod vanishes after the script has been closed. Not immediately, but some seconds after that. If it's of any use, ffdshow's tray icon remains visible until VDubMod vanishes. Without the YLevels line, the icon won't stay there when the script is closed. (I use ffdshow's mjpeg for my captures)

3. Forgot to mention, mt_masktools2.dll produces correct frames but the vanishing VDubMod issue still remains. I couldn't make it crash while advancing through the frames at least for several hundred frames.

[foxyshadis]

mt_merge seems to have some corruption problems that occasionally make it grab the second clip from some random point it memory. Whatever it is it's definitely not the clip I'm feeding it. The first and the mask always look correct though. Usually it has extra black or green bleeding through the mask. I took a screen cap one time it happened:

http://foxyshadis.slightlydark.com/random/supclipd0000.jpeg

It never occurs when viewing the clips or masks alone, and isn't even remotely reproduceable (reloading usually fixes it, for a time; exiting and reopening vdub always helps keep it at bay for a while). Also I suspect that it's the cause of occasional "Cannot locate a decompressor for format "XVID"" that I get several reloads after it happens and have to reopen vdub to fix.

Oh, and a question, did you ever implement Clouded's suggestion of user-defined functions callable from the lut?

[Manao]

foxyshadis : what is the date of your avisynth.dll ? What happens if you upgrade to latest 2.5.6 ( iirc from october 2005 ) ?

Boulder : I still don't know what's happening to you. tsp told me he got no issues with the new dll, but we hardly had the time to speak, and I don't know how your script behaved on his computer. Perhaps more information will come during the day.

[foxyshadis]

12/31/05, 2.5.7-pre from tritical. Haven't seen it yet from 2.5.6a, but it usually only shows up once an hour or less. I never saw it happen with the debug, only the non-debug 23/24, so I can't get you any logs. =\ Then again, there's also the possibility that it could be a rare bug in eedi2, hard to tell.

Oh, an interesting oddity, probably related to boulder's: virtualdub doesn't close if any instances of masktools 2 are in scripts. It just sits dead in the process list sucking up memory. Tried several of the mt2 functions and they all do the same, even if they're not accessed; only commenting them out fixes. Both versions of avisynth.dll do this.

[Manao]

foxyshadis : for the wrong frames, I once encountered the issue, and it appeared that avisynth was giving me the wrong frames to process. That happened only with 2.5.6 from july, not 2.5.6 from october. It also didn't happen when I increased the SetMemoryMax value.

I discarded the issue as an avisynth's bug. It may be on my side, but I can't see how I could make avisynth give me the wrong frame ( yet, it's a possibility I don't discard )

I'll try with the script that was causing the bug on my side, just to see if it happens again with tsp's avisynth dll.

In any case, I don't think the bug is in eedi2.

The closing issue seems to be on my side though. I'll create another debug version. In the meantime, can you post your script ?

Oh, and a question, did you ever implement Clouded's suggestion of user-defined functions callable from the lut?No. However, I hinted some tricks to ease the writing of the luts : write functions such as :function mt_min(string "x", string"y")
{
return x + " " + y + " < " + x + " " + y + " ?"
}or likefunction mt_scale(string "x", string "old_min", string "old_max", string "new_min", string "new_max")
{
old_range = old_max + " " + old_min + " -"
new_range = new_max + " " + new_min + " -"
return x + " " + old_min + " - " + old_range + " / " + new_range + " * " + new_min + " +"
}

[foxyshadis]

Huh, it really doesn't like my main encoding machine and server (Athlon XP, isse/sse, server 2003 sbs sp1, latest official avisynth), everything came out garbled. Glad I don't have to give you the nasty script I was working on before, heh.


AVISource("tb14.avi").crop(0,4,0,-8)
s0=trim(0,129)
s1=trim(130,2216)
s2=trim(2217,0)

s1.blendcombs()

function BlendCombs(clip orig, bool "slow") {
assert(Defined(orig),"CLIP PLZ")
slow = default(slow,true)
orig

fe=mt_average(orig,orig)

cm=CombMask().bilinearresize(width/2,height/2).bilinearresize(width,height).mt_expand()

fm=mt_merge(orig,fe,cm.fitY2UV(),chroma="process")

return fm

}


I'm pretty sure it's happening in blendcombs, that's the only place with masktools and the crud is in the shape of the masks again. As soon as that machine is done encoding something else I'll strip the script down and retry to make sure.

(Yeah, that's what I ended up doing for the luts.)

Edit: Yup, came down to just this and still shows a mess all over with 2.5.6a. 2.5.7pre is unaffected as far as I can tell however.

[Manao]

Ok, I spotted another silly mistake. Thanks a lot for the script, foxyshadis, because it's indeed a real killer that systematically shows the issue :)

So, http://manao4.free.fr/masktools-v2.0a25.zip

Changelog from alpha 24 to alpha 25 : * added : new html documentation.
* fixed : wrong frame issue.
* fixed : mt_merge with luma=true.

[Boulder]

I did some quick testing and it looks like alpha 18 is the last one that works correctly for me with MT. The script itself doesn't crash but again, when the script is closed, ffdshow's icon remains in the lower right corner and after a while VDubMod closes by itself (after which ffdshow's icon disappears as well).

I tried with MT v0.41 plus the latest official Avisynth v2.5.6 and with MT 0.5 plus tsp's latest Avisynth build. Both behave the same way.

[Manao]

And here is ( hopefully ) the last bug fix. I overlooked Boulder's observation on the ffdshow icon tray not disappearing, which prevented me from recreating the bug on my side. Luckily for me, he insisted a second time on that very point :). So big thanks, Boulder

http://manao4.free.fr/masktools-v2.0a26.zip

Changelog from alpha 25 to alpha 26 : * fixed : avs closing issue

[Boulder]

Hehe, you wanted them bugs to crawl out :p

It seems to work perfectly, thank you!

[foxyshadis]

I think mt_binarize is broken. upper=true and upper=false are supposed to be inverses, right? A lot of pixels seem to fall through the cracks, though I can't tell what the pattern is.

[Manao]

Well spotten. It happened when threshold < max ( pixels of video - 128 ).

Fixed with the new version : http://manao4.free.fr/masktools-v2.0a27.zip

Changelog : * fixed : mt_binarize asm optimizations that borked with some thresholds

[Egh]

* chroma("") : allow to set the mode of both chroma channels in a more user-friendly way. Possible strings are "process" ( u=v=3 ), "copy" or "copy first" ( u=v=1 ), "copy second" ( u=v=4 ), "copy third" ( u=v=5 ), "none" ( u=v=1 ) and finally, a number "xxx" ( u=v=-xxx )


I think in the first instance it should be u=v=2, right? :P

[Manao]

Yes, but the documentation has been rewritten in html, and the error isn't in the html file :p

[Egh]

@Manao: great to know that, but in readme it's still like that :)

Also i wanted some clarifications concerning mt_edge.

Which kernel is best in quality? Is there a difference in speed between those and how much? And why despite some claims it seems that prewitt is rather imprecise?

I'm currently making my own script for edge enhancement, mostly targeted for hi-rezo things like 720p raws, and thinking which kernel to use. At the moment I think "roberts" is likely best for my purposes.

Also, can you please clarify what exactly is created by mt_expand().mt_inpand() command applied to original image? This kind of thing is used in FastLineDarken, which i rewritten from mt1.5.1 to mt2.0. So I wonder if the results now are same as before...

[Manao]

Lots of questions :)

Which kernel is best in qualityThey all serve different purposes.Is there a difference in speed between those and how much?Yes, there's a great difference. Roberts is the fastest, closely followed by sobel. Then comes laplace and min/max. Then comes the unoptimized one : cartoon, and prewitt.it seems that prewitt is rather impreciseCan you be a bit more explicit ? imprecise in what way ? edge position, or edge detection ? Any sample to show ?what exactly is created by mt_expand().mt_inpand()It's called a overture ( inpand.expand is a closure ). It's a common way of denoising, but it can have other uses. I don't remember well fastlinedarken script, so I don't know why it was used there. Anyway, Expand and mt_expand behave exactly in the same way, and so does Inpand and mt_inpand.

[Egh]

Well. It's not that prewitt is bad, it's probably better for some applications like LimitedSharpen. But for my purpose i.e. searching primarily for character's edges it's not good.

http://xs68.xs.to/pics/06076/mosk.no.png <-- original
http://xs68.xs.to/pics/06076/mosk.roberts.png <-- good but probably too thin lines
http://xs68.xs.to/pics/06076/mosk.sobel.png <-- very good :)
http://xs68.xs.to/pics/06076/mosk.prewitt.png <-- thick lines but lots of hits background, which i would like to avoid. So in short prewitt sees edges which are not good from anime point of view :) On some specific backgounds the difference between prewitt and sobel can be very substantial. Like when prewitt covers half of backround image with edges roberts/sobel hardly see any at all. So although roberts/sobel produce thinner edges, I correct that with single mt_expand operation and the result produced is much better compared to prewitt. But again, it's due to the task of correcting dark lines only.

Btw, if you're interested, that's the video sample with lines enhanced. http://rapidshare.de/files/13553656/h264.mkv.html I guess in due time i'll release a whole episode of this as well. Original raw is very good but lines are too thin and vague to my taste. mfToon is probably good but it's such a bulky monster :) The script i did works a bit slower than FLD() but still with good speed and main difference from otherwise superior FLD() is that I try to smooth the edges as well, since fld() makes them too pixelated imo. Since it's hdtv which then will be dully downsampled a bit (from 720p to 576p) i don't even use supersampling for edge correction.

Concerning that expand.inpand. That's how the same frame looks after such operation: http://xs68.xs.to/pics/06076/mosk.png
I think it's not only for denoising in FLD().

[Manao]

Thresholds exist to avoid false hits in the background. Play with thY1, thy2, in order to avoid false edge detection.

Applying inpand.expand on a mask also takes care of small false detection, while expand.inpand fills in the small gaps inside the edges. So inpand.expand.expand.inpand would improve the output ( and slow things down... )

Another possibility is to use mt_hysteresis. You give it two masks ( one with thin edges and no false detection, the other with thick edges but some false detection ), and it'll make one grow into another, giving the best of both world ( at least, it's supposed to ). There again, a slowdown is to be expected.

[Egh]

Thresholds exist to avoid false hits in the background. Play with thY1, thy2, in order to avoid false edge detection.

Another possibility is to use mt_hysteresis. You give it two masks ( one with thin edges and no false detection, the other with thick edges but some false detection ), and it'll make one grow into another, giving the best of both world ( at least, it's supposed to ). There again, a slowdown is to be expected.

No. It's not only small artefacts on background. It's a whole load of thick lines detected where i don't need them to be (not on that pix i gave, but I can show others where the diffence is really huge). So for the purposes of edge enhancement i don't think prewitt will do. Thresholds are not easily played with, since i need some specific values there. thY1 is set to 10 to avoid some small artefacts though.

And at the moment i use only mt_expand to a mask produced with sobel kernel, thus both filling small gaps and enlargening areas for additional processing. And that's rather fast. How fast (or slow) is hysteresis, btw?

Any ideas why FLD() uses that expand/inpand on the image?

P.S. why in your signature it's a26 whilst you already released a27?

[foxyshadis]

I recall a thread here, perhaps by clouded, wherein the effectiveness of different edgemasks was discussed. Mostly for animated stuff, both cel-shaded and painted. There are some that give even thinner lines than roberts, because it's getting only the outline's edge, not the outline as a whole (which I assume you're after). That's an important distinction, because if the outline is more than a couple pixels wide, it can be seen as a solid area with its own edges by more kernels. Hysterizing sobel into prewitt would probably help.

Then there are some (maybe they were in the thread, not sure) that are 5x5 or more, with I guess mt_luts or mt_convolution, but of course slooooooooooooow. I would link to the literature or the thread, but I can't find it right now.

[Richard Berg]

Bug in 2.0a27 package - run this script:

---
Version
LimitedSharpenFaster(dest_x = 640, dest_y = 480)
---

Lots of diagonal artifacts.

[Manao]

Works on my computer, if I add converttoyv12 ( version outputs RGB )

[Didée]

LimitedSharpen doesn't handle RGB input correctly. It never did, in all of its versions. Only YV12 and YUY2 input are safe to use.

[Richard Berg]

Ok. The version of LimitedSharpen I have works ok on that script.

[foxyshadis]

Manao, would you mind putting masktools html docs on your site as well, so it could be more easily linked to?

[Manao]

Done : http://manao4.free.fr/mt_masktools.html

[foxyshadis]

Posting again, you probably know why by now... Thanks for the docs though!

Looks like vmtoon shows a bug in some sources; I'd seen it reported before but now that I could repro it, I have a nice simple test case.

http://rapidshare.de/files/15864704/Inuyasha_Macroblocking.demuxed.m2v.html


mpeg2source("Inuyasha_Macroblocking.demuxed.d2v")
mt_Expand()
greyscale


It causes sliding diagonal lines on every frame, looks like some kind of off-by-8 error. Occurs with at least expand and inpand.

[Manao]

I still can't reproduce it :(.

Two things to try on your side : add a greyscale() in order the chroma not to trouble what you're seeing ( since with your script, chroma is completely randomized ), and try to add "mmx=false, isse=false, d3now=false" in mt_expand(), to see if results are still the same. If not, screenshots would be welcomed ( with them, with some luck, i might be able to find the possible bug ).

[foxyshadis]

I was using greyscale, so it wasn't chroma. isse=false fixes it. Oh, and the lutxy had nothing to do with it, just: mt_Expand() does it for me. Hm. (But not on all sources, especially smaller ones.)

http://foxyshadis.slightlydark.com/random/off.jpg

This is a core duo, I don't recall it ever happening on the Pentium M, or the AXP. (Testing now.) Hope it's not a processor bug. >_<

Edit: Tested both, no error on AXP, same error on P-M. Main difference is P-M is SSE2 and AXP is not.

Edit2: Tested with 2.5.6a and it does not show. The AXP uses it, which also explains that. I normally use 2.5.7-cvs from tritical, versions at least since january show this. So now I guess I should take the report over to tritical and IanB?

It only shows on all 720x480 sources, not 320x240 or 704x480 so far.

[foxyshadis]

From PM:

I'll look, but I'm guessing off hand that it is a pitch/width problem. In 2.5.6 planar frames by default get 16 byte aligned pitch on Y and 8 on chroma, but in 2.5.7 chroma gets 16 and Y gets 32 which on a 720 width frame means the pitch no longer equals the width (736 vs 720). Whereas on a 704 or 320 width frame pitch will still equal width since both are divisible by 32.

[Manao]

Ok, thanks again for that nice report, I found the bug as soon as I installed avs 2.5.7.

http://manao4.free.fr/masktools-v2.0a28.zip

Changelog :Fixed : asm optimizations for mt_inpand/mt_expand/mt_edge ( mostly )

[IanB]

As noted by Tritical, 2.5.7 defaults to mod 16 alignment (SSE2 ready) for the chroma planes and mod 32 (chroma * 2) for the luma planes. This feature was introduced in 2.5.6 but is not enabled by default.

This feature is controlled by the SetPlanarLegacyAlignment(clip, bool) control function, bool==True -> Legacy mod 8 chroma, bool==False -> Advanced mod 16 chroma.

2.5.6 developers should test their plugins for future compatibility with the feature by putting a SetPlanarLegacyAlignment(False) at the end of their script.

In 2.5.7 for non-conformant old pluggins the feature may be restored to legacy mode by putting a SetPlanarLegacyAlignment(True) after the call to the problem pluggin.

In 2.6.0 the default alignment for all planes is 16. The alignment of each plane is independant of the other planes. i.e. your code must get the pitch for each plane independantly, pitch>>=1 will not work! SetPlanarLegacyAlignment only effect YV12 chroma plane alignment, forcing it to be half that of the luma plane.

[redfordxx]

@Manao

Is the speed of
mt_convolution("1 1 1 1 1 1 1 1 0 0 0 0 0 0 0","1 1 1 1 1 1 1 1 0 0 0 0 0 0 0")
and
mt_convolution("1 1 1 1 1 1 1 1 0","1 1 1 1 1 1 1 1 0")
same?

Tnx

[Manao]

No, the filter isn't smart enough to see that zeroes are useless ( and won't, ever ).

[Manao]

Ok, I fixed the bug described here (http://forum.doom9.org/showthread.php?p=834822#post834822)

Changelog : * fixed : intern float to integer conversion that were badly rounding negative number.
That resulted in -1.0 been rounded into 0 (!). Most notably affected was mt_convolution.
Thanks redfordxx for pointing that out.

[redfordxx]

a=mt_luts(o1,o1, mode = "avg", pixels = "-1 0 1 0", expr = "y" )
b=mt_luts(o1,o1, mode = "avg", pixels = "0 -1 0 1", expr = "y" )
c=mt_luts(o1,o1, mode = "avg", pixels = "1 -1 -1 1", expr = "y" )
d=mt_luts(o1,o1, mode = "avg", pixels = "-1 -1 1 1", expr = "y" )
e=mt_luts(o1,o1, mode = "max", pixels = "1 -1 -1 1", expr = "y" )
f=mt_luts(o1,o1, mode = "max", pixels = "-1 -1 1 1", expr = "y" )

a,b,e,f works
c,d==>blankscreen in VD-M2

[redfordxx]

BTW: Courious:
Why mt_lutf has no pixels parameter?
It would be useful.
Workaround is mt_luts("y") followed by mt_lutxy but wouldn't it be faster?

[Manao]

Ok, bug found, expect a release tomorrow.

mt_lutf doesn't have a pixel parameter, because its purpose is to adapt a local mask according to the value of the property of the whole picture. I fail to see how a pixels parameter would help. Can you give me an example of what you were thinking about ?

Finally, speed shouldn't be considered when playing with luts / lutf. I didn't inted these filters to be fast, only to be tweakable. Their purpose is to allow scripters who don't know how to code to have tools to make interesting scripts. From that, one could expect interesting filter's idea to come up, which, in turn, could be implemented into real filters, that would be fast.

[redfordxx]

Can you give me an example of what you were thinking about ?
s=mt_luts(o1,o1, mode = "avg", pixels = "1 -1 -1 1", expr = "y" )
mt_lutxy(o1,s, expr = "x y > x y ?" )
could be same as
mt_lutf(o1,o1, mode = "avg", pixels = "1 -1 -1 1", expr = "x y > x y ?" )
which would reconstruct diagonal lines..

Their purpose is to allow scripters who don't know how to code to have tools to make interesting scripts. From that, one could expect interesting filter's idea to come up, which, in turn, could be implemented into real filters, that would be fast.Yeah, that's 100% me.
My SmoothDeblock is slow as hell (0.25fps:devil:) and all I can do is to make it even slower. But, if I have time and am lucky, there will be something working from it, and maybe some1 will c-peed up it...

[redfordxx]

Well - throwing ideas:
How about making

mt_freerectangle : int left(0), int top(0), int width(2), int height(2)

So for example
mt_freerectangle(0,0,8,8)
would cover 8x8block right-down from pixel x inclusive

[Manao]

Ok, new version is up : http://manao4.free.fr/masktools-v2.0a30.zip

Changelog : * fixed : luts was crashing if some pixels didn't have any neighbours in the pixels list.
* added : mt_freerectangle, mt_freeellipse and mt_freelosange helpers.

[Manao]

Concerning lutf, it doesn't work like you think. lutf collects information on the whole frame, and compute from that information a single value per frame, defined by the "mode". "avg", for example, will compute the average pixel value for the whole frame, while "range" will compute the difference between the max and the min on the whole frame.

Then, "expr" is a 2 variables lut which will be applied on the whole frame. In that lut, x is the value previously computed, while y is the value of the pixel. So I really can't add a pixel list parameter to lutf.

Concerning the mt_freeforms helpers, just a sidenote : mt_freerectangle(0, 0, 2, 2 ) will create a 3x3 rectangle ( 2 is included ), so if you want a 8x8 rectangle, use (0,0,7,7).

[redfordxx]

Concerning lutf, it doesn't work like you think.
Well I already know how it works, but that's why I proposed it as a change. If parameter pixels would be added, then
pixels="" ... current behavior
pixels=someshape ... behaviour like reverse luts

By the way it seems to me now that many operations of luts on square can be done faster with inpand,expand, lutxy, pointresize combo.

For example I am making a function to fill n*n pixels block with its max value with expand-pointrezize combo, because I don't know any other way how to do it. It has nice property that the larger n the faster. (Well, this fact also means that there should be faster solution;))

But generally there can be really awsome things done with mt. Nice.

Last question: only being curious, because for implementation is maybe late.
Once I realized that lut works strange and then I realized:
mt_lut("1","2","3")
is not same as
mt_lut(yexpr="1",uexpr="2",vexpr="3")

iirc the defaults in old masktools were different there were only 3 expr parameters. In new masktools are 4 expr. What was there a reason? IMHO something like
yexpr=default(yexpr,"x")
uexpr=default(uexpr,yexpr)
vexpr=default(vexpr,uexpr)
would be best....