MaskTools2 - pfmod

[pinterf]

MaskTools2, high bit depth support.

Forked from tp7's masktools2 repository.
Thanks to all earlier contributors.

Get the latest release from here (https://github.com/pinterf/masktools/releases)

This fork incorporates tp7's abandoned initial work on 16 bit support (using stacked and interleaved 8 bit hacks back in 2013), which he has never released, but proved to be an invaluable help with my initial steps porting this basic plugin.

All filters are available for all bit depths. Avisynth+ color spaces with alpha (YUVA/RGBA) are not supported. XP version does not support mt_polish.

This release may have benefits for 8 bit users as well, see change log.

As always, report back bugs, regressions. Thank you.

Useful links:

Masktools2 info:
http://avisynth.nl/index.php/MaskTools2

Article by tp7
http://tp7.github.io/articles/masktools/

Project:
https://github.com/pinterf/masktools/tree/16bit/

Original version: tp7's MaskTools 2 repository.
https://github.com/tp7/masktools/

Masktools 2.0.a48 forum:
https://forum.doom9.org/showthread.php?t=98985

[Sparktank]

Amazing stuff! Thanks for the continued work on all the recent projects :)

[real.finder]

Thanks

so we will need edit all old scripts that has lut for > 8 bit now (adding the new syntax)? or not?

[pinterf]

There is no intuitive way deciding whether a constant is scalable or not.
I checked the idea on the examples here: http://avisynth.nl/index.php/MaskTools2/mt_lutspa
At some samples I could not tell for the first sight with 100% confidence which constants are to scale and which are not. An averager /2 is surely not scalable.

[real.finder]

There is no intuitive way deciding whether a constant is scalable or not.
I checked the idea on the examples here: http://avisynth.nl/index.php/MaskTools2/mt_lutspa
At some samples I could not tell for the first sight with 100% confidence which constants are to scale and which are not. An averager /2 is surely not scalable.

The changes log are not clear for me, did you added new bitdepth parameter? and what the default of it? 8 bit?

[pinterf]

The changes log are not clear for me, did you added new bitdepth parameter? and what the default of it? 8 bit?
You don't have to provide it, it is automatically passed when the expression is evaluated. It is used internally in #B and #F but you can check for it using variable name 'bitdepth'

[real.finder]

You don't have to provide it, it is automatically passed when the expression is evaluated. It is used internally in #B and #F but you can check for it using variable name 'bitdepth'

I already get the idea of #B and #F, but my point is for the script that already existing which don't has #B and #F or any thing for bit depth, how will they work in more than 8 bit?

[pinterf]

Existing 8 bit scripts will work as the expression arithmetic says and they will be incorrect on higher bit depths, unless you do only and "x y + 2 /" which means the same (x+y)/2 for all formats. So if you don't take care of the constants, it won't give proper results.

If you subtract N, it will subtract exactly N, even if N=128 and you know that in 8 bit world it means the half range. But in 16 bit this 128 can be a rounder or anything that is meant to be exactly 128.

I think we have no reason to intuitively guess from a constant that it should be scaled or not.

[real.finder]

Existing 8 bit scripts will work as the expression arithmetic says and they will be incorrect on higher bit depths

so, that mean we will need to edit most existing scripts

[real.finder]

another thing, let say that we want to write in 16 bit or float for some reason and in the same time we want it to work in 8 bit or 10 bit etc... too

[yup]

Hi pinterf!

Please explain, all filter from Your rewritten plugin MT_NICE_FILTER?
Where I can find ready to use list? For example QTGMC script use a lot of function and any could declared before use (for better performance).
I think that if filter MT_NICE_FILTER one will be work better if declared like MT_NICE_FILTER, but not default MT_MULTI_INSTANCE.

yup.

[pinterf]

They are auto registering themselves as nice filter.
EDIT:
info added to the first page

[yup]

pinterf!
They are auto registering themselves as nice filter.
And no need additional commands for this? All work from box?

yup.

[DJATOM]

pinterf!

And no need additional commands for this? All work from box?

yup.

Yes, if filter is registering himself, your mt setting will be ignored unless you'll force it (force=true).

[pinterf]

another thing, let say that we want to write in 16 bit or float for some reason and in the same time we want it to work in 8 bit or 10 bit etc... too
We are at the very beginning of these questions, so you'd like an easy way to tell the expression evaluator the default bit depth of the used constants, which is now 8 bits to help the relatively painless transition of the next years.
And the autoscaler would take it into account, and it scaled up and down the numbers if the default bitdepth for scaling is e.g. set to 16 bit instead of 8

EDIT:
maybe the expression string would contain control mnemonics at the beginning, hinting the default bitdepth of the scalable constants if they are not meant to be 8-bit base.
e.g. expr = "i16 x 32768 #B -" where "i16" (in general: i8/i10/i12/i14/i16 for integer formats or f32 for 32 bit float) hints the expression evaluator that the base bit depth of the scalable constants is 16 bits and has to scale it accordingly with the #B or #F operator?

[kgrabs]

Wow, thanks a lot for this! Just having the luts is fantastic, it's really a great thing you've got here

I tried porting over Dehalo_alpha but after about 2 hours it seemed to be a little beyond me. It looks kinda like Dehalo_alpha(darkstr=100, brightstr=100).
I'll post my (incorrect) work if someone feels like figuring it out, and to make it easier the mt_lutxy expressions of the original Dehalo_Alpha, via mt_infix, too:

http://i.imgur.com/Vqx9vhf.jpg

#gives incorrect output
#assumes stack16 input
Function DeHalo_Alpha16(clip input, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss", int "srad", bool "cs", bool "uv")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )
srad = default( srad, 1 ) # srad=3 is DeHalo_Alpha_2BD
cs = default( cs, false )
uv = default( uv, true )

assumeY = IsY(input)

clps = assumeY ? input
\ : input.ConvertToY()
clp = clps.ConvertFromStacked(bits=16)

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr * 256.0)
BRT = string(brightstr * 256.0)
ox = input.width()
oy = input.height()/2
ssx = Round(ox*ss)
ssy = Round(oy*ss)
dsx = Round(ox/rx)
dsy = Round(oy/ry)

halos = clps.dither_resize16(dsx,dsy,kernel="bicubic",U=1,V=1).dither_resize16(ox,oy,1,0,kernel="bicubic",U=1,V=1).ConvertFromStacked(bits=16)
are = mt_lutxy(clp.DHA16_expand(srad),clp.DHA16_inpand(srad),"x y -",U=1,V=1)
ugly = mt_lutxy(halos.DHA16_expand(srad),halos.DHA16_inpand(srad),"x y -",U=1,V=1)
so = mt_lutxy(ugly, are, "y x - y 0.256 + / 65535 * "+LOS+" - y 65535 + 131070 / "+HIS+" + *",U=1,V=1)
lets = mt_merge(halos,clp,so,U=1,V=1)
remove = (ss==1.0) ? clps.dither_repair16(lets.ConvertToStacked(), 1, -1, -1).ConvertFromStacked(bits=16)
\ : clps.dither_resize16(ssx,ssy,kernel="lanczos",taps=3,U=1,V=1).ConvertFromStacked(bits=16)
\ .mt_logic(lets.mt_expand(U=1,V=1).ConvertToStacked().dither_resize16(ssx,ssy,kernel="bicubic",U=1,V=1).ConvertFromStacked(bits=16),"min",U=1,V=1)
\ .mt_logic(lets.mt_inpand(U=1,V=1).ConvertToStacked().dither_resize16(ssx,ssy,kernel="bicubic",U=1,V=1).ConvertFromStacked(bits=16),"max",U=1,V=1)
\ .ConvertToStacked().dither_resize16(ox,oy,kernel="lanczos",taps=3,U=1,V=1).ConvertFromStacked(bits=16)
them = mt_lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=1,V=1).ConvertToStacked()
them = cs ? them.DHA16_contrasharp16(clps) : them
return assumeY ? them : uv ? ytouv(DHA16_utoy(input), DHA16_vtoy(input), them)
\ : them.DHA16_ConvertToX(input)
}

Function DHA16_utoy(clip c){
try {
utoy(c)
} catch ( error_msg ) {
utoy8(c)
}
}
Function DHA16_vtoy(clip c){
try {
vtoy(c)
} catch ( error_msg ) {
vtoy8(c)
}
}
Function DHA16_ConvertToX(clip output, clip input){return Is420(input) ? output.ConvertToYUV420() : Is422(input) ? output.ConvertToYUV422(): Is444(input) ? output.ConvertToYUV444() : output.ConvertToYV411()}
function DHA16_contrasharp16(clip after, clip before){return after.Dither_add16(Dither_sub16(after, after.Dither_RemoveGrain16(mode=11, modeU=-1, modeV=-1), U=1, V=1, dif=true).Dither_Repair16(Dither_sub16(before, after, U=1, V=1, dif=true), mode=13, modeU=-1, modeV=-1), U=1, V=1, dif=true)}
function DHA16_expand(clip c, int expand){return expand > 0 ? DHA16_expand(c.mt_expand(U=1, V=1), expand-1) : c}
function DHA16_inpand(clip c, int inpand){return inpand > 0 ? DHA16_inpand(c.mt_inpand(U=1, V=1), inpand-1) : c}

[real.finder]

I tried porting over Dehalo_alpha but after about 2 hours it seemed to be a little beyond me. It looks kinda like Dehalo_alpha(darkstr=100, brightstr=100).
I'll post my (incorrect) work if someone feels like figuring it out, and to make it easier the mt_lutxy expressions of the original Dehalo_Alpha, via mt_infix, too:

http://i.imgur.com/Vqx9vhf.jpg


I already intend to port dehalo_alpha and other scripts (https://forum.doom9.org/showthread.php?t=174121) but wait for pinterf to made the final decisions, no need to hurry

[real.finder]

We are at the very beginning of these questions, so you'd like an easy way to tell the expression evaluator the default bit depth of the used constants, which is now 8 bits to help the relatively painless transition of the next years.
And the autoscaler would take it into account, and it scaled up and down the numbers if the default bitdepth for scaling is e.g. set to 16 bit instead of 8

EDIT:
maybe the expression string would contain control mnemonics at the beginning, hinting the default bitdepth of the scalable constants if they are not meant to be 8-bit base.
e.g. expr = "i16 x 32768 #B -" where "i16" (in general: i8/i10/i12/i14/i16 for integer formats or f32 for 32 bit float) hints the expression evaluator that the base bit depth of the scalable constants is 16 bits and has to scale it accordingly with the #B or #F operator?

why just not add new exprBit parameter? with 8 bit default

edit: but your one is ok too

[pinterf]

New release, full high bit depth support (10-16, float) for all filters, new flavour in expression syntax.

For details, see first post.

Enjoy Masktools2 v2.2.2 (20170223) (https://github.com/pinterf/masktools/releases/tag/2.2.2)

Change list from v2.2.1


-All filters work in 10,12,14,16 bits and float (except mt_logic which is 8-16 only)
- mt_lutxyza 4D lut available with realtime=false!
(4 GBytes LUT table, slower initial lut table calculation)
Allowed only on x64. Be warned, it would take minutes.
- mt_gradient 10-16 bit / float
- mt_convolution 10-16 bit / float
- mt_motion 10-16 bit / float
- mt_xxpand and mt_xxflate to float
- mt_clamp to float
- mt_merge to float
- mt_binarize to float
- mt_invert to float
- mt_makediff and mt_adddiff to float
- mt_average to float
- Expression syntax supporting bit depth independent expressions:
Added configuration keywords i8, i10, i12, i14, i16 and f32 in order to inform the
expression evaluator about bit depth of the values that are to scale by #B and #F operators.

[real.finder]

ok, just did small port to DeHalo_alpha, I think we need IsVideoFloat() in avs+


# 2017.02.25 1st release
# DeHalo_alpha with high bit support (now only up to 16, no float yet), need masktools 2.2.2, avs25 dropped by now
# modified dehalo_alpha: here with increased search-radius for validation

function DeHalo_alpha_2BD(clip input, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss", bool "cs", int "search_rade", int "search_radi")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )
cs = default( cs, false )
sre = default( search_rade, max(round(max(rx,ry)),3) )
sri = default( search_radi, sre )

ssispmt = Findstr(VersionString(), "AviSynth+") != 0 && Findstr(VersionString(), "r1576") == 0

sislumaonly = ssispmt ? input.isy() : input.isy8()

clp = sislumaonly ? input : ssispmt ? input.converttoy() : input.converttoy8()

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr)
BRT = string(brightstr)
ox = clp.width()
oy = clp.height()

halos = clp.bicubicresize(round(ox/rx),round(oy/ry)).bicubicresize(ox,oy,1,0)
are = mt_lutxy(clp .srdha2e(sre,1),clp .srdha2i(sri,1),"x y -",U=1,V=1)
ugly = mt_lutxy(halos.srdha2e(sre,1),halos.srdha2i(sri,1),"x y -",U=1,V=1)
so = mt_lutxy( ugly, are, "y x - y 0.001 #F + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *" )
lets = mt_merge(halos,clp,so,U=1,V=1)
remove = (ss==1.0) ? clp.repair(lets,1,0)
\ : clp.lanczosresize(round(ox*ss),round(oy*ss))
\ .mt_logic(lets.mt_expand(U=1,V=1).bicubicresize(round(ox*ss),round(oy*ss)),"min",U=2,V=2)
\ .mt_logic(lets.mt_inpand(U=1,V=1).bicubicresize(round(ox*ss),round(oy*ss)),"max",U=2,V=2)
\ .lanczosresize(ox,oy)
them = mt_lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=2,V=2)

bb = cs ? them.removegrain(11) : nop()
xD = cs ? mt_makediff(bb,bb.repair(bb.repair(bb.medianblur(2,-333,-333),1),1)).mt_lut("x range_half - 2.49 #F * range_half +") : nop()
xDD = cs ? mt_lutxy(xD,mt_makediff(clp,them),"x range_half - y range_half - * 0 < range_half x range_half - abs y range_half - abs < x y ? ?") : nop()
them = cs ? them.mt_adddiff(xDD,U=2,V=2) : them

them = sislumaonly ? them : ssispmt ? CombinePlanes(them,input,planes="YUV",sample_clip=input) : ytouv(input.utoy8(),input.vtoy8(),them)

return( them )
}

Function srdha2e(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : srdha2e(i.mt_expand(), dr_rad, count+1)
}

Function srdha2i(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : srdha2i(i.mt_inpand(), dr_rad, count+1)
}

[pinterf]

BitsPerComponent=32? I hope it exists. I see you the script is using mt_logic. Is this the reason for checking float?
Ps.: I imagine how slow it can be with realtime lutxy :)

[real.finder]

BitsPerComponent=32? I hope it exists. I see you the script is using mt_logic. Is this the reason for checking float?
Ps.: I imagine how slow it can be with realtime lutxy :)

BitsPerComponent=32 work, but as there are already IsAudioFloat() I think it will make things more easy and clean

yes it's because mt_logic, and if mt_logic work with Float in future I think the expr need some special edit for Float, so IsVideoFloat() will make that less hard

[pinterf]

min and max. That makes sense in float.
Are there many script that are using explicite threshold parameter value? Such values are treated as-is, no scaling there.

[real.finder]

min and max. That makes sense in float.
Are there many script that are using explicite threshold parameter value? Such values are treated as-is, no scaling there.

didn't check, but I think yes, Didée alone did a lot of functions and most them use lut

[pinterf]

I'm thinking on not the expr in lut-like filters, but on the mt_xxxx direct parameters, e.g. thY1, thC1 in mt_edge, which are subject to scale and it is now the script writer's task to give properly scaled numbers for them.

[real.finder]

I'm thinking on not the expr in lut-like filters, but on the mt_xxxx direct parameters, e.g. thY1, thC1 in mt_edge, which are subject to scale and it is now the script writer's task to give properly scaled numbers for them.

for this, I think it's better to add ParametersBit parameter with 8 (or even better "i8") default

[real.finder]

did another port


# VHSHaloremover with high bit support, need masktools 2.2.2, avs25 dropped by now
# 2017.02.24 1st release
Function VHSHaloremover(clip input, int "xradius", int "yradius", int "darkthr", int "brightthr", float "blurmix")
{
ssispmt = Findstr(VersionString(), "AviSynth+") != 0 && Findstr(VersionString(), "r1576") == 0
sislumaonly = ssispmt ? input.isy() : input.isy8()
sislumaonly ? input : ssispmt ? input.converttoy() : input.converttoy8()
h = Height ()
w = Width ()
xradius = Default(xradius, 8)
yradius = Default(yradius, 1)
darkthr = Default(darkthr, 122)
brightthr = Default(brightthr, 134)
blurmix = Default(blurmix, 0.8)
cy = BicubicResize (w / xradius, h / yradius, 1, 0).BicubicResize (w, h, 1, 0)
cmd = mt_makediff (cy)
cm = cmd.mt_lut ("x "+String(brightthr)+" #F > range_max x "+String(darkthr)+" #F < range_max 0 ? ?")
cr = last.Merge (cy, blurmix)
mt_merge (last, cr, cm, y=3, u=2, v=2)
sislumaonly ? last : ssispmt ? CombinePlanes(last,input,planes="YUV",sample_clip=input) : ytouv(input.utoy8(),input.vtoy8(),last)
}


it should work with float but it gave me wrong result, is it bug in #F? or there are no float scale yet?

[pinterf]

Strange, when I replace
cr = last.Merge (cy, blurmix)
with
mask= last.mt_lut("f32 "+String(blurmix)+" #F")
cr = last.mt_merge (cy, mask, luma=true, y=3, u=3, v=3)
(is it o.k.?)
Then it works.
edit: arrrrgh. There was a bug in float merge (in Avs+ r2420 and before)

[martin53]

I currently look for steps towards a hdracg replacement (http://forum.doom9.org/showthread.php?p=1796949#post1796949) (the former paviko hdr contrast enhancer), since paviko did not release recent sources and cannot be contacted.
And I'd prefer a combination of basic plugin-powered building blocks used in a script function at least for the design phase, when experiments are more valuable than performance, over the closed plugin-does-all approach.

The 1st essential process for the paper hdragc was based on, is a special luma average of surrounding pixels - but restricted to those pixels with not too different luma from the center pixel.

mt_luts() can already yield the average of pixels' environments, but not with the stated restriction.
This could be solved by not using implicit weight=1 for all pixels, but using a weight expression when calculating the mt_luts() "mode" operation.
So mt_luts() then would e.g. have four new parameters string wxpr("1"), string ywxpr("1"), string uwxpr("1"), string vwxpr("1").
For the described task, wxpr would be e.g. mt_polish("y < x - 5 || y > x + 5 ? 0 : 1"), or something smoother like mt_polish("exp(-((x-y)/5)^4)").

Please note that also for modes "min" etc, an integer weight can be useful. E.g. with wxpr=mt_polish("y < x -10 ? 0 : 1") the darkest pixel in the environment can be found that is not darker than 10 increments below the center pixel.

I'm mostly asking pinterf, since you already studied the masktools plugin thoroughly: Do you see my point, and think you might add these parameters, or maybe a simpler/more powerful alternative?

(NOTE: Don't bother with me using the "||" OR operator for the example while masktools expect "|" instead - I know that and additional masktools operators "||", "&&" doing the same as "|" and "&" would have been a fine idea all the years )

[tormento]

ok, just did small port to DeHalo_alpha, I think we need IsVideoFloat() in avs+
What is m4_dham?

[pinterf]

I was so happy when I finally found out how plain mt_luts was working :) O.k., reading and trying to understand.

This would imply a second set lof 2D lut tables (or realtime expressions), where x is the center (current) pixel, and y is the pixel from the neighborhood.
In your idea, when we are e.g. in "average" mode, we are accumulating weighted sums, then this sum is divided by the sum of weights at the end.
And we find out something for the situation where the sum of weights is zero.

Now that I'm typing this post, I understand better the logic behind it.

[ajp_anton]

Would it be possible to add an alias to the ° xor symbol that's more common on keyboards? For example just being able to write "xor".

[pinterf]

More common :)
I think, yes. I used the copy paste method which is only one step from the perfection.

[real.finder]

What is m4_dham?

it was for resize limit in avs25, I forget to remove all of it, just did another update for the script

[pinterf]

Would it be possible to add an alias to the ° xor symbol that's more common on keyboards? For example just being able to write "xor".
Checked the source, you can use the @ sign instead. (and @u, @s)

[tormento]

it was for resize limit in avs25, I forget to remove all of it, just did another update for the script

Tried but did not gave me expected results with 28 days after:

http://i.imgur.com/47tc972t.jpg (http://imgur.com/47tc972)

Is the white shade around contrasted objects the "halo" you meant to remove with this script or it has another name?

[real.finder]

Tried but did not gave me expected results with 28 days after:

http://i.imgur.com/47tc972t.jpg (http://imgur.com/47tc972)

Is the white shade around contrasted objects the "halo" you meant to remove with this script or it has another name?

yes, try play with the parameters, the output should be same with original dehalo_alpha so this will be off topic, unless you see big difference between with 8 bit and more than 8 bit

[martin53]

This would imply a second set lof 2D lut tables (or realtime expressions), where x is the center (current) pixel, and y is the pixel from the neighborhood.
Yes. That's what I guess, too.
In your idea, when we are e.g. in "average" mode, we are accumulating weighted sums
And we find out something for the situation where the sum of weights is zero.
Right. There is already a similar faulty situation today: When the 'pixels' parameter contains no pixel, then there is already 0/0, although for the whole frame. Checked mt_luts() currently returns "x" in this case
colorbars(pixel_type="YV12")
mt_luts(last, last.invert, mode="avg", pixels="")

Thank you for reading and making familiar with the matter

[pinterf]

New release.

Download Masktools2 v2.2.3 (https://github.com/pinterf/masktools/releases/tag/2.2.3)

This version now has full support for popular 8-32 bit formats (though still no YUVA/RGBA), partial AVX/AVX2 for those rare non-SSE2 users :)

There is a new addition to mt_luts for martin53. See first post. Once it is finalized, I hope, martin53 will help in the documentation (and answering all questions), because my trial of describing the new features in English needs polishing :)

Changelog
**v2.2.3 (20170227)**
- mt_logic to 32 bit float (final filter lacking it)
- get CpuInfo from Avisynth (avx/avx2 preparation)
Note: AVX/AVX2 prequisites
- recent Avisynth+ which reports extra CPU flags
- 64 bit OS (but Avisynth can be 32 bits)
- Windows 7 SP1 or later
- mt_merge: 8-16 bit: AVX2, float:AVX
- mt_logic: 8-16 bit: AVX2, float:AVX
- mt_edge: 10-16 bit and 32 bit float: SSE2/SSE4 optimization
- mt_edge: 32 bit float AVX
- new: mt_luts: weight expressions as an addon for then main expression(s) (martin53's idea)
- wexpr
- ywExpr, uwExpr, vwExpr
If the relevant parameter strings exist, the weighting expression is evaluated
for each source/neighborhood pixel values (lut or realtime, depending on the bit depth and the "realtime" parameter).
Then the usual lut result is premultiplied by this weight factor before it gets accumulated.

[martin53]

- new: mt_luts: weight expressions
- wexpr
- ywExpr, uwExpr, vwExpr

:thanks: will soon show usefulness (hopefully) in 'hdragc alternative' thread :)

[pinterf]

:thanks: will soon show usefulness (hopefully) in 'hdragc alternative' thread :)
I hope it will be working the way you were proposing, it worked nicely with fix constants though, wexpr = "0.1", "1.4", "10.8", this latter multiplier was enough to show up nasty overflow errors. Good luck, I'm looking forward to your experiments.

[real.finder]

I made DeHalo_alpha work in float, but I don't think it is the perfect way, cuz we have extra lut for float now to remove the negative pixels that occurred in the mt_lutxy


# 2017.02.27 1st release
# DeHalo_alpha with high bit support, need masktools 2.2.3, avs25 dropped by now
# modified dehalo_alpha: here with increased search-radius for validation

function DeHalo_alpha_2BD(clip input, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss", bool "cs", int "search_rade", int "search_radi")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )
cs = default( cs, false )
sre = default( search_rade, max(round(max(rx,ry)),3) )
sri = default( search_radi, sre )

ssispmt = Findstr(VersionString(), "AviSynth+") != 0 && Findstr(VersionString(), "r1576") == 0

sislumaonly = ssispmt ? input.isy() : input.isy8()

clp = sislumaonly ? input : ssispmt ? input.converttoy() : input.converttoy8()

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr)
BRT = string(brightstr)
ox = clp.width()
oy = clp.height()

halos = clp.bicubicresize(round(ox/rx),round(oy/ry)).bicubicresize(ox,oy,1,0)
are = mt_lutxy(clp .srdha2e(sre,1),clp .srdha2i(sri,1),"x y -",U=1,V=1)
ugly = mt_lutxy(halos.srdha2e(sre,1),halos.srdha2i(sri,1),"x y -",U=1,V=1)
so = clp.BitsPerComponent==32 ? mt_lutxy( ugly, are, "y 255 * x 255 * - y 255 * 0.001 + / 255 * "+LOS+" - y 255 * 256 + 512 / "+HIS+" + * 255 /" ).mt_lut("x 0 < 0 x ?")
\ : mt_lutxy( ugly, are, "y x - y 0.001 + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *" )
lets = mt_merge(halos,clp,so,U=1,V=1)
remove = (ss==1.0) ? clp.repair(lets,1,0)
\ : clp.lanczosresize(round(ox*ss),round(oy*ss))
\ .mt_logic(lets.mt_expand(U=1,V=1).bicubicresize(round(ox*ss),round(oy*ss)),"min",U=2,V=2)
\ .mt_logic(lets.mt_inpand(U=1,V=1).bicubicresize(round(ox*ss),round(oy*ss)),"max",U=2,V=2)
\ .lanczosresize(ox,oy)
them = mt_lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=2,V=2)

bb = cs ? them.removegrain(11) : nop()
xD = cs ? mt_makediff(bb,bb.repair(bb.repair(bb.medianblur(2,-333,-333),1),1)).mt_lut("x range_half - 2.49 #F * range_half +") : nop()
xDD = cs ? mt_lutxy(xD,mt_makediff(clp,them),"x range_half - y range_half - * 0 < range_half x range_half - abs y range_half - abs < x y ? ?") : nop()
them = cs ? them.mt_adddiff(xDD,U=2,V=2) : them

them = sislumaonly ? them : ssispmt ? CombinePlanes(them,input,planes="YUV",sample_clip=input) : ytouv(input.utoy8(),input.vtoy8(),them)

return( them )
}

Function srdha2e(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : srdha2e(i.mt_expand(), dr_rad, count+1)
}

Function srdha2i(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : srdha2i(i.mt_inpand(), dr_rad, count+1)
}

[martin53]

Hi pinterf,

possible thing to do with high bitdepth:
I think the mt_polish() helper function might not yet know the new predefined high bitdepth constants?

E.g. mt_polish("x < 100 ? x : range_max") should return
"x 100 < x range_max ?" but returns only the invalid expression fragment "x 100 < x".

[tormento]

New release.Download Masktools2 v2.2.3 (https://github.com/pinterf/masktools/releases/tag/2.2.3)


github-cloud.s3.amazonaws.com down :(

[martin53]

expression-weighted mt_luts() works well! :thanks: I am struggling a bit with a good working example (is more complicated than I expected), will publish soon.

github-cloud.s3.amazonaws.com down :(
Try again? Works fine here

[pinterf]

Hi pinterf,

possible thing to do with high bitdepth:
I think the mt_polish() helper function might not yet know the new predefined high bitdepth constants?

E.g. mt_polish("x < 100 ? x : range_max") should return
"x 100 < x range_max ?" but returns only the invalid expression fragment "x 100 < x".
Uh-oh. Will look at it.

EDIT:
Thank you, fixed. Added the 'a' letter of mt_lutxyza, names of the scaling helper constants, the #B and #F operators (they should appear as a one operand function e.g. #F(255)).
And there were other missing operators as well, mainly bit-operators (&u, |s, etc...), and the "@" (synonym for the never-found-degree-sign-for-xor).
No release yet.

[real.finder]

hi pinterf, what about add something (option) to clamp to 1.0-0.0 in float in lut?

and what range_size will do in float? will return 1.0?

[pinterf]

hi pinterf, what about add something (option) to clamp to 1.0-0.0 in float in lut?

and what range_size will do in float? will return 1.0?

Range_size is 1.0.

Though it is a basic rule that floats are not clamped, but sometimes, in known situations it is necessary, if we want to mimic the algorithm of the integer color spaces.

I was thinking on the problem, after I looked at your script with if float do_clamped_workaround else do_normal.

One solution can be an extra parameter for all lut functions, another workaround would be to introduce a keyword that can be used inside the expression similar to the i8..i16,f32.
"clamp_f" will only affect and clamp the result to 0..1 when the input clip format is float.

And this propagates another problem, Vapoursynth float format is -0.5..+0.5 for chroma, which should be the normal representation in Avisynth+, but for historical reasons it is 0..1.0 for all planes.

If once in the future it would change then ... I don't know..

And this means that for chroma things the clamping should occur on +/-0.5, and it needs another hinting keyword, clamp_f_chroma.

I wonder how VS users handle the 'let's write full generic scripts' situation, perhaps they don't care it because they don't mix integer and float together, or branch the relevant parts of the scripts?

[real.finder]

clamp_f sound good, and if it (clamp_f) make any x and y etc... from 1.0-0.0 to 255.0-0.0 for process like 8 bit then back to 1.0-0.0 will be better and we don't worry about do something special for float, in other word we will have one expr for both

and that also mean clamp_f will make #F and Range_size etc... keep or convert to like 8 bit expr (255.0-0.0)

[pinterf]

Why do you need that float input autoscaling? The usual #B and #F are working properly for the base float range.

[real.finder]

Why do you need that float input autoscaling? The usual #B and #F are working properly for the base float range.

yes, usual #B and #F work but some expr that use range_size (dehalo alpha) and range_max both return 1.0 in float and that will affect multiplication operations

[pinterf]

clamp_f sound good, and if it (clamp_f) make any x and y etc... from 1.0-0.0 to 255.0-0.0 for process like 8 bit then back to 1.0-0.0 will be better and we don't worry about do something special for float, in other word we will have one expr for both

and that also mean clamp_f will make #F and Range_size etc... keep or convert to like 8 bit expr (255.0-0.0)
I have to understand why multiplication is affected before we make changes in syntax that will probably held till the end of the XXIth century. The other way is more logical to me that we keep everything normalized to the 0..1.0 range. (but anyway, it's only the question of being documented)

[real.finder]

I have to understand why multiplication is affected before we make changes in syntax that will probably held till the end of the XXIth century. The other way is more logical to me that we keep everything normalized to the 0..1.0 range. (but anyway, it's only the question of being documented)

y x - y 0.001 + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *

is ((((y-x)/y+0.001)*range_max)-los #F)*(((y+range_size)/512 #F)+his)

if we have x=0.4 y=0.5 (float)
then we have 0.3802586358762255 (96.96595214843751 in 255)

if we did it in 255.0-0.0 (x=102, y=127.5) then we have 0.0048961718903666 (1.248523832043474 in 255)

[martin53]

working example (is more complicated than I expected), will publish soon.

a bit late - link to example (https://forum.doom9.org/showthread.php?p=1799122#post1799122)

[pinterf]

y x - y 0.001 + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *

is ((((y-x)/y+0.001)*range_max)-los #F)*(((y+range_size)/512 #F)+his)

if we have x=0.4 y=0.5 (float)
then we have 0.3802586358762255 (96.96595214843751 in 255)

if we did it in 255.0-0.0 (x=102, y=127.5) then we have 0.0048961718903666 (1.248523832043474 in 255)
I see. Why is 0.001 not scaled?

Meanwhile I had an idea that instead of having #B and #F operators inside the script, we would scale the inputs (not only floats) instead, to a desired bitdepth. But in this case we'd have to specify for each input, what method they are to be scaled with (e.g. bit-shifts or full-scale stretch.) if they are different from the logical default method.

For YUV the default would be the shift method (e.g. x << 8), for rgb inputs: full-stretch (e.g. x*65535/255), for greyscale:??? we don't know, they can be masks (full-stretch needed) or a YUV plane (bit-shift is needed).

Anyway, it takes one conversion at the input (per variable), and one at the output for the result. (if input's bitdepth is different that is specified).

[real.finder]

I see. Why is 0.001 not scaled?



I did scale it back then but it make 10-16 somehow wrong, and in float it not did anything right too cuz there are range_size and range_max problems

Meanwhile I had an idea that instead of having #B and #F operators inside the script, we would scale the inputs (not only floats) instead, to a desired bitdepth. But in this case we'd have to specify for each input, what method they are to be scaled with (e.g. bit-shifts or full-scale stretch.) if they are different from the logical default method.

For YUV the default would be the shift method (e.g. x << 8), for rgb inputs: full-stretch (e.g. x*65535/255), for greyscale:??? we don't know, they can be masks (full-stretch needed) or a YUV plane (bit-shift is needed).

Anyway, it takes one conversion at the input (per variable), and one at the output for the result. (if input's bitdepth is different that is specified).

I know, #B and #F better for most cases, so I ask for do 255.0-0.0 in float with clamp_f (special cases) only, so it will be special method for special cases

edit: #B and #F also will have effect if the expr is 10-16 bit (i10, i12 etc...)

[martin53]

y x - y 0.001 + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *


Can you please provide LOS and HIS so I can review the whole calculation?
I transformed

y x - y 0.001 + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *

manually to

(y-x) / (y+0.001) * range_max - #F(LOS) * ((y+range_size) / #F(512) + HIS)

and I'm wondering if HIS sholuld also be #F(HIS).

@pinterf: I used the helper function mt_infix() to check my transform. And I saw it transform the #F operator in a way that surprised me.
mt_infix("17 #F") is #F(17,133168,8). My question is: Where does the helper function get the 2nd and 3rd parameter from? It looks like ~2^17 and 8 bits. But the helper function works without any bit depth context; it is just a string rearranger.

Edit: mt_polish(mt_infix("17 #F")) should not produce an error, but probably be "17 #F", maybe with braces.
I.e. I think mt_infix("17 #F") should just be "#F(17)" and mt_polish("#F(17)") should be "17 #F"

EDIT 2: Please see the red braces I added after your quote. The * operand clearly affects the whole sum.

[real.finder]

Can you please provide LOS and HIS so I can review the whole calculation?
I transformed

y x - y 0.001 + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *

manually to

(y-x) / (y+0.001) * range_max - #F(LOS) * (y+range_size) / #F(512) + HIS

and I'm wondering if HIS sholuld also be #F(HIS).

@pinterf: I used the helper function mt_infix() to check my transform. And I saw it transform the #F operator in a way that surprised me.
mt_infix("17 #F") is #F(17,133168,8). My question is: Where does the helper function get the 2nd and 3rd parameter from? It looks like ~2^17 and 8 bits. But the helper function works without any bit depth context; it is just a string rearranger.

see DeHalo_alpha function (it's by Didée since 2006, but there are port in post above too)

[martin53]

see DeHalo_alpha function (it's by Didée since 2006, but there are port in post above too)

Calculation in 8 bits, LOS=HIS=50 (the defaults),
omitting the coward denominator adder 0.001 for simplicity, #F is not there,
x=102, y=128, range_max=255, range_size=256.
(128 - 102) / 128 * 255 - 50 * ((128 + 256) / 512 + 50)
26/128*255 is 51.8, the term after the minus sign is 2538.

I think the braces I added in my 1st post with the formula could be incorrect. Then:
(128 - 102) / 128 * 255 - 50 * (128 + 256) / 512 + 50
is 51.8 - 37.5 + 50
= 64.3
This is far from your result (1.248523832043474 in 255).

[pinterf]

For infix you have to use parentheses like #F(value) as if it was a function like sin

[real.finder]

Calculation in 8 bits, LOS=HIS=50 (the defaults),
omitting the coward denominator adder 0.001 for simplicity, #F is not there,
x=102, y=128, range_max=255, range_size=256.
(128 - 102) / 128 * 255 - 50 * ((128 + 256) / 512 + 50)
26/128*255 is 51.8, the term after the minus sign is 2538.

I think the braces I added in my 1st post with the formula could be incorrect. Then:
(128 - 102) / 128 * 255 - 50 * (128 + 256) / 512 + 50
is 51.8 - 37.5 + 50
= 64.3
This is far from your result (1.248523832043474 in 255).

you forget HIS = string(highsens/100.0)

and it's ((((y-x)/y+0.001)*range_max)-#F(los))*(((y+range_size)/#F(512))+his)

so you must dealing with brackets First

anyway, why you did this? pinterf already get what I mean :)

[StainlessS]

braces like #F(value)

Small point but those are parenthesis (), these are braces {}. EDIT: and these are brackets <>.

[real.finder]

Small point but those are parenthesis (), these are braces {}. EDIT: and these are brackets <>.

US VS UK? :rolleyes: http://english.stackexchange.com/questions/3379/bracket-vs-brace

[pinterf]

Fresh, new build, test it. If the new things (autoconverting ducks to geese) are not working as expected, I can fix it later next week.

This release is also addressing some of martin53's report (mt_polish and probably mt_infix)

Masktools2 2.2.4 (https://github.com/pinterf/masktools/releases/tag/2.2.4)

**v2.2.4 (20170304)**
- mt_polish to recognize:
- new v2.2.x constants and variables
a, bitdepth, sbitdepth, range_half, range_max, range_size, ymin, ymax, cmin, cmax
- v2.2.x scaling functions (written as #B(expression) and #F(expression) for mt_polish)
#B() #F
- single operand unsigned and signed negate introduced in 2.0.48
~u and ~s (written as ~u(expression)and ~s(expression) for mt_polish)
- other operators introduced in 2.0.48:
@, &u, |u, °u, @u, >>, >>u, <<, <<u, &s, |s, °s, @s, >>s, <<s
- mt_infix: don't put extra parameter after #F( and #B(
- new expression syntax: auto scale modifiers for float clips (test for real.finder):
Keyword at the beginning of the expression:
- clamp_f_i8, clamp_f_i10, clamp_f_i12, clamp_f_i14 or clamp_f_i16 for scaling and clamping
- clamp_f_f32 or clamp_f: for clamping the result to 0..1

Input values 'x', 'y', 'z' and 'a' are autoscaled by 255.0, 1023.0, ... 65535.0 before the expression evaluation,
so the working range is similar to native 8, 10, ... 16 bits. The predefined constants 'range_max', etc. will behave for 8, 10,..16 bits accordingly.

The result is automatically scaled back to 0..1 _and_ is clamped to that range.
When using clamp_f_f32 (or clamp_f) the scale factor is 1.0 (so there is no scaling), but the final clamping will be done anyway.
No integer rounding occurs.

[real.finder]

Fresh, new build

mt_lutxy( ugly, are, "clamp_f_i8 y x - y 0.001 + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *" )

quick test and work fine for both ducks and geese :thanks:

[real.finder]

funny bug in yv411

mt_edge("sobel",7,7,5,5) give me unsupported error (same old masktools2 in alpha and beta)

mt_edge("sobel") work fine!

[martin53]

anyway, why you did this? pinterf already get what I mean :)
I am a bit worried why a float-specific clamping problem might need expression syntax. I'd like to understand that and hope there will also be a solution without float-specific, complex syntax.

Rationale
When expressions need different syntax for float, aside from being harder to understand & manage and more time consuming when executed in realtime, it will be impossible forever to provide scripts that are equally usable for all clip types (restricted to those the plugin can process). It should be the highest plugin designer priority to have syntax pixel_type independent. Otherwise newbies will have an unneccessarily bad AviSynth experience.

[martin53]

Especially in the context of the masktools thread I'd like to point out a tiny new plugin StrResolve (https://forum.doom9.org/showthread.php?p=1799667#post1799667) that takes arbitrary strings, looks if they contain names of currently defined variables and silently replaces the names by the values in the output.

[pinterf]

Float is non-saturated, and as such, will need special attention regarding clamping. This syntax extension helps a bit for having consistent bit depth independent expressions without using extra parameters in the filters.
Float is an expert use case, and may not be identical to integer color spaces because of lacking the default clamped behaviour.

[martin53]

Float is non-saturated, and as such, will need special attention regarding clamping. This syntax extension helps a bit for having consistent bit depth independent expressions without using extra parameters in the filters.
Float is an expert use case, and may not be identical to integer color spaces because of lacking the default clamped behaviour.

Would the disadvantages be bigger, if clamped behaviour were default, and non clamping an expert use case than the advantages of identical expressions?

[real.finder]

Would the disadvantages be bigger, if clamped behaviour were default, and non clamping an expert use case than the advantages of identical expressions?

clamped behaviour is not default in float (whether now or before I asked for it)

people used to write expr that may give out of range values, lut is clamped it to 0-255, and so in 10-16 bit

but in float there are no clamped, so these kind of expr will be wrong in float, so I ask pinterf to add option to do that, so I used this option in dehalo alpha and Masked_DHA until now and it work well, and there are other scripts that need this option too I will port them soon

if you don't like it simply write new avs script function with new expr that don't give out of range :)

[martin53]

clamped behaviour is not default in float

Using the 'clip' operator at the end of the expression (for all pixel_types, since it's done anyway for fixed point) is not an option? Seems I don't get the difference.

[real.finder]

Using the 'clip' operator at the end of the expression (for all pixel_types, since it's done anyway for fixed point) is not an option? Seems I don't get the difference.

don't know, but I think yes

Clip[x,{min,max}]
gives x for min≤x≤max, min for x<min and max for x>max

but in this case you will need more things to write (the min and the max)

and it will not solve range_max and range_size problems in multiplication and division (cuz they give 1.0), unless you did two expr one for float and one for integer

[martin53]

two expr one for float and one for integer
Two expressions is clearly not the better option, I feel, because then everyone modifying it must understand/test/... both of them.

[real.finder]

hi pinterf

just start port FastLineDarkenMOD and note that

prot=5
mt_expand(thy=255/(prot+1)).mt_inpand()

give different results with different bit depth!

mt_edge (mode="prewitt", thY1=0, thY2=255) (from finedehalo)

this too

[real.finder]

this mt_binarize(70, mode="0 255") will not work in more than 8 bit, right?

[pinterf]

0 255 will work, mode names are unchanged. Input threshold parameter values however are not scaled

[real.finder]

mt_inflate(155) in more than 8 bit give

http://i.imgur.com/g8etO4R.png

[pinterf]

Isn't it 411?

[real.finder]

Isn't it 411?

it's 420

[pinterf]

it's 420
Could not reproduce, and the color space check is common for all mt_xxxx filters. Are you sure the right masktools plugin is loaded?
(I corrected the error message, anyway)

[real.finder]

Could not reproduce, and the color space check is common for all mt_xxxx filters. Are you sure the right masktools plugin is loaded?
(I corrected the error message, anyway)

did you try with xp one? cuz I am use xp in VirtualBox for testing

edit : even in x64 non xp

ColorBars
converttoyv12
convertbits(16)
mt_inflate(155)
convertbits(8)

[pinterf]

did you try with xp one? cuz I am use xp in VirtualBox for testing

edit : even in x64 non xp

ColorBars
converttoyv12
convertbits(16)
mt_inflate(155)
convertbits(8)
Works for me. xp, x64, all. What does avsmeter64 -avsinfo say?

[martin53]

Hi Ferenc,

operator #B scales from 8 bit to current bit depth using bit-shifts
operator #F scales from 8 bit to current bit depth using full range stretch


I suggest that you reconsider choosing #B and #F for these operators.

The point is that they collide with AviSynth's standard inline comment character '#'. David Horman says his RPN filters accept multiline strings with # comments (https://forum.doom9.org/showthread.php?p=1800428#post1800428) already, so he was the first to introduce this character for this purpose. Maybe you like alternatives like @B & @F (seems to me these can be fairly distinguished from @s, @u and the XOR @ operator itself), or '<<<' for bitshift, '^^^' for fullscale, or reserved words like you introduced them for the other new constants.

[pinterf]

I don't like them, either. I'd say yes on the changes before real.finder ports all the scripts in the world to the new syntax :)
real.finder?
Edit:
scaleb and scalef?

[real.finder]

Works for me. xp, x64, all. What does avsmeter64 -avsinfo say?

I don't think avsmeter64 -avsinfo will say something different in this case

I think I know were is the problem, I have mt_masktools-25.dll in plugins folder and made avs26.avsi to load the 2.6 one from subfolder if the avs is 2.6 or up, if I remove mt_masktools-25.dll it will work, but all my plugins work fine like this, why now there are some collision?

[real.finder]

I don't like them, either. I'd say yes on the changes before real.finder ports all the scripts in the world to the new syntax :)
real.finder?
Edit:
scaleb and scalef?

you can change it, no problem, I can use find and replace

[pinterf]

I don't think avsmeter64 -avsinfo will say something different in this case

I think I know were is the problem, I have mt_masktools-25.dll in plugins folder and made avs26.avsi to load the 2.6 one from subfolder if the avs is 2.6 or up, if I remove mt_masktools-25.dll it will work, but all my plugins work fine like this, why now there are some collision?
Using SetLogParams("log.txt", LOG_DEBUG) in the avs script (or may be a less restrictive setting) will list the possible collision.
Like for me:
"WARNING: VerticalCleaner() is defined by multiple plugins. Calls to this filter might be ambiguous and could result in the wrong function being called."

[real.finder]

Using SetLogParams("log.txt", LOG_DEBUG) in the avs script (or may be a less restrictive setting) will list the possible collision.
Like for me:
"WARNING: VerticalCleaner() is defined by multiple plugins. Calls to this filter might be ambiguous and could result in the wrong function being called."

http://pastebin.com/E1FALB7C

nothing special

[pinterf]

Nothing special?
WARNING: mt_inflate() is defined by multiple plugins.
Calls to this filter might be ambiguous and could result in the wrong function being called.
then
ERROR: mt_inflate : unsupported colorspace. masktools only support planar YUV colorspaces (YV12, YV16, YV24)

[real.finder]

Nothing special?
WARNING: mt_inflate() is defined by multiple plugins.
Calls to this filter might be ambiguous and could result in the wrong function being called.
then
ERROR: mt_inflate : unsupported colorspace. masktools only support planar YUV colorspaces (YV12, YV16, YV24)

defined by multiple plugins is something no special in my case cuz as I say in #86, the 2.6 one should being used cuz it load after 2.5 one, and that what happen except few cases in masktools2 in high bits

anyway, I can do something for this case by move the 2.5 one in another subfolder and call it if avs is 2.5

edit: just did this and note that the output of mt_inflate(155) in more than 8 bit is different

[pinterf]

Different, because filter parameters like threshold are not scaled, this was mentioned earlier. To scale or not to scale, that is the question, ideally this threshold is either a percent 0-100 or normalized to the 0.0-1.0 range. Does it have any drawback if we are not able to specify exactly e.g . 32765 in a 16 bit range, only a (full range scaled) 127/255*65535? Of course since this parameter is float, one can specify 127.6 if it is important.

You know the use cases better, I accept any logical proposal.

(And I have to check the plugin load behaviour, in particular the order of the filter function registrations)

[real.finder]

what about add bitdepth_scale parameter in the end of parameters?

and float 0.0-255.0 will be ok too (it will be compatible with old script in all bit depth since float parameter accept integer)

[pinterf]

The parameter type is float already because filters accept unscaled values for float clips

[real.finder]

The parameter type is float already because filters accept unscaled values for float clips

but it's not in 255.0 range with scale

anyway, bitdepth_scale (or if you have better name) can be in string to control scale method (like what you did in lut), can even be set to no scale like what is now, but the Default will be scale from 255 range (whether float or integer)

[pinterf]

The choice is @B @F. Scaling will assume parameter values on 8 bit scale but can be overridden to 10-16 bits or float or no scaling at all

[real.finder]

isn't better to use something away from @ and # etc...? like scaleb and scalef

[kgrabs]

Weird idea here, since mt_makediff and mt_adddiff now support working with 256< pixel values, would it be possible to make some sort of internal workaround for the limited range of diff clips by extending the range using a higher bitdepth? I figure if anything the math involved would be a nightmare, but I thought it'd be worth asking

Edit: nvm figured it out
bri=mt_lutxy(a,b,"x y < y x - 0 ?")
drk=mt_lutxy(a,b,"x y > x y - 0 ?")
mt_lutxyz(a,bri,drk,"x y + z -")

[pinterf]

isn't better to use something away from @ and # etc...? like scaleb and scalef
You and martin53 are a kind of powerusers of the new features, I appreciate your feedback, it is much easier to me to follow an advice than find things out of my mind without proper experience.

[martin53]

isn't better to use something away from @ and # etc...? like scaleb and scalef

It's not a surprise I confirm my above proposal.
Reason is 'scaleb' and 'scalef' do not carry any special character, i.e. they look like constants, but in fact they are a combination of a resolution-varying constant and a bitshift- or multiplication operation. Admittedly, 'sin', etc. also are functions and have non-operator-names.

scaleb and scalef are quite long tokens too (as are the other recently introduced constants), which is, in a formula, often not the preferred concept for readability (as opposed to text, formulas often need to be looked at as the whole thing / recursively, cannot just be read from start to end).

I admit @ is equivocal with the XOR operator and 'do a bitshift to range' is maybe semantically closer to '<<R' when compared with '<<s'. That's why I mentioned alternatives '<<<' (triple < can be understood somehow as 'special left shift', can't it?) and '^^^' (special scale up), which might also be '***' :D

So what about '<<r' and '*r', (r for range or resolution) which closest follows the 'OperatorLetter' principle of all the earlier &u ... >>s two step operations .

[real.finder]

It's not a surprise I confirm my above proposal.
Reason is 'scaleb' and 'scalef' do not carry any special character, i.e. they look like constants, but in fact they are a combination of a resolution-varying constant and a bitshift- or multiplication operation. Admittedly, 'sin', etc. also are functions and have non-operator-names.

scaleb and scalef are quite long tokens too (as are the other recently introduced constants), which is, in a formula, often not the preferred concept for readability (as opposed to text, formulas often need to be looked at as the whole thing / recursively, cannot just be read from start to end).

I admit @ is equivocal with the XOR operator and 'do a bitshift to range' is maybe semantically closer to '<<R' when compared with '<<s'. That's why I mentioned alternatives '<<<' (triple < can be understood somehow as 'special left shift', can't it?) and '^^^' (special scale up), which might also be '***' :D

So what about '<<r' and '*r', (r for range or resolution) which closest follows the 'OperatorLetter' principle of all the earlier &u ... >>s two step operations .

I see << in vs py script, and there are another things like dup operator, but I will vote for SF and SB

[pinterf]

<< like operators would mean >>, depending on the defined base bitmap, the operator would do either scaling up or down.

[real.finder]

I see avs script from 2011 use << operator (YAHRmask), the guides and change logs of masktools didn't mention that << is supported, is there another hidden operators and functions that masktools2 support?

[pinterf]

They are in change log of 2.0.47/48?

[real.finder]

They are in change log of 2.0.47/48?

no, that why I ask, since you see the source code and edit it

even in the wiki http://avisynth.nl/index.php/MaskTools2#Reverse_polish_notation

[pinterf]

Then I'll refresh it in wiki and doc, but I'm sure I've seen them somewhere.

[raffriff42]

Then I'll refresh it in wiki and doc, but I'm sure I've seen them somewhere.They've been there for a while, but only in the source tree(s) AFAICT
<source root>/masktools/documentation/mt_masktools.html#reverse_polish_notationV) Reverse polish notation.
...Signed and unsigned unary and binary operators exists : "&u", "|u", "°u" ("@u"), "~u", "<<u", ">>u" round and clip their operands to the nearest integer in [0..2^64-1], then do a "and", "or", "xor", "complement to 1", "left shift", or "right shift". Accordingly, "&s", "|s", "°s" (@s), "~s", "<<s", ">>s" do the same operations, but round and clip to the nearest integer in [-2^63..2^63-1] beforehand. "<<" and ">>" are equivalent to "<<u" and ">>u". The second operand of shifts is always considered signed : a left shift by a negative value is equivalent to a right shift by the opposite of the value.

...
Changelog
Alpha 48 : added : signed and unsigned binary shifts to LUTs

[real.finder]

They've been there for a while, but only in the source tree(s) AFAICT
<source root>/masktools/documentation/mt_masktools.html#reverse_polish_notation

yes, I just note they are in masktools-v2.0a48, I usually see the wiki for documentation, and didn't interested in reverse polish notation before beta by tp7, and since he didn't put the documentation with the binaries I just see the wiki link as it should has the last update for beta changes


that aside, pinterf, can you add stack manipulation operators, swap and dup (as vapoursynth expr)?

[pinterf]

Updated MaskTools Wiki
http://avisynth.nl/index.php/MaskTools2

[real.finder]

Updated MaskTools Wiki
http://avisynth.nl/index.php/MaskTools2

:thanks:

and where is Reel.Deal? he interested in documentation and wiki

you already flooded porting plugins so I think someone else should did the documentation

[pinterf]

New release
masktools2 v2.2.6 (https://github.com/pinterf/masktools/releases/tag/2.2.6)
EDIT: new 2.2.6 link instead of v2.2.5 (hotfix)

The most important change in this release is the autoscaling of parameters (this is not the in-expression case!). You can use the same parameters as you do in 8 bit scripts.
E.g. mt_binarize threshold "80" will be autoscaled to 80*256 for a 16 bit clip. You can override this behaviour with a new parameter "paramscale" in order to have such parameters interpreted as native 10-16 bit or float values, or disable scaling with "none".

By introducing float support, filter signatures have been changed to allow floats for Y, U, V and other parameters. Unfortunately this resulted in a strange situation: when the new masktools2 was loaded later with LoadPlugin, and there was an old masktools2.dll autoloaded earlier, then Avisynth was finding the old filters from the 8 bit masktools2 version. This happened when the filter was invoked with integer parameters (best parameter type match).
Now the filters are defined with dual parameter list, one with the new float types, and another one with the old integer-type list, to prevent finding the old masktools2 filters rather than the new one.

The operator of the in-expression bit depth scaling (#F and #B) was changed to @B and @F

Then, after the request from real.finder, dup and swap keywords are supported now.


**v2.2.6 (20170401)
- fix: >>u operator AV error
**v2.2.5 (20170330)
- Change #F and #B operators to @B and @F
- Alias scaleb for @B
- Alias scalef for @F
- New: automatic scaling of parameters (threshold-like, sc_value) from the usual 8 bit range
Scripts need no extra measures to work for all bit depths with the same "command line"
- New parameter "paramscale" for filters working with threshold-like parameters
Filters: mt_binarize, mt_edge, mt_inpand, mt_expand, mt_inflate, mt_deflate, mt_motion, mt_logic, mt_clamp
paramscale can be "i8" (default), "i10", "i10", "i12", "i14", "i16", "f32" or "none" or ""
Using "paramscale" tells the filter that parameters are given at what bit depth range.
- dual function signatures (float and int), for backward compatibility with integer-type parameter list, prevent usage of earlier plugin-loaded masktools version
- keep old parameter ordering: parameters which are non-existant in 2.0b1 are inserted at the end of the parameter list, not before the common parameters Y, U, V
- new: "swap" keyword in expressions
- new: "dup" keyword in expressions
- a bit faster realtime lut calculation for 10+ bit depths

[Dreamland]

First:Thx a lot for great work
I have an issue with MaskTools2 v2.2.5


https://t9.pixhost.org/thumbs/224/38966724_appunti01.jpg (https://pixhost.org/show/224/38966724_appunti01.jpg)

my AVS works with MaskTools2 v2.2.4

:)

[pinterf]

First:Thx a lot for great work
I have an issue with MaskTools2 v2.2.5


https://t9.pixhost.org/thumbs/224/38966724_appunti01.jpg (https://pixhost.org/show/224/38966724_appunti01.jpg)

my AVS works with MaskTools2 v2.2.4

:)
Thanks. The operator >>u was called as a function with one parameter but it has two params. Hotfix is coming soon.

A lot of code has been changed in expression handling, in order to have dup, swap, and a general speedup in realtime calculations where lut is not avaliable (until VS-like Expr will appear - not very soon though). Malformed expressions can still cause troubles, but I think they weren't checked in earlier versions either.

EDIT: 2.2.6 is out.

[pinterf]

New version available with hotfix (>>u operator).
Masktools 2.2.6 (https://github.com/pinterf/masktools/releases/tag/2.2.6)

[Dreamland]

Perfect!
now it works ..many thanks! :)

[real.finder]

hi pinterf

remember this? https://forum.doom9.org/showthread.php?p=1794976#post1794976

some encoders use this formula in lutxyz
((clip1*(255-mask))+(clip2*mask))/255

ColorBars(pixel_type="yv12")
src = mt_lutspa(expr="x 255 *")
mask = mt_lut(y=-255)

smaskmerge (https://pastebin.com/YHM68SvX)(mt_lut(y=-0), src, mask,u=2,v=2) #will show all frame in zero, and that what should
#~ mt_merge(mt_lut(y=-0), src, mask,u=2,v=2) #will not show all frame in zero

mt_lutxy(last, src, "x y - abs 75 *").grayscale()

edit: and here smaskmerge with another formula https://pastebin.com/aLP9Mb3z

so, it's better replace the one in mt_merge or put some option to change the formula for compatible?

I vote for put option and the default will be the new formula in non float format

[pinterf]

Lut always works in 64 bit double precision, integer roundings are no problem there.

10-16 bit code path already has special treatment of min (0) and max (e.g. 1023,... 65535) masks
https://github.com/pinterf/masktools/blob/16bit/masktools/filters/merge/merge16.cpp#L15
https://github.com/pinterf/masktools/blob/16bit/masktools/filters/merge/merge16.cpp#L116

8 bit code kept the original implementation. Maybe it is reasonable to handle the situation in a similar way.
https://github.com/pinterf/masktools/blob/16bit/masktools/filters/merge/merge.cpp#L96

[pinterf]

New release

Masktools2 v2.2.7 (https://github.com/pinterf/masktools/releases/tag/2.2.7)

Changelog
**v2.2.7 (20170421)
- fix: mt_edge 10,12,14 bits: clamp mask value from 65535 to 1023 (10 bits), 4095 (12 bits) and 16383 (14 bits)
- fix: mt_merge 10-16 bits + non mod-16 width + luma=true + 4:2:2 colorspace, correct right side pixels
- fix: mt_merge 8 bit clips: keep original pixels from clip1/2 when mask is exactly 0 or 255
- YUVA, RGBAP support 8-32 bits
- "A" parameter like "Y", "U" and "V". Default value for "A" is 1 (do nothing, same as for "U" and "V")
- "alpha" parameter like "chroma" - overrides default plane mode
- aExpr parameter for lut-type filters like uExpr, and vExpr
- awExpr parameter for mt_luts like uwExpr, and vwExpr
- dual signature filters (both integer and float) are provided in separate binaries
In some cases specifying two different parameter lists with the same variables can cause troubles.
(dual signature version can be used to override an earlierly loaded different masktools version
(e.g. a 2.5 plugin) by defining the filters with both integer parameters _AND_ the new float parameter lists)
- Make "scaleparams" to work consistent with all filters and parameters:
parameters "Y","U","V" and "A" negative (memset) values are scaled automatically to the current bit depth from a default 8-bit value.
- New plane mode: 6 (copy from fourth clip) for "Y", "U", "V" and "A"
New "chroma" and "alpha" plane mode override: "copy fourth"
Use for mt_lutxyza which has four clips

[real.finder]

New release

Masktools2 v2.2.7 (https://github.com/pinterf/masktools/releases/tag/2.2.7)


:thanks:

but seems the dual filter signatures is same as the non dual filter signatures, at least for x86 xp (give me Unsupported color space as before)

[pinterf]

:thanks:

but seems the dual filter signatures is same as the non dual filter signatures, at least for x86 xp (give me Unsupported color space as before)
EDIT: Replaced those two DLLs (dual-xp x86/x64), sorry for the inconvenience.

[Motenai Yoda]

- fix: mt_merge 10-16 bits + non mod-16 width + luma=true + 4:2:2 colorspace, correct right side pixels
not 4:2:0 too?

[pinterf]

The original reason of 420 nonmod16 right edge garbage was the mt_edge thing.

[real.finder]

The original reason of 420 nonmod16 right edge garbage was the mt_edge thing.

mt_merge with luma=true in 4:2:0 use mpeg2 chroma place, right?

[pinterf]

mt_edge with luma=true in 4:2:0 use mpeg2 chroma place, right?
You mean mt_merge.
Yes, I can see no special clever subsampling shifts in the code.

[real.finder]

You mean mt_merge.
Yes, I can see no special clever subsampling shifts in the code.

yes, mt_merge

do you think mt_merge will need some Parameter for mpeg2 vs mpeg1 in 420 with luma=true?

[MysteryX]

M = mt_merge(M, BHard, Sc, luma=true)


This code applies ChangeFps (BHard) on scene changes (Sc).

If I instead write Sc.ConvertToY8(), even though luma=true, the BHard is only being applied to Luma and not to chroma planes.

Is this intentional or a bug?

(why should I process YV12 masks when only Y8 is needed?)

[StainlessS]

@MysteryX

u and v are defaulted to 2 (that way, the resulting clip contains the chroma of clip1, and looks right).

These three values describe the actual processing mode that is to be used on each plane / channel. Here is how the modes are coded :

x=-255..0 : all the pixels of the plane will be set to -x.
x=1 : the plane will not be processed. That means the content of the plane after the filter is pure garbage.
x=2 : the plane of the first input clip will be copied.
x=3 : the plane will be processed with the processing the filter is designed to do.
x=4 (when applicable) : the plane of the second input clip will be copied.
x=5 (when applicable) : the plane of the third input clip will be copied.

As you can see, defaults parameters are chosen to only process the luma, and not to care about the chroma. It's because most video processing doesn't touch the chroma when handling 4:2:0.
string "chroma" ("")

When defined, the value contained in this string will overwrite the u & v processing modes. This is a nice addition proposed by mg262 that makes the filter more user friendly. Allowed values for chroma are :

"process" : set u = v = 3.
"copy" or "copy first" : set u = v = 2.
"copy second" : set u = v = 4.
"copy third" : set u = v = 5.
"xxx", where xxx is a number : set u = v = -xxx.


Set u=3, v=3 or Chroma="Process".

[MysteryX]

Thanks, that works. There is one more issue.


file = "Motion Estimation Torture Clip.avi"
C = LWLibavVideoSource(file, cache=False).ConvertToYV12()
BHard = C.ChangeFPS(60)
B = C.ConvertFPS(60)
Sc = BlankClip(C, color=color_white).ConvertToY8()
mt_merge(B, BHard, Sc, luma=true, chroma="process")


This "should" apply ChangeFps to the clip and repeat frames. However, about 5% of ConvertFps is showing up so repeated frames aren't identical.

What is causing this?

[MysteryX]

Sc is in the 16-235 range which causes blending of every frame. This fixes the issue


Sc = Sc.mt_binarize(128)

Is there a better way to do it?

[Gavino]

Sc is in the 16-235 range which causes blending of every frame. This fixes the issue

Sc = Sc.mt_binarize(128)

Is there a better way to do it?
Instead of

Sc = BlankClip(C, color=color_white).ConvertToY8()

use
Sc = BlankClip(C, pixel_type="Y8", color_yuv=$ff0000)

[raffriff42]

In mt_binarize, the threshold argument is scaled from 0-255 to the current bit depth,
but the chroma argument assumes the current scale. I assume this is a bug?
Which one's "right?"

https://www.dropbox.com/s/z7gxg6uozubah5y/mt_bin14-1.jpg?raw=1

https://www.dropbox.com/s/ybyqasbe9b8weou/mt_bin14-2.jpg?raw=1

https://www.dropbox.com/s/vh58lykg4ztf6se/mt_bin14-3.jpg?raw=1

By the way, for the wiki, I'd like to how to describe a variable like threshold:
multi-bit-depth? deep-color-aware? autoscale? (asking for opinions)

[pinterf]

In mt_binarize, the threshold argument is scaled to the current bit depth,
but the chroma argument is always on a 0-255 scale. I assume this is a bug?

Yes. It should behave like Y,U,V and A in this case. I hope there are no more hidden variables. Thanks for the report.

[pinterf]

By the way, for the wiki, I'd like to how to describe a variable like threshold:
multi-bit-depth? deep-color-aware? autoscale? (asking for opinions)
Perhaps autoscale or autoscalable.

(just to refresh our minds: depending on "paramscale" such parameters can be autoscaled, default base is from 8 bits, paramscale="none" does nothing for users who like to keep the behaviour in their hands)

[pinterf]

New release
Masktools2 2.2.8 (https://github.com/pinterf/masktools/releases/tag/2.2.8)

Thanks to raffriff42 for the report.

Recent changes:

**v2.2.8 (20170427)
- Fix: "chroma" and "alpha" parameter should be scaled like "Y","U","V" and "A" when providing negative (memset) values

**v2.2.7 (20170421)
- fix: mt_edge 10,12,14 bits: clamp mask value from 65535 to 1023 (10 bits), 4095 (12 bits) and 16383 (14 bits)
- fix: mt_merge 10-16 bits + non mod-16 width + luma=true + 4:2:2 colorspace, correct right side pixels
- fix: mt_merge 8 bit clips: keep original pixels from clip1/2 when mask is exactly 0 or 255
- YUVA, RGBAP support 8-32 bits
- "A" parameter like "Y", "U" and "V". Default value for "A" is 1 (do nothing, same as for "U" and "V")
- "alpha" parameter like "chroma" - overrides default plane mode
- aExpr parameter for lut-type filters like uExpr, and vExpr
- awExpr parameter for mt_luts like uwExpr, and vwExpr
- dual signature filters (both integer and float) are provided in separate binaries
In some cases specifying two different parameter lists with the same variables can cause troubles.
(dual signature version can be used to override an earlierly loaded different masktools version
(e.g. a 2.5 plugin) by defining the filters with both integer parameters _AND_ the new float parameter lists)
- Make "scaleparams" to work consistent with all filters and parameters:
parameters "Y","U","V" and "A" negative (memset) values are scaled automatically to the current bit depth from a default 8-bit value.
- New plane mode: 6 (copy from fourth clip) for "Y", "U", "V" and "A"
New "chroma" and "alpha" plane mode override: "copy fourth"
Use for mt_lutxyza which has four clips

[hello_hello]

I accidentally posted this in the MVTools2 thread originally, so sorry about the cross-posting.

pinterf,
In case you'd care to take a look at it, there appears to be a problem with your flavour of MaskTools2 (any version) and the SeeSaw sharpening script.
https://forum.doom9.org/showthread.php?p=1808548#post1808548

Thanks.

Hi, I have tried the new version of masktools and there is a problem

Evaluate: Unhandled C++ exception!

(SeeSaw.avs, line 128)

(SeeSaw.avs, line 80)

[burfadel]

I accidentally posted this in the MVTools2 thread originally, so sorry about the cross-posting.

pinterf,
In case you'd care to take a look at it, there appears to be a problem with your flavour of MaskTools2 (any version) and the SeeSaw sharpening script.
https://forum.doom9.org/showthread.php?p=1808548#post1808548

Thanks.

As I posted in the other thread, does seesaw() work by itself, and if so what additional filter causes it not to work for you?

EDIT: Tried it. A while ago it only didn't work with using it with lsfmod etc, now it just doesn't work at all!

[manolito]

As outlined in the first post of the SeeSaw thread the required additional tools are RemoveGrain and MaskTools 1.5.8.
https://forum.doom9.org/showthread.php?p=758021#post758021

The current pinterf versions of MaskTools are not compatible, you need the old masktools.dll 1.5.8 from Manao.

Cheers
manolito

[hello_hello]

As I posted in the other thread, does seesaw() work by itself, and if so what additional filter causes it not to work for you?

Odd....
I wasn't thinking when you asked the question about LSFMod in the other thread, because obviously you feed SeeSaw a denoised clip, not a sharpened one....

but so far it's all working as expected with the non-pinterf versions of MaskTools2 I tried.

SeeSaw()

or

a = last
b = a.SMDegrain()
SeeSaw(a, b)

Or even

SeeSaw()
LFSMod()

All work, and it is sharpening. I checked in case that's what you meant by "not working".
Replacing MaskTools2 with a pinterf version results in the exception error, even when using just SeeSaw() on it's own.

manolito,
The only version of SeeSaw I've been able to find (and it's the one I'm using) specifies MaskTools2.
There's a doom9 link at the top of the script, but unfortunately it doesn't work so I don't know if it was part of the original SeeSaw thread or why the post/thread was deleted.
http://avisynth.nl/index.php/SeeSaw

[StainlessS]

manolito,
The only version of SeeSaw I've been able to find (and it's the one I'm using) specifies MaskTools2.
There's a doom9 link at the top of the script, but unfortunately it doesn't work so I don't know if it was part of the original SeeSaw thread or why the post/thread was deleted.
http://avisynth.nl/index.php/SeeSaw

The non working link in script is this one:- http://forum.doom9.org/showthread.php?p=1748710#post1748710

Which is I think a link to post where I modded SeeSaw as new Avs v2.6 change broke things:, Force int call arguments to user script function float params to be explicit floats.

I went looking for that post/thread, could not find it. (looks like thread was deleted [perhaps an InGoldie thread, he has habit of deletion]).

[Reel.Deel]

The non working link in script is this one:- http://forum.doom9.org/showthread.php?p=1748710#post1748710

Which is I think a link to post where I modded SeeSaw as new Avs v2.6 change broke things:, Force int call arguments to user script function float params to be explicit floats.

I went looking for that post/thread, could not find it. (looks like thread was deleted [perhaps an InGoldie thread, he has habit of deletion]).

I remember the thread, according to upload log on the wiki (http://avisynth.nl/index.php/File:SeeSaw.avs) you modified the script twice.


The only version of SeeSaw I've been able to find (and it's the one I'm using) specifies MaskTools2.
There's a doom9 link at the top of the script, but unfortunately it doesn't work so I don't know if it was part of the original SeeSaw thread or why the post/thread was deleted.
http://avisynth.nl/index.php/SeeSaw

SeeSaw using MaskTools2 has been on the wiki since early 2008.
https://web.archive.org/web/20080516150126/http://avisynth.org:80/mediawiki/SeeSaw

As outlined in the first post of the SeeSaw thread the required additional tools are RemoveGrain and MaskTools 1.5.8.
https://forum.doom9.org/showthread.php?p=758021#post758021

The current pinterf versions of MaskTools are not compatible, you need the old masktools.dll 1.5.8 from Manao.


Just a public service announcement. :)

Do not use the old SeeSaw script.
Do not use MaskTools 1.5.8.
Use RgTools over RemoveGrain.

[pinterf]

New version.

Download Masktools2 2.2.9 (https://github.com/pinterf/masktools/releases/tag/2.2.9)

Changes
**v2.2.9 (20170608)
- Add "none" and "ignore" to valid values for "chroma" and "alpha" parameters.
- Report error for invalid "chroma" or "alpha" parameter values instead of exception

[StainlessS]

Thank you oh wise one, we loves you !

[real.finder]

there are another bugs that hnn mention here https://forum.doom9.org/showthread.php?p=1808701#post1808701

the one in GradFun2DBmod reproduced, it's in mt_luts(input,input,mode="range",pixels=mt_square(radius),expr="y",u=1,v=1)

if I remove expr="y" then it will work

the other one didn't reproduced for me

[pinterf]

Thanks, found it, fix along with some serious speed improvements in 16 bit xxflate/xxpand will be coming soon

[pinterf]

New build with an important fix for mt_luts.
And a bonus performance increase in 10-16bits xxpand/xxflate for users having at least SSE4.

Download MaskTools2 2.2.10 (https://github.com/pinterf/masktools/releases/tag/2.2.10)

Changelog
**v2.2.10 (20170612)
- Fix: luts internal buffer overflow (crash)
- Speed: mt_inpand/mt_expand: 10-16 bits SSE4 (10-15x speed)
- Speed: mt_inflate/mt_deflate 10-16 bits SSE4 (4x speed)

[real.finder]

New build with an important fix for mt_luts.
And a bonus performance increase in 10-16bits xxpand/xxflate for users having at least SSE4.

Download MaskTools2 2.2.10 (https://github.com/pinterf/masktools/releases/tag/2.2.10)

Changelog
**v2.2.10 (20170612)
- Fix: luts internal buffer overflow (crash)
- Speed: mt_inpand/mt_expand: 10-16 bits SSE4 (10-15x speed)
- Speed: mt_inflate/mt_deflate 10-16 bits SSE4 (4x speed)


thanks, I can feel the speed up :thanks:

[Groucho2004]

thanks, I can feel the speed up :thanks:I decided to measure it and yes, your feeling was spot on. :D

Nice work Ferenc!

Edit: The same applies to MVTools.

[edcrfv94]

#invert PC range and invert back
mt_lut("x 255 - abs", y=3, u=3, v=3)
mt_lut("x 255 - abs", y=3, u=3, v=3)



#invert limited range and invert back
mt_lut(yexpr="x 235 - abs 16 +", uexpr="x 240 - abs 16 +", vexpr="x 240 - abs 16 +", y=3, u=3, v=3)
mt_lut(yexpr="x 235 - abs 16 +", uexpr="x 240 - abs 16 +", vexpr="x 240 - abs 16 +", y=3, u=3, v=3)



#There is no problem in Python
for x in range(16, 236):
x = abs(x -235) + 16
print (x)


limited range can not invert back to original image, it is bug?

[jpsdr]

Could it be that your limited range trig threshold, and so thresholded values cannot be reverted because information is lost ?

[edcrfv94]

Could it be that your limited range trig threshold, and so thresholded values cannot be reverted because information is lost ?

But limited range video use:

mt_lut("x 255 - abs", y=3, u=3, v=3)
mt_lut("x 255 - abs", y=3, u=3, v=3)
#e.g: 16 to 239 back to 16

can invert back to original image.
Where can disabled limited range threshold?

[pinterf]

Check your original clip.

First, run with a simple Limiter() first.
http://avisynth.nl/index.php/Limiter

If mt_lut conversions are now O.K., go back and check by Limiter(...,show="luma_grey") where the original clip exceeds luma (or chroma: show="chroma_grey) limits.

[edcrfv94]

Check your original clip.

First, run with a simple Limiter() first.
http://avisynth.nl/index.php/Limiter

If mt_lut conversions are now O.K., go back and check by Limiter(...,show="luma_grey") where the original clip exceeds luma (or chroma: show="chroma_grey) limits.

mt_lut conversions are work fine.
Because original clip out of tv range(max luma Y:237), so can not invert back to original image.
Default PC range is more safe, it my bad.

[StainlessS]

Hi Pinterf,
When you get the time, can you please put note on first post noting most recent version to support Mt_Polish/Mt_Infix for XP. Thanx :)

[javidial]

Hi,
I'm trying to use QTGMC. I've downloaded all the plugins necesary, like the latest Masktools2, but still when I preview my script in MeGUI, it pops an error saying "there is no function named MSuper".

The function supposed to be included in the masktools2.dll, right? what am I missing. The masktools2 seems to load correctly cos it doesn't give me any error loading it.

Can somebody help me? Thanks

[Motenai Yoda]

Hi,
I'm trying to use QTGMC. I've downloaded all the plugins necesary, like the latest Masktools2, but still when I preview my script in MeGUI, it pops an error saying "there is no function named MSuper".

The function supposed to be included in the masktools2.dll, right? what am I missing. The masktools2 seems to load correctly cos it doesn't give me any error loading it.

Can somebody help me? Thanks

msuper is from mvtools

[javidial]

msuper is from mvtools

Thanks! There're so many filters that I'm getting super confused sometimes.

[MysteryX]

you mean, getting msuper confused

[hello_hello]

Can somebody help me? Thanks

Doesn't mvtools require a runtime flavour, depending on the version? I think the latest needs Microsoft Visual C++ Redistributable 2015 Update 3. Maybe that's the problem.

[real.finder]

hi pinterf

check this https://forum.doom9.org/showthread.php?t=174752

and what about adding mt_merge parameter (https://forum.doom9.org/showthread.php?p=1804907#post1804907) for mpeg2 vs mpeg1 in 420 with luma=true?

[pinterf]

hi pinterf

check this https://forum.doom9.org/showthread.php?t=174752

and what about adding mt_merge parameter (https://forum.doom9.org/showthread.php?p=1804907#post1804907) for mpeg2 vs mpeg1 in 420 with luma=true?
I have bookmarked it, but will look at it later (summertime lazyness)

[edcrfv94]

when I change mt_lutxy 8bit to 16bit from 410 fps drop to 0.8 fps.

8bit: 410 fps

SetMemoryMax(3000)

colorbars(width = 1920, height = 1080, pixel_type = "yv12")

ConvertToY8()
trim(0, 5000)

test_in = last#.ConvertBits(bits=16)

a = test_in
b = test_in

kf_limit_dif8(a, b, thr=1, elast=36, y=3, u=1, v=1).kf_limit_dif8(b, thr=1, elast=36, y=3, u=1, v=1)

#ConvertToStacked().DitherPost(mode=6, ampo=1)

Function kf_limit_dif8(clip filtered, clip original, bool "smooth", float "thr", float "elast", float "darkthr", int "Y", int "U", int "V")
{
smooth = Default(smooth, True )
thr = Default(thr, 1.0 )
elast = Default(elast, smooth ? 3.0 : 255./thr)
darkthr = Default(darkthr,thr )
Y = Default(Y, 3 )
U = Default(U, 3 )
V = Default(V, 3 )

Y = min(Y, 4)
U = min(U, 4)
V = min(V, 4)

thr = max(min( thr, 255.0), 0.0)
darkthr = max(min(darkthr, 255.0), 0.0)
elast = max(elast, 1.0)
mode = thr == 0 && darkthr == 0 ? 4 : thr == 255 && darkthr == 255 ? 2 : 3
smooth = elast==1 ? False : smooth

diffstr = " x y - "
elaststr = " "+string(elast)+" "

thrstr = diffstr+" 0 > "+string(darkthr)+" scalef "+string(thr)+" scalef ? "
alphastr = elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstr+" * / ? "
betastr = thrstr+elaststr+" * "
sexpr = smooth ? alphastr+diffstr+" * "+betastr+diffstr+" abs - * y + "
\ : thrstr+diffstr+diffstr+" abs / * y + "
expr = diffstr+" abs "+thrstr+" <= x "+diffstr+" abs "+betastr+" >= y "+sexpr+" ? ? "

thrstrc = " "+string(thr)+" scalef "
alphastrc= elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstrc+" * / ? "
betastrc = thrstrc+elaststr+" * "
sexprc = smooth ? alphastrc+diffstr+" * "+betastrc+diffstr+" abs - * y + "
\ : thrstrc+diffstr+diffstr+" abs / * y + "
exprc = diffstr+" abs "+thrstrc+" <= x "+diffstr+" abs "+betastrc+" >= y "+sexprc+" ? ? "

# diff = filtered - original
# alpha = 1 / (thr * (elast - 1))
# beta = elast * thr
# When smooth=True :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + alpha * diff * (beta - abs(diff))
# When smooth=False :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + thr * (diff / abs(diff))

return mode == 4 ? original
\ : mode == 2 ? filtered
\ : mt_lutxy(filtered, original, yExpr=expr, uExpr=exprc, vExpr=exprc, Y=Y, U=U, V=V)
}


16bit: 0.8 fps

SetMemoryMax(3000)

colorbars(width = 1920, height = 1080, pixel_type = "yv12")

ConvertToY8()
trim(0, 5000)

test_in = last.ConvertBits(bits=16)

a = test_in
b = test_in

kf_limit_dif8(a, b, thr=1, elast=36, y=3, u=1, v=1).kf_limit_dif8(b, thr=1, elast=36, y=3, u=1, v=1)

ConvertToStacked().DitherPost(mode=6, ampo=1)

Function kf_limit_dif8(clip filtered, clip original, bool "smooth", float "thr", float "elast", float "darkthr", int "Y", int "U", int "V")
{
smooth = Default(smooth, True )
thr = Default(thr, 1.0 )
elast = Default(elast, smooth ? 3.0 : 255./thr)
darkthr = Default(darkthr,thr )
Y = Default(Y, 3 )
U = Default(U, 3 )
V = Default(V, 3 )

Y = min(Y, 4)
U = min(U, 4)
V = min(V, 4)

thr = max(min( thr, 255.0), 0.0)
darkthr = max(min(darkthr, 255.0), 0.0)
elast = max(elast, 1.0)
mode = thr == 0 && darkthr == 0 ? 4 : thr == 255 && darkthr == 255 ? 2 : 3
smooth = elast==1 ? False : smooth

diffstr = " x y - "
elaststr = " "+string(elast)+" "

thrstr = diffstr+" 0 > "+string(darkthr)+" scalef "+string(thr)+" scalef ? "
alphastr = elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstr+" * / ? "
betastr = thrstr+elaststr+" * "
sexpr = smooth ? alphastr+diffstr+" * "+betastr+diffstr+" abs - * y + "
\ : thrstr+diffstr+diffstr+" abs / * y + "
expr = diffstr+" abs "+thrstr+" <= x "+diffstr+" abs "+betastr+" >= y "+sexpr+" ? ? "

thrstrc = " "+string(thr)+" scalef "
alphastrc= elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstrc+" * / ? "
betastrc = thrstrc+elaststr+" * "
sexprc = smooth ? alphastrc+diffstr+" * "+betastrc+diffstr+" abs - * y + "
\ : thrstrc+diffstr+diffstr+" abs / * y + "
exprc = diffstr+" abs "+thrstrc+" <= x "+diffstr+" abs "+betastrc+" >= y "+sexprc+" ? ? "

# diff = filtered - original
# alpha = 1 / (thr * (elast - 1))
# beta = elast * thr
# When smooth=True :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + alpha * diff * (beta - abs(diff))
# When smooth=False :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + thr * (diff / abs(diff))

return mode == 4 ? original
\ : mode == 2 ? filtered
\ : mt_lutxy(filtered, original, yExpr=expr, uExpr=exprc, vExpr=exprc, Y=Y, U=U, V=V)
}

[MysteryX]

According to this table (Feature Matrix)
http://avisynth.nl/index.php/MaskTools2

mt_lutxy will evaluate in real-time instead of using a LUT table when bits>=14, otherwise the LUT table would be too large. You can override this behavior with the "realtime" parameter.

[edcrfv94]

According to this table (Feature Matrix)
http://avisynth.nl/index.php/MaskTools2

mt_lutxy will evaluate in real-time instead of using a LUT table when bits>=14, otherwise the LUT table would be too large. You can override this behavior with the "realtime" parameter.

realtime=True 0.8 fps
realtime=False 10 min still initialization LUT tables?

[real.finder]

realtime=True 0.8 fps
realtime=False 10 min still initialization LUT tables?

you mustn't use SetMemoryMax when realtime=False

and you must use avs+ 64 and have large ram (6g or more)

[MysteryX]

and yes several minutes can be normal, that's why LUT is disabled by default in such cases. It grows exponentially with the bitrate. Now sure how much it actually takes.

[TheFluff]

A LUT is just a dictionary that translates each possible value in the input to a value in the output. A 16-bit mt_lutx needs 2^16 = 65,536 entries per plane, and each entry takes 2 bytes, so that's 65536*2*3 = 384 kB. That's gonna be really slow since it doesn't fit in CPU cache, but it's technically "usable". mt_lutxy though takes two clips, so now there's 65535*65536 possible values, and 65536*65536*3*2 is 25,769,803,776 bytes or 24.5 gigabytes. Have fun allocating that, and don't even think about mt_lutxyz.

Masktools is in practice unusable for high bitdepth until and unless someone ports the Vapoursynth expr filter (which works by compiling the RPN expression to SSE2-optimized native code (https://forum.doom9.org/showthread.php?p=1807048#post1807048)).

[real.finder]

A LUT is just a dictionary that translates each possible value in the input to a value in the output. A 16-bit mt_lutx needs 2^16 = 65,536 entries per plane, and each entry takes 2 bytes, so that's 65536*2*3 = 384 kB. That's gonna be really slow since it doesn't fit in CPU cache, but it's technically "usable". mt_lutxy though takes two clips, so now there's 65535*65536 possible values, and 65536*65536*3*2 is 25,769,803,776 bytes or 24.5 gigabytes. Have fun allocating that, and don't even think about mt_lutxyz.

Masktools is in practice unusable for high bitdepth until and unless someone ports the Vapoursynth expr filter (which works by compiling the RPN expression to SSE2-optimized native code (https://forum.doom9.org/showthread.php?p=1807048#post1807048)).

what about https://github.com/tp7/CLExpr ? since people can use opencl even with cpu only

[TheFluff]

Didn't know that was a thing! Sounds promising though.

[edcrfv94]

Dither tools must faster than masktools at 16bit

Dither tools:59 fps

SetMemoryMax(6000)

colorbars(width = 1920, height = 1080, pixel_type = "yv12")

ConvertToY8()
trim(0, 5000)

test_in = last.ConvertBits(bits=16)

a = test_in
b = test_in

kf_limit_dif16(a, b, thr=1, elast=36, y=3, u=1, v=1).kf_limit_dif16(b, thr=1, elast=36, y=3, u=1, v=1)

ConvertToStacked().DitherPost(mode=6, ampo=1)

Function kf_limit_dif16(clip filtered, clip original, bool "smooth", float "thr", float "elast", float "darkthr", int "Y", int "U", int "V")
{
Is16bit = (filtered.BitsPerComponent == 16) ? True : False

filtered = Is16bit ? filtered.ConvertToStacked() : filtered
original = Is16bit ? original.ConvertToStacked() : original

smooth = Default(smooth, True )
thr = Default(thr, 1.0 )
elast = Default(elast, smooth ? 3.0 : 128./thr)
darkthr = Default(darkthr,thr )
Y = Default(Y, 3 )
U = Default(U, 3 )
V = Default(V, 3 )

Y = min(Y, 4)
U = min(U, 4)
V = min(V, 4)
Yt = Y == 3
Ut = U == 3
Vt = V == 3
Y31 = Yt ? 3 : 1
U31 = Ut ? 3 : 1
V31 = Vt ? 3 : 1

thr = max(min( thr, 128.0), 0.0)
darkthr = max(min(darkthr, 128.0), 0.0)
elast = max(elast, 1.0)
mode = thr == 0 && darkthr == 0 ? 4 : thr == 128 && darkthr == 128 ? 2 : 3
smooth = elast==1 ? False : smooth
thr = thr / 255 * 65535
darkthr = darkthr / 255 * 65535

diffstr = " x 32768 - "
elaststr = " "+string(elast)+" "

thrstr = diffstr+" 0 > "+string(darkthr)+" "+string(thr)+" ? "
alphastr = elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstr+" * / ? "
betastr = thrstr+elaststr+" * "
sexpr = smooth ? alphastr+diffstr+" * "+betastr+diffstr+" abs - * 32768 + "
\ : thrstr+diffstr+diffstr" abs / * 32768 + "
expr = diffstr+" abs "+thrstr+" <= x "+diffstr+" abs "+betastr+" >= 32768 "+sexpr+" ? ? "

thrstrc = " "+string(thr)+" "
alphastrc= elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstrc+" * / ? "
betastrc = thrstrc+elaststr+" * "
sexprc = smooth ? alphastrc+diffstr+" * "+betastrc+diffstr+" abs - * 32768 + "
\ : thrstrc+diffstr+diffstr" abs / * 32768 + "
exprc = diffstr+" abs "+thrstrc+" <= x "+diffstr+" abs "+betastrc+" >= 32768 "+sexprc+" ? ? "

# diff = filtered - original
# alpha = 1 / (thr * (elast - 1))
# beta = elast * thr
# When smooth=True :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + alpha * diff * (beta - abs(diff))
# When smooth=False :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + thr * (diff / abs(diff))

diff = Dither_sub16(filtered, original, y=Y31, u=U31, v=V31, dif=True)
ldiff = Dither_lut16(diff, yexpr=expr, uexpr=exprc, vexpr=exprc, y=Y31, u=U31, v=V31)
merged = Dither_add16(original, ldiff , y=Y31, u=U31, v=V31, dif=True)
merged = Y==2 || U==2 || V==2 || Y==4 || U==4 || V==4 ? mt_lutxyz(filtered, original, merged, Y=Y==3?5:Y, U=U==3?5:U, V=V==3?5:V) : merged

out = mode == 4 ? original
\ : mode == 2 ? filtered
\ : merged

return Is16bit ? out.ConvertFromStacked() : out
}


cl_exprxy 11 fps with GPU(i7 3770k intel HD4000) 13x speedup than mt_lutxy


SetMemoryMax(6000)

colorbars(width = 1920, height = 1080, pixel_type = "yv12")

ConvertToY8()
trim(0, 5000)

test_in = last.ConvertBits(bits=16).ConvertToStacked()

a = test_in
b = test_in

kf_limit_dif8(a, b, thr=1, elast=36, y=3, u=1, v=1).kf_limit_dif8(b, thr=1, elast=36, y=3, u=1, v=1)

DitherPost(mode=6, ampo=1)

Function kf_limit_dif8(clip filtered, clip original, bool "smooth", float "thr", float "elast", float "darkthr", int "Y", int "U", int "V")
{
smooth = Default(smooth, True )
thr = Default(thr, 1.0 )
elast = Default(elast, smooth ? 3.0 : 255./thr)
darkthr = Default(darkthr,thr )
Y = Default(Y, 3 )
U = Default(U, 3 )
V = Default(V, 3 )

Y = min(Y, 4)
U = min(U, 4)
V = min(V, 4)

thr = max(min( thr, 255.0), 0.0)/255*65535
darkthr = max(min(darkthr, 255.0), 0.0)/255*65535
elast = max(elast, 1.0)
mode = thr == 0 && darkthr == 0 ? 4 : thr == 255 && darkthr == 255 ? 2 : 3
smooth = elast==1 ? False : smooth

diffstr = " x y - "
elaststr = " "+string(elast)+" "

thrstr = diffstr+" 0 > "+string(darkthr)+" "+string(thr)+" ? "
alphastr = elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstr+" * / ? "
betastr = thrstr+elaststr+" * "
sexpr = smooth ? alphastr+diffstr+" * "+betastr+diffstr+" abs - * y + "
\ : thrstr+diffstr+diffstr+" abs / * y + "
expr = diffstr+" abs "+thrstr+" <= x "+diffstr+" abs "+betastr+" >= y "+sexpr+" ? ? "

thrstrc = " "+string(thr)+" "
alphastrc= elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstrc+" * / ? "
betastrc = thrstrc+elaststr+" * "
sexprc = smooth ? alphastrc+diffstr+" * "+betastrc+diffstr+" abs - * y + "
\ : thrstrc+diffstr+diffstr+" abs / * y + "
exprc = diffstr+" abs "+thrstrc+" <= x "+diffstr+" abs "+betastrc+" >= y "+sexprc+" ? ? "

# diff = filtered - original
# alpha = 1 / (thr * (elast - 1))
# beta = elast * thr
# When smooth=True :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + alpha * diff * (beta - abs(diff))
# When smooth=False :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + thr * (diff / abs(diff))

return mode == 4 ? original
\ : mode == 2 ? filtered
\ : cl_exprxy(filtered, original, yExpr=expr, uExpr=exprc, vExpr=exprc, Y=Y, U=U, V=V, lsb=True)
}

[real.finder]

edcrfv94

dithertools don't has any full 16bit lut except Dither_lut16

don't know and didn't check the way you use to make something like mt_lutxy with dithertools but I think it's will not be real thing or just a hack

[edcrfv94]

edcrfv94

dithertools don't has any full 16bit lut except Dither_lut16

don't know and didn't check the way you use to make something like mt_lutxy with dithertools but I think it's will not be real thing or just a hack

kf_limit_dif16 copy from HQDeringmod_v1.8.avsi and small modify added avs+ 16bit support .

will AVS+ compatible VapourSynth Plugins, scripts and core Function easily than VapourSynth compatible AVS Plugins? like MP_Pipeline 32bit/64bit compatible way

dithertools only can calculated, other way will becomes very complicated.

[GMJCZP]

Friends, I have a question:

I'm practicing with mt_infix and mt_polish and I'm getting different frames with mt_lut on screen. Here is the script:

expr_RPN="x 126 < 126 x 130 > 130 x ? ?"
expr_INF=mt_infix(expr_RPN)
expr_INF2="(x<126) ? 126 : (x>130) ? 130 : x"
expr_RPN2=mt_polish(expr_INF2)

I do not know if I'm missing something, but with mt_infix(expr_RPN) I'm getting "(((x<126)) ? 126 : (((x>130)) ? 130 : x))" which is equivalent to "(x<126) ? 126 : (x>130) ? 130 : x".

The problem is that with mt_lut (expr_RPN) I'm getting a different result than with mt_lut (expr_RPN2), what's the reason? Is it a rounding problem?

[TheFluff]

In this code:
expr_INF2="(x<126) ? 126 : (x>130) ? 130 : x"
expr_RPN2=mt_polish(expr_INF2)

expr_RPN2 evaluates to

x 126 < 126 x 130 > ? 130 x ?

which, translated back to infix again is (after removing redundant parens):

((x<126) ? 126 : (x>130)) ? 130 : x

Which is obviously not what you want. This is because unlike the arithmetic operators which are generally left-associative (https://en.wikipedia.org/wiki/Operator_associativity), the ternary operator in C (and most other languages that use it, except PHP because of course PHP does everything wrong) is right-associative, so the entire expression "(x<126) ? 126 : (x>130)" is evaluated as the condition for the following ?.

edit: no, wait, hold on, I have my head screwed on backwards. If it was actually right-associative, it'd do what you want. mt_infix is parsing it as if it was left-associative, it seems.

[GMJCZP]

Thank you, I will analyze your answer because I confess that I am not yet clear enough.

Edit: to overcome associativity I must generously apply parentheses, do not I?

[TheFluff]

Yes, but I was confused when I wrote my post. Your original way of writing it actually should work if the mt_infix ternary operator worked like it does in C (i.e. right-associative). In Perl, which also has a right-associative ternary operator, this works as expected:

$x = 120;
$x = ($x<126) ? 126 : ($x>130) ? 130 : $x;
print "$x"; # prints 126

In PHP, however, which has a left-associative ternary operator:
$x = 120;
$x = ($x<126) ? 126 : ($x>130) ? 130 : $x;
echo $x; # prints 130!

You have to add parens around the third operand (that is, after the colon) to prevent the following ? from grabbing everything to its left as its first operand. Like so:
$x = 120;
$x = ($x<126) ? 126 : (($x>130) ? 130 : $x);
echo $x; # prints 126, as expected

[Gavino]

The problem is that with mt_lut (expr_RPN) I'm getting a different result than with mt_lut (expr_RPN2), what's the reason?
I think this is the bug I reported a while back (Feb 2014), where mt_polish() parses nested conditionals incorrectly.

I can't remember if it was ever supposedly corrected.

[GMJCZP]

Thank you to both.

For me the simplest solution would be, as I stated earlier, to apply parentheses generously, but I would be putting the garbage under the carpet. I want to learn this perfectly and it is difficult for me to understand the associativity of : on the left.

[TheFluff]

Operator associativity is similar to operator precedence, but for operators that have the same precedence. For example, 2*3+1 is (2*3)+1 = 7, not 2*(3+1) = 8, because * has higher precedence than +. However, if you nest operators with the same precedence, you have to decide in which direction the operations should be grouped. For example, consider the expression 10-4-2. The way I intuitively read this is from left to right - we first group the two first operands (10-4), evaluate the result (we get 6) and then take that result as the operand for the next minus operator and end up with 6-2 = 4. Or to write it another way, (10-4)-2. This is also the way it is evaluated in basically all programming languages. We say that the arithmetic subtraction operator is left-associative, because the operations are grouped from left to right.

On the other hand, if it were right-associative instead, the same expression 10-4-2 would be grouped from right to left instead, and we'd end up with 10-(4-2) = 8. That is very unintuitive to me, and no programming language I know of has a right-associative subtraction operator.

Now, the ternary operator is a bit more complex because it takes three operands, not two, but it's essentially the same thing. When looking at an expression like the one you originally posted, all the parser sees is:
operand_A ? operand_B : operand_C ? operand_D : operand_E
Now, how should it read this? Each ?: needs three operands, but we actually only seem to have five total. Is operand_C the third operand for the leftmost ternary op, or is it the first operand for the rightmost ternary op? If you group from the right (like in every language other than PHP), the rightmost ternary operator is evaluated first and the righthand side sub-expression (operand_C ? operand_D : operand_E) becomes the third operand for the leftmost ternary op. If you instead group from the left like in PHP and mt_polish, the expression (operand_A ? operand_B : operand_C) becomes the first operand to the rightmost ternary op. Or, with parens:
# right associative (almost everything)
operand_A ? operand_B : (operand_C ? operand_D : operand_E)

# left associative (PHP, mt_polish)
(operand_A ? operand_B : operand_C) ? operand_D : operand_E

Does that make it any clearer?

[StainlessS]

I think this is the bug I reported a while back (Feb 2014), where mt_polish() parses nested conditionals incorrectly.

I can't remember if it was ever supposedly corrected.

Nope, dont think so, tis horrible the way that it works, but cant see how it could be fixed without causing perhaps a great deal of upset. [EDIT: Nah, no upset necessary]

@GMJCZP, did my head in when I first came across that.

[GMJCZP]

Let's look at this:

$x = 120;
$x = ($x<126) ? 126 : ($x>130) ? 130 : $x;

To see if I understand, both php and mt_polish are doing this to me:

$x = 120;
$x = ($x<126) ? (126 : ($x>130) ? 130) : $x;

That's why it gives 130?

[TheFluff]

No, it's doing this:
(($x<126) ? 126 : ($x>130)) ? 130 : $x
This part:
(($x<126) ? 126 : ($x>130))
returns 126 if $x is less than 126, or else it returns ($x>130). ($x>130) evaluates to 1 if $x is greater than 130, else 0. So if $x is 120, you get
120 ? 130 : $x
and 120 is obviously true. That's why it returns 130.

[GMJCZP]

Thanks once again.

I honestly had a hard time understanding this. Now my only doubt is:

120 ? 130 : $x

Are you sure it's 120? will not be:

126 ? 130 : $x

Or

True ? 130 : $x

[TheFluff]

You're right, that's a typo. It should be 126, not 120.

[GMJCZP]

Ok. Thanks.

$x = 120
126 ? 130 : $x

Do you start from here why X = 130?

[FranceBB]

MPEG2Source ("video_sd.d2v")

Dither_convert_yuv_to_rgb (matrix="601", output="rgb48y")
Dither_resize16 (1280, 720, y=3, u=1, v=1)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=false, mode=0)

I get an error:
http://i.imgur.com/Lup0lFi.png

I'm trying to understand if it's intentional or if it's the dither tool wiki that is wrong.

[Groucho2004]

MPEG2Source ("video_sd.d2v")

Dither_convert_yuv_to_rgb (matrix="601", output="rgb48y")
Dither_resize16 (1280, 720, y=3, u=1, v=1)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=false, mode=0)

I get an error:
http://i.imgur.com/Lup0lFi.png

I'm trying to understand if it's intentional or if it's the dither tool wiki that is wrong.I tried your script and works fine with a PAL DVD source.
- Is this your complete script?
- Run AVSMeter with the switches "-avsinfo -log" and post the "avsinfo_x86.log" file.

[FranceBB]

Looks like it doesn't work. Not sure whether it's related to virtualbox (I use Linux - Fedora) or not.
If I try "avsmeter.exe test.avs" it crashes as well.
That's my full avs, 'cause I was testing out of curiosity (it's not gonna be used for a real encode).
OS: Windows XP Professional x86
I'll test it in my server, which uses Windows Server 2008 R2 x64 (the one I actually use to encode) if XP it's a problem.

The instruction "0x03d9f6c0" referenced memory at "0x03d9f6c0". The required data was not placed into memory because of an I/O error status of "0x75b2b11c".

http://i.imgur.com/HwTNrkA.png

[Groucho2004]

Looks like it doesn't work. Not sure whether it's related to virtualbox (I use Linux - Fedora) or not.
If I try "avsmeter.exe test.avs" it crashes as well.
I never tried Virtualbox but have no problem running AVSMeter in a VMWare VM. XP should not be the problem.

[edcrfv94]

mt_expand(thY=42, u=3, v=3)
not equal
mt_lutsx(last, last, last, mode="max", pixels=mt_square(1), yexpr="y x - 42 > x 42 + y ?", u=3, v=3)

mt_expand(thY=255, u=3, v=3)
equal
mt_lutsx(last, last, last, mode="max", pixels=mt_square(1), yexpr="y x - 255 > x 255 + y ?", u=3, v=3)

Look like thX limit not work.

[sl1pkn07]

@FranceBB for curiosity, why you use virtualbox for avisynth(+)? wine(-staging) works ok with it (more or less)

[FranceBB]

Some filters are just .avsi, other are DLLs that could be either statically or dynamically compiled and could require other dependencies to run, like .NET Framework or C++ Redistributable etc... Wine and Mono might not be the best solution. On the other hand, there are other Linux -friendly alternatives like AVXSynth and Vapoursynth, but I got used to Avisynth. I started using it in 2006 lurking in this forum for quite some time without registering an account. I also don't encode on my PC: it's just an i7 6700HQ 4c/8th with 16 GB of RAM DDR4, I prepare scripts, see the preview and then encode in my Windows Server 2008 R2 which has a far better Xeon processor and way more RAM (even though I've never actually used more than 5 GB of RAM in an encode).

[DJATOM]

FranceBB
wine-staging is quite good. I'm using it on the server with Debian 8 and it works fine with my chains (there are 64-bit versions of masktools2, mvtools2, dfttest, f3kdb, dither tools, rgtools, eedi3, nnedi3 and some other filters).

[FranceBB]

I see... well, I guess I'll give it a shot, then. :)

[pinterf]

A LUT is just a dictionary that translates each possible value in the input to a value in the output. A 16-bit mt_lutx needs 2^16 = 65,536 entries per plane, and each entry takes 2 bytes, so that's 65536*2*3 = 384 kB. That's gonna be really slow since it doesn't fit in CPU cache, but it's technically "usable". mt_lutxy though takes two clips, so now there's 65535*65536 possible values, and 65536*65536*3*2 is 25,769,803,776 bytes or 24.5 gigabytes. Have fun allocating that, and don't even think about mt_lutxyz.

Masktools is in practice unusable for high bitdepth until and unless someone ports the Vapoursynth expr filter (which works by compiling the RPN expression to SSE2-optimized native code (https://forum.doom9.org/showthread.php?p=1807048#post1807048)).
True, true, true, I know it since I have tried mt_lutxy(za) in real life for anything over 8 bits.

But I'm still reluctant to use ("steal") anything from Vapoursynth project, I feel that they have worked hard for giving the Expr functionality to their own community and being more modern and different to Avisynth.

[MysteryX]

and being more modern and different to Avisynth.
Pinterf, you've already done the damage

[real.finder]

True, true, true, I know it since I have tried mt_lutxy(za) in real life for anything over 8 bits.

But I'm still reluctant to use ("steal") anything from Vapoursynth project, I feel that they have worked hard for giving the Expr functionality to their own community and being more modern and different to Avisynth.

what about clexpr?

and it's not steal cuz it's open source Policy :) Vapoursynth did so with avs so why avs+ can't with Vapoursynth?

edit: and ultim in the first place aimed to make avs+ modern just like Vapoursynth if you back to https://forum.doom9.org/showthread.php?t=168856 discussions and Myrsloik (Vapoursynth author) encourage supports avs+


Session Start: Thu Dec 18 17:06:03 2014
Session Ident: #avs-plus
03[17:06] * Now talking in #avs-plus
03[17:06] * Topic is 'Avisynth+ official development channel | Help test MT! Experimental build at http://goo.gl/e0VFYn | Help filling mt modes: https://pad.riseup.net/p/avs_plus_mt_modes | Intrinsics guide: http://asm.avs-plus.net/ | Plugins porting queue: http://goo.gl/kPywxZ'
03[17:06] * Set by ultim!~ultim@somloi.galuska.rulez on Thu Mar 06 02:12:36
[17:06] <***> Buffer Playback...
[17:06] <Myrsloik> [06:18:30] has ultim been seen here recently?
[17:06] <torch> [06:19:55] last message seems to have been three months ago
[17:06] <@tp7> [06:42:32] I'm still hoping my PR will get merged one day
[17:06] <Myrsloik> [06:51:56] it'd be nice if this project undied and killed off all the other avisynth branches

[Myrsloik]

True, true, true, I know it since I have tried mt_lutxy(za) in real life for anything over 8 bits.

But I'm still reluctant to use ("steal") anything from Vapoursynth project, I feel that they have worked hard for giving the Expr functionality to their own community and being more modern and different to Avisynth.

Uh, that's not how open source is supposed to work...

[TheFluff]

True, true, true, I know it since I have tried mt_lutxy(za) in real life for anything over 8 bits.

But I'm still reluctant to use ("steal") anything from Vapoursynth project, I feel that they have worked hard for giving the Expr functionality to their own community and being more modern and different to Avisynth.
I guess I'm more guilty than most when it comes to complaining about how awful Avisynth is, but really now, it's not like anyone actually wants you to make it even more awful, or for you to keep it awful just because that's how it's always been. I mean, I think work spent on improving Avisynth provides little gain for the amount of effort spent because of all the legacy annoyances, so what you really don't need is to decide to reinvent wheels just because elitist Vapoursynth nerds are acting smugly superior on an internet forum.

Also, as was already mentioned, Vapoursynth has "stolen" a ton of Avisynth code and Avisynth plugins. Re-solving solved problems in a software niche this small with this few developers benefits absolutely nobody, so port away.

[StainlessS]

Open source is a joint community effort, it is in everyone's interest that the concept of open source succeed, no matter which team you support.
Don't waste any further time wondering about whether or not you personally have to invent every wheel that you use, just steal steal steal,
anything and everything (open source) that can be of use. I'm quite sure that along the way, you may inadvertently create source that will be stolen
back and reused by others in the community.

Go Pinterf, pinterficate like the wind :) And, Thank You for being Pinterf.

[pinterf]

Thanks for sharing your thoughts.

[burfadel]

Open source means you can use it, as long as you don't charge for borrowed code and you acknowledge the source. Maybe some of the legacy support code can be removed, particularly if it is impinging in any way. The 64 bit avisynth shouldn't have any of the 32 bit compatibility stuff since you can't use those filters. Maybe all the compatibility code that is still valid but not useful for most modern filters can be moved to a plugin, no point limiting Avisynth with unuseful constraints.

[real.finder]

hi pinterf

check this https://forum.doom9.org/showthread.php?t=174752

and what about adding mt_merge parameter (https://forum.doom9.org/showthread.php?p=1804907#post1804907) for mpeg2 vs mpeg1 in 420 with luma=true?

any news? LSFmod port depends on that

[burfadel]

It seems that maybe the scaling feature doesn't work as intended. I was using mt_lut with range of 256, with bit depth 12, and the result was as if there was no scaling (all black). If I convert back to 8 bit first, run it, and then convert back to 12 bits it works properly.

[pinterf]

It seems that maybe the scaling feature doesn't work as intended. I was using mt_lut with range of 256, with bit depth 12, and the result was as if there was no scaling (all black). If I convert back to 8 bit first, run it, and then convert back to 12 bits it works properly.
What was your expression string? Constants inside expressions are scaled only when you specify scaleb or scalef for them.

[real.finder]

so, since the expr is added to avs+, are you going to make mt_lut* use it with some option (https://github.com/pinterf/masktools/issues/1#issuecomment-340471206) or it's not possible?

[pinterf]

I was just thinking about it while rideing homeward. Sure, it won't be a default behaviour.
It has performance penalty. Scaling inputs to a common range requires a floating multiplication within the expression, right after reading the source pixels, unless the common bitdepth is the same as source clip bitdepth. I suppose - knowing that this behaviour was requested because of the easy conversion of old scripts - that this common bit depth is in 8 bit scale 0-255. So for 16 bit input clips the multiplier is 1/256. For 8 bit input, there is no performance loss in this scenario.
A second conversion occurs before storing the result back.

Another ambiguity comes on whether the source is a limited range yuv or full scale. Limited range can nicely be scaled by bit shift method, but this method will give wrong results if we use it on a full scale source.

Other.

In Expr not all operators/functions are implemented, there are masktools-only syntax elements. Do you know scripts that are using these operators? Modulo, sin, cos, all kinds of rounding?

[edcrfv94]

You can try this first, mt_lut at 16bit still faster than Expr 10% speed.


Function kf_expr_x(clip clip1, string "expr", string "yExpr", string "uExpr", string "vExpr", string "aExpr", int "Y", int "U", int "V", int "A", bool "sse2", bool "avx2", bool "optSSE2", bool "optSingleMode", bool "optAvx2")
{
sCSP = clip1.kf_GetCSP()
IsY8 = sCSP == "Y8"
IsRGBA = sCSP == "RGBA"

sBit = clip1.BitsPerComponent()
use_mt_expr = (sBit == 8)

yExpr = Default(yExpr, expr)
uExpr = Default(uExpr, yExpr)
vExpr = Default(vExpr, yExpr)
aExpr = Default(aExpr, yExpr)

optSSE2 = Default(optSSE2, sse2)
optAvx2 = Default(optAvx2, avx2)

Y = Default(Y, 3)
U = Default(U, 1)
V = Default(V, 1)
A = Default(A, 1)

yExpr = (Y == 3) ? yExpr : ""
uExpr = (U == 3) ? uExpr : ""
vExpr = (V == 3) ? vExpr : ""
aExpr = (A == 3) ? aExpr : ""

out = use_mt_expr ? mt_lut(clip1, expr=expr, yExpr=yExpr, uExpr=uExpr, vExpr=vExpr, aExpr=aExpr, Y=Y, U=U, V=V, A=A, sse2=sse2, avx2=avx2)
\ : IsY8 ? Expr(clip1, yExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)
\ : !IsRGBA ? Expr(clip1, yExpr, uExpr, vExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)
\ : Expr(clip1, yExpr, uExpr, vExpr, aExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)

return out
}

Function kf_expr_xy(clip clip1, clip clip2, string "expr", string "yExpr", string "uExpr", string "vExpr", string "aExpr", int "Y", int "U", int "V", int "A", bool "sse2", bool "avx2", bool "optSSE2", bool "optSingleMode", bool "optAvx2")
{
sCSP = clip1.kf_GetCSP()
IsY8 = sCSP == "Y8"
IsRGBA = sCSP == "RGBA"

sBit = clip1.BitsPerComponent()
use_mt_expr = (sBit == 8)

yExpr = Default(yExpr, expr)
uExpr = Default(uExpr, yExpr)
vExpr = Default(vExpr, yExpr)
aExpr = Default(aExpr, yExpr)

optSSE2 = Default(optSSE2, sse2)
optAvx2 = Default(optAvx2, avx2)

Y = Default(Y, 3)
U = Default(U, 1)
V = Default(V, 1)
A = Default(A, 1)

yExpr = (Y == 3) ? yExpr : ""
uExpr = (U == 3) ? uExpr : ""
vExpr = (V == 3) ? vExpr : ""
aExpr = (A == 3) ? aExpr : ""

out = use_mt_expr ? mt_lutxy(clip1, clip2, expr=expr, yExpr=yExpr, uExpr=uExpr, vExpr=vExpr, aExpr=aExpr, Y=Y, U=U, V=V, A=A, sse2=sse2, avx2=avx2)
\ : IsY8 ? Expr(clip1, clip2, yExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)
\ : !IsRGBA ? Expr(clip1, clip2, yExpr, uExpr, vExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)
\ : Expr(clip1, clip2, yExpr, uExpr, vExpr, aExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)

return out
}

Function kf_expr_xyz(clip clip1, clip clip2, clip clip3, string "expr", string "yExpr", string "uExpr", string "vExpr", string "aExpr", int "Y", int "U", int "V", int "A", bool "sse2", bool "avx2", bool "optSSE2", bool "optSingleMode", bool "optAvx2")
{
sCSP = clip1.kf_GetCSP()
IsY8 = sCSP == "Y8"
IsRGBA = sCSP == "RGBA"

sBit = clip1.BitsPerComponent()
use_mt_expr = (sBit == 8)

yExpr = Default(yExpr, expr)
uExpr = Default(uExpr, yExpr)
vExpr = Default(vExpr, yExpr)
aExpr = Default(aExpr, yExpr)

optSSE2 = Default(optSSE2, sse2)
optAvx2 = Default(optAvx2, avx2)

Y = Default(Y, 3)
U = Default(U, 1)
V = Default(V, 1)
A = Default(A, 1)

yExpr = (Y == 3) ? yExpr : ""
uExpr = (U == 3) ? uExpr : ""
vExpr = (V == 3) ? vExpr : ""
aExpr = (A == 3) ? aExpr : ""

out = use_mt_expr ? mt_lutxyz(clip1, clip2, clip3, expr=expr, yExpr=yExpr, uExpr=uExpr, vExpr=vExpr, aExpr=aExpr, Y=Y, U=U, V=V, A=A, sse2=sse2, avx2=avx2)
\ : IsY8 ? Expr(clip1, clip2, clip3, yExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)
\ : !IsRGBA ? Expr(clip1, clip2, clip3, yExpr, uExpr, vExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)
\ : Expr(clip1, clip2, clip3, yExpr, uExpr, vExpr, aExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)

return out
}

Function kf_expr_xyza(clip clip1, clip clip2, clip clip3, clip clip4, string "expr", string "yExpr", string "uExpr", string "vExpr", string "aExpr", int "Y", int "U", int "V", int "A", bool "sse2", bool "avx2", bool "optSSE2", bool "optSingleMode", bool "optAvx2")
{
sCSP = clip1.kf_GetCSP()
IsY8 = sCSP == "Y8"
IsRGBA = sCSP == "RGBA"

sBit = clip1.BitsPerComponent()
use_mt_expr = (sBit == 8)

yExpr = Default(yExpr, expr)
uExpr = Default(uExpr, yExpr)
vExpr = Default(vExpr, yExpr)
aExpr = Default(aExpr, yExpr)

optSSE2 = Default(optSSE2, sse2)
optAvx2 = Default(optAvx2, avx2)

Y = Default(Y, 3)
U = Default(U, 1)
V = Default(V, 1)
A = Default(A, 1)

yExpr = (Y == 3) ? yExpr : ""
uExpr = (U == 3) ? uExpr : ""
vExpr = (V == 3) ? vExpr : ""
aExpr = (A == 3) ? aExpr : ""

out = use_mt_expr ? mt_lutxyza(clip1, clip2, clip3, clip4, expr=expr, yExpr=yExpr, uExpr=uExpr, vExpr=vExpr, aExpr=aExpr, Y=Y, U=U, V=V, A=A, sse2=sse2, avx2=avx2)
\ : IsY8 ? Expr(clip1, clip2, clip3, clip4, yExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)
\ : !IsRGBA ? Expr(clip1, clip2, clip3, clip4, yExpr, uExpr, vExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)
\ : Expr(clip1, clip2, clip3, clip4, yExpr, uExpr, vExpr, aExpr, optSSE2=optSSE2, optSingleMode=optSingleMode, optAvx2=optAvx2)

return out
}

Function kf_GetCSP(clip c)
{
try {
csp = c.kf_GetCSP_avsPlus()
} catch (error_msg) {
csp = c.kf_GetCSP_avs()
}
return csp
}


Function kf_GetCSP_avs(clip c)
{
return c.IsPlanar ? c.IsYV12 ? "YV12" :
\ c.IsYV16 ? "YV16" :
\ c.IsYV24 ? "YV24" : c.kf_GetCSP_Y8_YV411() :
\ c.IsYUY2 ? "YUY2" :
\ c.IsRGB32 ? "RGB32" :
\ c.IsRGB24 ? "RGB24" : "Unknown"

Function kf_GetCSP_Y8_YV411(clip c) {
try {
c.UtoY
csp = "YV411"
} catch (error_msg) {
csp = "Y8"
}
return csp
}
}

Function kf_GetCSP_avsPlus(clip c)
{
return c.Is420 ? "YV12" :
\ c.IsY ? "Y8" :
\ c.Is422 ? "YV16" :
\ c.Is444 ? "YV24" :
\ c.IsYUVA ? "YUVA" :
\ c.IsYV411 ? "YV411" :
\ c.IsYUY2 ? "YUY2" :
\ c.IsRGB32 ? "RGB32" :
\ c.IsRGB24 ? "RGB24" :
\ c.IsPlanarRGB ? "RGB" :
\ c.IsPlanarRGBA ? "RGBA" :
\ c.IsPackedRGB ? "RGBIL" : "Unknown"
}

[real.finder]

In Expr not all operators/functions are implemented, there are masktools-only syntax elements. Do you know scripts that are using these operators? Modulo, sin, cos, all kinds of rounding?

well, I can't count all scripts, and aside from those in wiki there are many that not listed there, and they are more than these in wiki

so for safe choice, useexpr parameter should be:-

useexpr="internal" or "none" (default)

so anyone update some function that has mt_lut* can make it faster if it possible by set it to "internal", and the "internal" will be "none" automatically if normal avs or old avs+ is used

aside from that now, maybe in future if someone update the clexpr (https://github.com/tp7/CLExpr), then it will be another options with "internal" and "none"

[real.finder]

You can try this first, mt_lut at 16bit still faster than Expr 10% speed.


didn't try it but what about RAM usage :devil:?

[edcrfv94]

didn't try it but what about RAM usage :devil:?

Almost no different, mt_lut at 16bit use 1mb ram more than Expr.

[pinterf]

You can try this first, mt_lut at 16bit still faster than Expr 10% speed.

I think it depends on the expression itself. What expression string did you use for comparison? (and that 10% means that lut is faster by 10% or lut needs only 10% time of Expr?)

And I'd like to ask you (or someone) with AVX2, could you please compare the performance of lut/lutxy with Expr on your machines (for a basic expression like "x x +" for lut and "x y -" for lut_xy, and a more complex one?) with optAvx2=False and optAVX2=true. For 8, 10 and 16 bits. (10 bit lut_xy has not that much memory overhead)

Thanks.

[edcrfv94]

I think it depends on the expression itself. What expression string did you use for comparison? (and that 10% means that lut is faster by 10% or lut needs only 10% time of Expr?)

And I'd like to ask you (or someone) with AVX2, could you please compare the performance of lut/lutxy with Expr on your machines (for a basic expression like "x x +" for lut and "x y -" for lut_xy, and a more complex one?) with optAvx2=False and optAVX2=true. For 8, 10 and 16 bits. (10 bit lut_xy has not that much memory overhead)

Thanks.

3770k did not support AVX2 only has AVX.(When I9 PC arrivals I can test again maybe need a month.)

mt_lut 8bit-16bit 10%+ faster with complex Expr.
mt_lutxy 8bit faster Expr, other way Expr much faster.

I7 3770k 4.2g

16bit Expr Y8: 201fps 49% cpu 42Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

ConvertBits(bits=16)
last.Expr("x x +")
ConvertToStacked().DitherPost(mode=6, ampo=1)


16bit mt_lut Y8: 200fps 45% cpu 43Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

ConvertBits(bits=16)
last.mt_lut("x x +", y=3, u=1, v=1)
ConvertToStacked().DitherPost(mode=6, ampo=1)


16bit Expr Y8: 155fps 34% cpu 43Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

ConvertBits(bits=16)
last.Expr("x 111 + 3 * 100 - 2 / 2 ^ 0.02 * 4 ^ 11 +")
ConvertToStacked().DitherPost(mode=6, ampo=1)


16bit mt_lut Y8: 201fps 47% cpu 43Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

ConvertBits(bits=16)
last.mt_lut("x 111 + 3 * 100 - 2 / 2 ^ 0.02 * 4 ^ 11 +", y=3, u=1, v=1)
ConvertToStacked().DitherPost(mode=6, ampo=1)


16bit Expr Y8: 159fps 34% cpu 50Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

ConvertBits(bits=16)

p1 = last
p2 = p1.Invert("Y")

Expr(p1, p2, "x y -")
ConvertToStacked().DitherPost(mode=6, ampo=1)


16bit mt_lutxy Y8: 36fps 16% cpu 50Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

ConvertBits(bits=16)

p1 = last
p2 = p1.Invert("Y")

mt_lutxy(p1, p2, "x y -", y=3, u=1, v=1)
ConvertToStacked().DitherPost(mode=6, ampo=1)


8bit Expr Y8: 708fps 12% cpu 35Mib Memory


SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

#ConvertBits(bits=16)

p1 = last
p2 = p1.Invert("Y")

Expr(p1, p2, "x y -")
#ConvertToStacked().DitherPost(mode=6, ampo=1)


8bit mt_lutxy Y8: 636fps 12% cpu 35Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

#ConvertBits(bits=16)

p1 = last
p2 = p1.Invert("Y")

mt_lutxy(p1, p2, "x y -", y=3, u=1, v=1)
#ConvertToStacked().DitherPost(mode=6, ampo=1)


8bit Expr Y8: 276ps 12% cpu 35Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

#ConvertBits(bits=16)

p1 = last
p2 = p1.Invert("Y")

Expr(p1, p2, "x y < x x y - 0.8 * - x x y - 0.9 * - ?")
#ConvertToStacked().DitherPost(mode=6, ampo=1)


8bit mt_lutxy Y8: 614fps 12% cpu 35Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

#ConvertBits(bits=16)

p1 = last
p2 = p1.Invert("Y")

mt_lutxy(p1, p2, "x y < x x y - 0.8 * - x x y - 0.9 * - ?", y=3, u=1, v=1)
#ConvertToStacked().DitherPost(mode=6, ampo=1)


10bit Expr Y8: 220ps 12% cpu 47Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

ConvertBits(bits=10)

p1 = last
p2 = p1.Invert("Y")

Expr(p1, p2, "x y < x x y - 0.8 * - x x y - 0.9 * - ?")
#ConvertToStacked().DitherPost(mode=6, ampo=1)


10bit mt_lutxy Y8: 225fps 12% cpu 48Mib Memory

SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12")

ConvertToY8()
trim(0, 5000)

ConvertBits(bits=10)

p1 = last
p2 = p1.Invert("Y")

mt_lutxy(p1, p2, "x y < x x y - 0.8 * - x x y - 0.9 * - ?", y=3, u=1, v=1)
#ConvertToStacked().DitherPost(mode=6, ampo=1)

[pinterf]

3770k did not support AVX2 only has AVX.(When I9 PC arrivals I can test again maybe need a month.)

Many thanks!
Can you check again these two numbers? I feel that the difference is too big between them.
"x y -"
16bit Expr Y8: 159fps 34% cpu 50Mib Memory
8bit Expr Y8: 708fps 12% cpu 35Mib Memory

edit: the difference occured because of the final stacked + dither post conversion

This script eliminates the conversion overhead before the lut/expr (and no conversion occurs after, avsmeter can report the real comparison of the methods)
SetMemoryMax(3000)

bits=16 # set 8, 10, .. 16
lut=true # true:lut, false:expr
clipcount=1 # 1: clip/lut, 2: two clips/lutxy
simpleexpr=false # choose simple (true) or a more complex expression

format="YUV420P"+String(bits)
colorbars(width=1920, height=1080, pixel_type=format)
ConvertToY()
trim(0, 5000)
p1 = last
p2 = p1.Invert("Y").trim(0,-1).loop(p1.framecount()) # all cached, no speed penalty

expr_1d_simple = "x x +"
expr_1d_complex = "x 111 + 3 * 100 - 2 / 2 ^ 0.02 * 4 ^ 11 +"
expr_2d_simple = "x y -"
expr_2d_complex = "x y < x x y - 0.8 * - x x y - 0.9 * - ?"

expr = clipcount==1 ? (simpleexpr ? expr_1d_simple : expr_1d_complex) : (simpleexpr ? expr_2d_simple : expr_2d_complex)

if(clipcount==1) {
result = lut ? mt_lut(p1, expr, y=3, u=1, v=1) : Expr(p1, expr)
} else {
result = lut ? mt_lutxy(p1, p2, expr, y=3, u=1, v=1) : Expr(p1, p2, expr)
}
result

My results for avs+ x86 r2544 and masktools2 2.2.10:
i7-3770 @ 3.40 GHz (No AVX2)
results in [fps] reported by AvsMeter

Bits: 8 Lut/Expr
Simple, 1 Clip 790/700
Complex, 1 Clip 796/306
Simple, 2 Clips 540/684
Complex, 2 Clips 540/219

Bits: 16 Lut/Expr
Simple, 1 Clip 655/713
Complex, 1 Clip 664/300
Simple, 2 Clips 41.2/684 # masktools2 16 bit lutxy is not lookup but realtime calc
Complex, 2 Clips 6.45/224 # masktools2 16 bit lutxy is not lookup but realtime calc

[real.finder]

the top is Expr, then Expr with optAvx2=False, then mt_lutxy, using this https://ark.intel.com/products/75459/Intel-Core-i5-4200U-Processor-3M-Cache-up-to-2_60-GHz

https://i.imgur.com/x0gDHb1.png

colorbars()
converttoyv12
d=last
Expr(last,d,"x y -")
#Expr(last,d,"x y -",optAvx2=False)
#mt_lutxy(last,d,"x y -")

[pinterf]

What OS? Not much difference for this kind of expression with or w/o avx2

[real.finder]

What OS? Not much difference for this kind of expression with or w/o avx2

win7 64 sp1, not my pc btw

[edcrfv94]

kf_limit_dif8_128_expr_test(src_f, src, thr=1.0, elast=1.0, y=3, u=3, v=3)

expr: Stack unbalanced at end of expression. Need to have exactly one value on the stack to return.


kf_limit_dif8_128_expr_test(src_f, src, thr=1.0, elast=1.0, y=3, u=3, v=3)

mt_lut:Output is garbage


kf_limit_dif8_128_expr_test(src_f, src, thr=1.0, elast=1.01, y=3, u=3, v=3)
kf_limit_dif8_128_mt_test(src_f, src, thr=1.0, elast=1.01, y=3, u=3, v=3)

Work fine.

Maybe "- Expr optimization: eliminate ^1 +0 -0 *1 /1 " cause problems?


SetMemoryMax(3000)

colorbars(width=1920, height=1080, pixel_type="yv12").killaudio().assumefps(25, 1)

#ConvertToY8()
trim(0, 5000)
#Limiter()
#InvertNeg()
#VToY()

src = last
src_f = src.RemoveGrain(11, 11, 11)

kf_limit_dif8_128_expr_test(src_f, src, thr=1.0, elast=1.0, y=3, u=3, v=3)
#kf_limit_dif8_128_mt_test(src_f, src, thr=1.0, elast=1.0, y=3, u=3, v=3)
#kf_limit_dif8_128_expr_test(src_f, src, thr=1.0, elast=1.01, y=3, u=3, v=3)
#kf_limit_dif8_128_mt_test(src_f, src, thr=1.0, elast=1.01, y=3, u=3, v=3)

Function kf_limit_dif8_128_expr_test(clip filtered, clip original, bool "smooth", float "thr", float "elast", float "darkthr", int "Y", int "U", int "V")
{
smooth = Default(smooth, True )
thr = Default(thr, 1.0 )
elast = Default(elast, smooth ? 3.0 : 128./thr)
darkthr = Default(darkthr,thr )
Y = Default(Y, 3 )
U = Default(U, 3 )
V = Default(V, 3 )

Y = min(Y, 4)
U = min(U, 4)
V = min(V, 4)
Yt = Y == 3
Ut = U == 3
Vt = V == 3
Y31 = Yt ? 3 : 1
U31 = Ut ? 3 : 1
V31 = Vt ? 3 : 1

thr = max(min( thr, 128.0), 0.0)
darkthr = max(min(darkthr, 128.0), 0.0)
elast = max(elast, 1.0)
mode = thr == 0 && darkthr == 0 ? 4 : thr == 128 && darkthr == 128 ? 2 : 3
smooth = elast==1 ? False : smooth

diffstr = " x range_half - "
elaststr = " "+string(elast)+" "

thrstr = diffstr+" 0 > "+string(darkthr)+" scalef "+string(thr)+" scalef ? "
alphastr = elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstr+" * / ? "
betastr = thrstr+elaststr+" * "
sexpr = smooth ? alphastr+diffstr+" * "+betastr+diffstr+" abs - * range_half + "
\ : thrstr+diffstr+diffstr" abs / * range_half + "
expr = diffstr+" abs "+thrstr+" <= x "+diffstr+" abs "+betastr+" >= range_half "+sexpr+" ? ? "

thrstrc = " "+string(thr)+" scalef "
alphastrc= elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstrc+" * / ? "
betastrc = thrstrc+elaststr+" * "
sexprc = smooth ? alphastrc+diffstr+" * "+betastrc+diffstr+" abs - * range_half + "
\ : thrstrc+diffstr+diffstr" abs / * range_half + "
exprc = diffstr+" abs "+thrstrc+" <= x "+diffstr+" abs "+betastrc+" >= range_half "+sexprc+" ? ? "

# diff = filtered - original
# alpha = 1 / (thr * (elast - 1))
# beta = elast * thr
# When smooth=True :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + alpha * diff * (beta - abs(diff))
# When smooth=False :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + thr * (diff / abs(diff))

diff = mt_makediff(filtered, original, y=Y31, u=U31, v=V31)
ldiff = expr(diff, expr, exprc, exprc)
merged = mt_adddiff(original, ldiff, y=Y31, u=U31, v=V31)
merged = Y==2 || U==2 || V==2 || Y==4 || U==4 || V==4 ? mt_lutxyz(filtered, original, merged, Y=Y==3?5:Y, U=U==3?5:U, V=V==3?5:V) : merged

return mode == 4 ? original
\ : mode == 2 ? filtered
\ : merged
}

Function kf_limit_dif8_128_mt_test(clip filtered, clip original, bool "smooth", float "thr", float "elast", float "darkthr", int "Y", int "U", int "V")
{
smooth = Default(smooth, True )
thr = Default(thr, 1.0 )
elast = Default(elast, smooth ? 3.0 : 128./thr)
darkthr = Default(darkthr,thr )
Y = Default(Y, 3 )
U = Default(U, 3 )
V = Default(V, 3 )

Y = min(Y, 4)
U = min(U, 4)
V = min(V, 4)
Yt = Y == 3
Ut = U == 3
Vt = V == 3
Y31 = Yt ? 3 : 1
U31 = Ut ? 3 : 1
V31 = Vt ? 3 : 1

thr = max(min( thr, 128.0), 0.0)
darkthr = max(min(darkthr, 128.0), 0.0)
elast = max(elast, 1.0)
mode = thr == 0 && darkthr == 0 ? 4 : thr == 128 && darkthr == 128 ? 2 : 3
smooth = elast==1 ? False : smooth

diffstr = " x range_half - "
elaststr = " "+string(elast)+" "

thrstr = diffstr+" 0 > "+string(darkthr)+" scalef "+string(thr)+" scalef ? "
alphastr = elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstr+" * / ? "
betastr = thrstr+elaststr+" * "
sexpr = smooth ? alphastr+diffstr+" * "+betastr+diffstr+" abs - * range_half + "
\ : thrstr+diffstr+diffstr" abs / * range_half + "
expr = diffstr+" abs "+thrstr+" <= x "+diffstr+" abs "+betastr+" >= range_half "+sexpr+" ? ? "

thrstrc = " "+string(thr)+" scalef "
alphastrc= elaststr+" 1 <= 0 1 "+elaststr+" 1 - "+thrstrc+" * / ? "
betastrc = thrstrc+elaststr+" * "
sexprc = smooth ? alphastrc+diffstr+" * "+betastrc+diffstr+" abs - * range_half + "
\ : thrstrc+diffstr+diffstr" abs / * range_half + "
exprc = diffstr+" abs "+thrstrc+" <= x "+diffstr+" abs "+betastrc+" >= range_half "+sexprc+" ? ? "

# diff = filtered - original
# alpha = 1 / (thr * (elast - 1))
# beta = elast * thr
# When smooth=True :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + alpha * diff * (beta - abs(diff))
# When smooth=False :
# output = diff <= thr ? filtered : \
# diff >= beta ? original : \
# original + thr * (diff / abs(diff))

diff = mt_makediff(filtered, original, y=Y31, u=U31, v=V31)
ldiff = mt_lut(diff, yexpr=expr, uexpr=exprc, vexpr=exprc, y=Y31, u=U31, v=V31)
merged = mt_adddiff(original, ldiff, y=Y31, u=U31, v=V31)
merged = Y==2 || U==2 || V==2 || Y==4 || U==4 || V==4 ? mt_lutxyz(filtered, original, merged, Y=Y==3?5:Y, U=U==3?5:U, V=V==3?5:V) : merged

return mode == 4 ? original
\ : mode == 2 ? filtered
\ : merged
}

[pinterf]

Missing + sign?
I can see
diffstr+diffstr"
instead of
diffstr+diffstr+"

Edit: elast=1 -> smooth=False, different expressions, syntax error was in the smooth=false branch

[edcrfv94]

Missing + sign?
I can see
diffstr+diffstr"
instead of
diffstr+diffstr+"

Edit: elast=1 -> smooth=False, different expressions, syntax error was in the smooth=false branch

You are right, thanks! *mt_lut no error message at Stack mistakes?

[real.finder]

any news on scaling things for both expr and lut?

[pinterf]

I am still working on Expr but somehow stuck because this version of jit asm used in the ad-hoc assembler code generation is failing and generates false code for the feature I'm experimenting with, resulting in random nice access violation messages. Ordinary expressions are not affected however. I'd like to understand why it fails. It already took a lot of days and I'm not seeing the end of the tunnel. When I'm ready, I can start making the feature list more similar between mt_lut family and Expr. So it's not forgotten just takes more time than expected.

[wonkey_monkey]

I've been writing an assembler and have looked (briefly) at expr, so if you need another pair eyes on it... got any machine code output to share?

[pinterf]

I've been writing an assembler and have looked (briefly) at expr, so if you need another pair eyes on it... got any machine code output to share?
Thanks. The problem is that jitasm tries to be smart. When there are not enough working registers, it swaps one to two to a memory slot, use the register(s) for the specific purpose, then restores their previous state. Sometimes jit makes optimization in such a way that it exchanges the appropriate working registers (e.g. xchg esi,ecx). For example let's have a read pointer in esi, frame width in ecx. Normally jitasm generates code that is reading pixel from esi which is our pixel pointer. But if we use labels and jumps, jit may generate register exchanges but forgets about the exchange, and uses ecx for pixel readout, which is obviously not a pointer.
See here (https://github.com/pinterf/AviSynthPlus/commit/4d69a103bc58a5897c960696158a1ce1b488839e)
(Later I found your work, and I am working on similar features, namely x-y offsets and storing current result into a variable slot.)

[raffriff42]

Consider the following test:function _test(clip C)
{
C
M = BlankClip(Last) [* any old mask *]
\ .Subtitle("TEXT", text_color=color_white, size=Height, align=2)
\ .Tweak(cont=0.5, bright=32)
\ .BilinearResize(Width/8, Height/8)
\ .GaussResize(Width, Height, p=19)
#return M

#[[ pick one
return mt_merge(Grayscale, M, luma=true).Subtitle("mt_merge")
#return Overlay(Grayscale, mask=M).Subtitle("Overlay")
#]]
}
BlankClip(width=512, height=256, pixel_type="YV12", length=1024, color=color_red)
FadeOut(600)
_test
return Last


Here is the expected output:
https://www.dropbox.com/s/x6hsj7k6ju5k78g/mt_merge-bug-00.png?raw=1

Here is mt_merge:
https://www.dropbox.com/s/cdnv4lwj8tyotsl/mt_merge-bug-01.png?raw=1

Tested with YV12, YUV420P10..YUV420PS - all do it (although the pattern changes a bit)

[poisondeathray]

I can reproduce rr42's bug in avisynth+ x64 with masktools2.dll 2.2.10.0.

But it doesn't seem to affect the same masktools2.dll 2.2.10.0 in x86 flavour in avisynth "classic" . I don't have avisynth+ x86 installed to test right now

[Motenai Yoda]

it works for me
avs+ v2574 x86 & x86_64
masktools2 v2.2.10.0

[pinterf]

It's wrong when avx2 is used.
This works:
return mt_merge(Grayscale, M, luma=true, avx2=false).Subtitle("mt_merge")
I'll prioritize the fix.

EDIT: fixed on git, no release yet

[real.finder]

any news for scale things?

[pinterf]

Not forgotten. Already have started to rework the internals first to have some of the Expr syntax that is not implemented in masktools. Regarding your old request for mpeg1 chroma option in luma=true, would it really be visibly superior than without it?

[real.finder]

Not forgotten. Already have started to rework the internals first to have some of the Expr syntax that is not implemented in masktools.

thanks, will wait for that then

Regarding your old request for mpeg1 chroma option in luma=true, would it really be visibly superior than without it?

it will not be a big difference, but it always good to have the right things, similar to chroma resize bug (always center, so it will be technically wrong in most cases) in internal avs resizes, there will be people care about it

[pinterf]

New build, important fix for AVX2 users for mt_merge, luma=true
Download Masktools2 2.2.11 (https://github.com/pinterf/masktools/releases/tag/2.2.11)

**v2.2.11 (20180105)
- Fix: mt_merge luma=true: broken output when: 8-16 bits AVX2, 32 bit float: SSE2, AVX
- move project to VS2017, vs141_xp toolset

[raffriff42]

Sorry to say, the test fails with widths that are not mod32 -- a little "crud" (mask image wraparound?) is visible on the right side.

EDIT
1) if you use my test above, BilinearResize will have to be modded to support arbitrary sizes. Of course
2) avx2=false fixes the problem, and it is a little faster for me!

[pinterf]

Sorry to say, the test fails with widths that are not mod32 -- a little "crud" (mask image wraparound?) is visible on the right side.

EDIT
1) if you use my test above, BilinearResize will have to be modded to support arbitrary sizes. Of course
2) avx2=false fixes the problem, and it is a little faster for me!
Indeed, there are problems with 10-16 bit 420 and 422 sources, when they are not mod8 (SSE2) or mod16 (AVX2) width, on the right side, where no fast SIMD but only the C code works. For 8 bits I can see no artifacts, what is your exact script?

[raffriff42]

Sorry, I meant to say, not mod64 AND bit depth > 8 && < 32.
Didn't mean to imply there was a new bug where there was none before.

EDIT my script -BlankClip(width=512-4, height=256, pixel_type="YUV420P10", color=color_red)

M = BlankClip(Last)
\ .Subtitle("X", text_color=$ffffff, size=Height, align=2)
#return M

s="""
mt_merge(Grayscale, M, luma=true,
\ sse2=1>0, sse3=1>0, ssse3=1>0,
\ sse4=1>0, avx=1>0, avx2=0>0)
"""
Eval(s)

Subtitle(PixelType
\ +"\n"+"width="+String(Width)
\ +ReplaceStr(s, Chr(10), "\n"),
\ lsp=0)
return Last

https://www.dropbox.com/s/8bxnfy2vu0jrxy0/mt_merge-bug-03.png?raw=1

https://www.dropbox.com/s/bqx134sn3q03wl5/mt_merge-bug-04.png?raw=1

[pinterf]

Sorry, I meant to say, not mod64 AND bit depth > 8 && < 32.

Thanks, this one is fixed, probably the last one from this group.
Now I have run the speed tests (mt_merge alone), encoding tests (whole script above) to have the exact same files for avx2/nonAvx2 case, and visual tests with all combinations of 420, 422, 444 / 8, 14, 16 bits and float / avx2=false or true / width=512 and 512-4.
(10-14 and 16 bit versions differs a bit internally)

[pinterf]

New build, fixing an old bug, thank you raffriff42!

Download masktools2 2.2.12 (https://github.com/pinterf/masktools/releases/tag/2.2.12)

**v2.2.12 (20180107)**
- Fix: mt_merge 10-16 bits: right side artifacts when clip has non-mod 8 (non-AVX2) or mod16 (AVX2) width

[8day]

Either I'm doing something wrong, or MT_LUTxyza doesn't work: it always returns "z"-clip instead of "a"-clip. Tested using 2.2.0 and >=2.2.9 at Windows 7 with AviSynth 2.6 using different filter chains.
MT_LUTxyza(
\ ColorBars(pixel_type="YV12").Subtitle("X"),
\ ColorBars(pixel_type="YV12").Subtitle("Y"),
\ ColorBars(pixel_type="YV12").Subtitle("Z"),
\ ColorBars(pixel_type="YV12").Subtitle("A"),
\ "a"
\ )

[poisondeathray]

Either I'm doing something wrong, or MT_LUTxyza doesn't work: it always returns "z"-clip instead of "a"-clip. Tested using 2.2.0 and >=2.2.9 at Windows 7 with AviSynth 2.6 using different filter chains.
MT_LUTxyza(
\ ColorBars(pixel_type="YV12").Subtitle("X"),
\ ColorBars(pixel_type="YV12").Subtitle("Y"),
\ ColorBars(pixel_type="YV12").Subtitle("Z"),
\ ColorBars(pixel_type="YV12").Subtitle("A"),
\ "a"
\ )


This works correctly for me
avs+ r2574 x64 , masktools2-v2.2.12, win8.1

[8day]

This works correctly for me
avs+ r2574 x64 , masktools2-v2.2.12, win8.1

Sorry, my bad. Turns out it works. On the other hand, I figured out what caused the issue -- MT_Polish(). This is the code that doesn't work:
MT_LUTxyza(
\ ColorBars(pixel_type="YV12").Subtitle("X"),
\ ColorBars(pixel_type="YV12").Subtitle("Y"),
\ ColorBars(pixel_type="YV12").Subtitle("Z"),
\ ColorBars(pixel_type="YV12").Subtitle("A"),
\ MT_Polish("a")
\ )


That being said, despite that MT_LUTxyza() returns any of these clips when I use raw string, doing any calculations breaks something pretty heavy. E.g., "a-z", "a+128", "x*1" or even "x+" results in same darkened frame. Thought it may be something wrong with AvsPmod, but VirtualDub shows same image. v2.2.1, the first build with this function, has same issues.

[burfadel]

There seems to be something wrong with MT_Lut in regards to bit depth handling. For one, even though the docs says 'When not defined, all threshold-like and Y, U, V, A parameter values are treated as 8 bit values and will be autoscaled accordingly to match the clip bit depths. '. It doesn't! You can't use the scaleparams function either, as it says mt_lut doesn't have that function. Not sure if the other lut functions are affected.

This is with the latest 2.2.12

[pinterf]

Sorry, what you found, is a typo, will fix it.
Search for paramscale
http://avisynth.nl/index.php/MaskTools2

[burfadel]

Sorry about the second thing, that was my typo :D. The first thing is still valid though, in that it doesn't scale automatically like the other functions.

[pinterf]

Sorry about the second thing, that was my typo :D. The first thing is still valid though, in that it doesn't scale automatically like the other functions.
Found something. Are you trying with chroma="somenegativevalue"?

[burfadel]

The colour format is an extracted Luma channel. I use it for a mask using
passmask = mt_lut (cy, "x "+string(32)+" < 0 x "+string(45)+" > "+string(255)+" 0 x "+string(35)+" - "+string(255)+" "+string(32)+" "+string(65)+" - / * - ? ?")

Where cy is a 12 bit Luma extract using ExtractY. If you view the mask as 8 bit extract it works, but in 12 bit it is essentially black due to not being scaled. On the same 12 bit clip functions like mt_binarize(cy, 128) works fine and scales to 12 bit automatically.

[pinterf]

Ok, then I found a bug that you haven't think of. We can nicely work together :)
ConvertBits(10).mt_lut("x 2 *",chroma="-128")
# sets a YUV clip to greyscale. 8 bit OK, 10-14 and 32 bits fail ("chroma" parameter issue only)

Back to your problem: the expression interpreter cannot decide from a constant whether it is scalable or not (think of a "x y + 2 /", is "2" something to scale or not?).

scaleparam is only for parameter (Y,U,V, thresholds, etc..) scaling, not the expression itself.

So you have to insert manually the scaleb (bitshift scale, usually for YUV) or scalef (full range scale). Like "235 scaleb" which will mean luma max for each bitdepth.
You can use some predefined constants which will have always a properly scaled value, like range_max (255/1023/4095/16383/65535 or 1.0 for float), range_size --> 256/1024...65536, ymin, ymax, cmin, cmax --> the 16/235 and 16/240 limits autoscaled.

Usually you would use range_max instead of 255, and scaleb for the other constants in your sample.

Same rules apply on the Expr filter syntax in Avisynth+.

[burfadel]

Ah ok, so I can only apply it on the source with chroma, or 8-bit, when not specifically stating to scale in 8-bit?

[pinterf]

Ah ok, so I can only apply it on the source with chroma, or 8-bit, when not specifically stating to scale in 8-bit?
The sample above should work (and will work in a soon-to-be released 2.2.13) for all bit depths because parameter scaling is from 8 bits (paramscale="i8" default) and it works fine for other parameters that are using negative numbers for pixel-fill.

[pinterf]

New build

Masktools2 v2.2.13 (https://github.com/pinterf/masktools/releases/tag/2.2.13)

**v2.2.13 (20180201)
- Fix: rare crash in multithreading environment at the very first frames
(keeping XP compatibility with /Z:threadsafeinit- caused troubles!)
- mt_edge: AVX2 (1.4-1.9x speed) for 8 and 10-16 bits
- fix: "chroma" parameter with negative (memset) values were not working properly for 10-14 bits and 32bit float

That stupid crash took me weeks to catch until it turned out that a mysterious access violation is happening right in masktools2.

The original report was: ffdshow x64 crashed with a script using maa2. I have almost completely un-mmx'd ffdshow, and moved it to avs 2.6 interface from v2.5, changed things that existed in boost many years ago but now C++14/17 adopted them, so it finally could be compiled and debugged with VS2017 (with some features disabled). But the crash was still there, pointing to avisynth or one of its filters. Then I succeeded to reproduce the crash w/o ffdshow with a simple script:
c=BlankClip(width=620,height=600,length=1000,pixel_type="YV12")
mt_merge(c,c,c,y=-2,u=-2,v=-2,luma=true)
#Exception 0xC0000005 [STATUS_ACCESS_VIOLATION]
#Module: C:\Windows\System32\KernelBase.dll
#Address: 0x00007FF9A5313FB8
Prefetch(2)


It was the side effect of the /Zthreadsafeinit- switch. This option is used for keeping stupid XP compatibility and caused a static class to be initialized later in one thread than it was used from another thread. Specifically in MT environment GetFrame(0) started the initializing some variables but GetFrame(1) from another thread kicked in too early and couldn't yet see the changes that the first thread was doing.

Of course all this happened only in Release mode because it needed strict timing conditions, maybe sub-milliseconds so it could not be debugged. When will XP die????

[TheFluff]

When will XP die????

You are as free as a bird. You can kill it whenever you like.

[VS_Fan]

When will XP die????This can help: According to statcounter's "Desktop Windows Version Market Share Worldwide (http://gs.statcounter.com/windows-version-market-share/desktop/worldwide#monthly-201701-201801)":

Windows XP market share dropped from almost 6% to about 3% during the last year.
Windows XP market share has consistently lost more than 3% market share yearly since January 2015, three years ago, when it had almost 14%.
Windows XP’s big brother, windows 2003, disappeared from those statistics in January 2015, three years ago.

Support for Windows XP has ended (https://www.microsoft.com/en-us/WindowsForBusiness/end-of-xp-support) almost 4 years ago:
After 12 years, support for Windows XP ended April 8, 2014. Microsoft will no longer provide security updates or technical support for the Windows XP operating system

[Andouille]

Microsoft will no longer provide security updates or technical support for the Windows XP operating system

And ? Video processing fails for that reason ?

[Morku]

And ? Video processing fails for that reason ?

Feel free to use outdated software for an outdated OS. No need for current software or fix compatibility yourself.

TheFluff already said everything.

[Andouille]

Feel free to use outdated software for an outdated OS or fix compatibility yourself.

TheFluff already said everything.

Define "outdated OS" for video processing.

[Groucho2004]

Not this again. :devil:

If one wants to keep using XP one has to live with the fact that more and more software won't work on it. It's as simple as that.

[Myrsloik]

This can help: According to statcounter's "Desktop Windows Version Market Share Worldwide (http://gs.statcounter.com/windows-version-market-share/desktop/worldwide#monthly-201701-201801)":

Windows XP market share dropped from almost 6% to about 3% during the last year.
Windows XP market share has consistently lost more than 3% market share yearly since January 2015, three years ago, when it had almost 14%.
Windows XP’s big brother, windows 2003, disappeared from those statistics in January 2015, three years ago.

Support for Windows XP has ended (https://www.microsoft.com/en-us/WindowsForBusiness/end-of-xp-support) almost 4 years ago:
After 12 years, support for Windows XP ended April 8, 2014. Microsoft will no longer provide security updates or technical support for the Windows XP operating system

All those 3% hang out on doom9 so your argument is invalid (locally) :D

[TheFluff]

Define "outdated OS" for video processing.

You know, this really is an exceptionally poorly thought out argument to make when a very annoying and hard-to-debug problem caused by attempting to use Windows XP's outdated multithreading support was described like three posts above yours.

More generally speaking though, it's not so much that the OS itself is badly suited to video processing (it definitely is though, but in indirect ways) - it's more that developing software for it is becoming harder because development tools are dropping support for it and some kernel-level API's (such as the aforementioned multithreading stuff) aren't available on it. Sure, you could go out of your way to implement replacement functionality for it yourself, but why would you go to all that effort for something used by like three people? It really isn't reasonable to demand that from a hobbyist developer. It's open source, fix it yourself if it's that important to you. Or use older versions.

[Andouille]

My point is who cares about "security" when processing audio/video ?
We are not speaking about internet browsing or piracy sites.

[Andouille]

@TheFluff
My MT avisynth is just as stable as a rock.
I don't even ask new plugins.
The only thing would be source plugin. (Thx Quyot)

[TheFluff]

@TheFluff
My MT avisynth is just as stable as a rock.
I don't even ask new plugins.
The only thing would be source plugin. (Thx Quyot)

why are you here hurfing durf about it then

[Andouille]

why are you here hurfing durf about it then

My hair is fluffy.

[real.finder]

That stupid crash took me weeks to catch until it turned out that a mysterious access violation is happening right in masktools2.

The original report was: ffdshow x64 crashed with a script using maa2. I have almost completely un-mmx'd ffdshow, and moved it to avs 2.6 interface from v2.5, changed things that existed in boost many years ago but now C++14/17 adopted them, so it finally could be compiled and debugged with VS2017 (with some features disabled). But the crash was still there, pointing to avisynth or one of its filters. Then I succeeded to reproduce the crash w/o ffdshow with a simple script:
c=BlankClip(width=620,height=600,length=1000,pixel_type="YV12")
mt_merge(c,c,c,y=-2,u=-2,v=-2,luma=true)
#Exception 0xC0000005 [STATUS_ACCESS_VIOLATION]
#Module: C:\Windows\System32\KernelBase.dll
#Address: 0x00007FF9A5313FB8
Prefetch(2)


It was the side effect of the /Zthreadsafeinit- switch. This option is used for keeping stupid XP compatibility and caused a static class to be initialized later in one thread than it was used from another thread. Specifically in MT environment GetFrame(0) started the initializing some variables but GetFrame(1) from another thread kicked in too early and couldn't yet see the changes that the first thread was doing.

Of course all this happened only in Release mode because it needed strict timing conditions, maybe sub-milliseconds so it could not be debugged. When will XP die????

so it was bug in masktools after all, about 1 or 2 month ryrynz told me that there crash happen with maa2 in avs+ 64 if using mt

if only x64 has this bug then you can let xp compatibility die for x86_64 and keep it only x86 in 32bit

people that use x64 pc should not use xp anyway since their pc can run another more modern os

[manolito]

For the market share stats it really depends where you look...
According to netmarketshare the numbers are a little different:
https://www.netmarketshare.com/operating-system-market-share.aspx?options=%7B%22filter%22%3A%7B%22%24and%22%3A%5B%7B%22deviceType%22%3A%7B%22%24in%22%3A%5B%22Desktop%2Flaptop%22%5D%7D%7D%5D%7D%2C%22dateLabel%22%3A%22Trend%22%2C%22attributes%22%3A%22share%22%2C%22group%22%3A%22platformVersion%22%2C%22sort%22%3A%7B%22share%22%3A-1%7D%2C%22id%22%3A%22platformsDesktopVersions%22%2C%22dateInterval%22%3A%22Monthly%22%2C%22dateStart%22%3A%222017-02%22%2C%22dateEnd%22%3A%222018-01%22%2C%22segments%22%3A%22-1000%22%7D

But whatever, who cares if XP has a market share of 3% or 7%? Fact is that it still is #3 (maybe #4) of all desktop OSs.


I would like to make a different point. As someone who always has 2 machines running in parallel, one ancient desktop on XP and a newer Thinkpad T530 on Win7-64, I do know the differences. And I do know why I prefer XP... :devil:

Having read all the stuff here at Doom9 about AVS+ and all the plugin modernization efforts, I can only say that most of this stuff is work in progress, it is far from stable. Mostly this refers to Pnterf's stuff (AVS+, MVTools2, MaskTools2 and more). I commend him for doing all the work he did, but the results often look like he updated stuff when he really did not understand the original code.

After playing a lot with the "modernized" plugins on my Win7 machine, I finally reverted to the old and stable tools (which also work under XP). Sure they are slower and do not offer "modern" features like high bit depth and advanced color spaces. But THEY JUST WORK. And I use these tools for "real" conversions, I do not have the time to chase down bugs.


Cheers
manolito

[VS_Fan]

After playing a lot with the "modernized" plugins on my Win7 machine, I finally reverted to the old and stable tools (which also work under XP). Sure they are slower and do not offer "modern" features like high bit depth and advanced color spaces. But THEY JUST WORK. And I use these tools for "real" conversions, I do not have the time to chase down bugs.
That statement makes one think you don't even use MT and, more importantly, you didn’t understand pinterf’s question:
New build ...
- Fix: rare crash in multithreading environment at the very first frames
(keeping XP compatibility with /Z:threadsafeinit- caused troubles!)...
That stupid crash took me weeks to catch until it turned out that a mysterious access violation is happening right in masktools2.
... When will XP die????

[manolito]

That statement makes one think you don't even use MT

You are absolutely correct:
I use plain vanilla AviSynth 2.60 which is 32bit and single threaded. And I plan to stick with it for the time being. My encoding bottle neck is not AVS filters, it is the encoder.


Cheers
manolito

[burfadel]

Can I just confirm the behaviour of mt_lut with different bit depths? I'll give some example screenshots of what is happening (including with 2.2.13). It's a random video frame.

Source, 8 bit:
https://i.imgur.com/8GxF68D.jpg

mt_lut with 8 bit:
https://i.imgur.com/obqEzeB.jpg

mt_lut 10 bit by:
convertbits(10)
mt_lut ( "x "+string(32)+" < 0 x "+string(45)+" > "+string(255)+" 0 x "+string(35)+" - "+string(255)+" "+string(32)+" "+string(65)+" - / * - ? ?" )
convertbits(8)
https://i.imgur.com/nihtUVR.jpg

Same as above regarding convertbits but with paramscale (slightly different frame of scame clip, but it's the same effect right through the video as expected:
mt_lut ( "x "+string(32)+" < 0 x "+string(45)+" > "+string(255)+" 0 x "+string(35)+" - "+string(255)+" "+string(32)+" "+string(65)+" - / * - ? ?", paramscale="i8")
https://i.imgur.com/M72wif0.jpg

Now, here's the same frame as the first images but using mt_binarize in 10 bit:
convertbits(10)
mt_binarize(64)
convertbits(8)
https://i.imgur.com/JlDJxcr.jpg

So, regardless of anything, mt_lut does not scale to the 8-bit values when not under 8 bit, so the mask wouldn't work. However, mt_binarize does automatically scale to the 8 bit values, so the 64 works.

Is that correct behaviour of mt_lut in comparison to mt_binarize?

[pinterf]

Can I just confirm the behaviour of mt_lut with different bit depths?
[...]
mt_lut ( "x "+string(32)+" < 0 x "+string(45)+" > "+string(255)+" 0 x "+string(35)+" - "+string(255)+" "+string(32)+" "+string(65)+" - / * - ? ?", paramscale="i8")

So, regardless of anything, mt_lut does not scale to the 8-bit values when not under 8 bit, so the mask wouldn't work. However, mt_binarize does automatically scale to the 8 bit values, so the 64 works.

Is that correct behaviour of mt_lut in comparison to mt_binarize?
Expression string is not an additional parameter which is affected by paramscale in any way. Unlike the threshold parameter in binarize. A filter can see nothing from your string contacenating, it will see only the assembled result. String(number) is not a parameter, just a part of a string expression which is resolved by Avisynth during script parsing.

Try this one:
mt_lut ( "x 32 scaleb < 0 x 45 scaleb > range_max 0 x 35 scaleb - range_max 32 scaleb 65 scaleb - / * - ? ?")

[FranceBB]

@VS_Fan... multi-threading in Avisynth totally depends on how the coder wrote the filter, 'cause Avisynth itself is single thread.
In other words: whoever is writing a plugin should implement multi-threading, otherwise it's gonna be single-thread.
MT mode it's a way to speed things up by running multiple instances of single-thread filters, but it's not perfect, it doesn't work with every single filter and should not be used with multi-threading filters, 'cause it would lower performances or worse (make AVS crash).

When will XP die?
I'm using Windows Server 2016 on my server and Linux on my home desktop, but I'm also running Windows XP Professional x86 on a VM and I'm planning to keep it 'till the end of extended support (April 2019).
Anyway, I understand that supporting XP might require some additional work, but I think it would be really useful to keep XP compatibility.
And, by the way, I really appreciate what you have done in these years; I really do. A big fat thank you ^_^

@manolito... I'm using standard Avisynth too and I admit that although it's not bullet proof, it's actually very reliable.

[StainlessS]

Whether tis 3% or 7% or some other, that is one helluva lot of machines, and will likely be slanted due to many XP m/c's being used offline,
just because you cannot see/count them, dont mean that they dont exist.

[feisty2]

I'm just curious, what's so special about windows xp? there're many windows operating systems predating windows xp, how come some people could upgrade from those even earlier operating systems to xp and then just stopped, why aren't they still using stuff like DOS, oh no, why are they using computers and electricity?

[manolito]

Because Windows did indeed improve up to XP (with the exception of the Millennium version which came out after Win98SE). After WinXP everything went downhill. Win8 and Win8.1 was a joke (at least for users without touchscreens), and Win10 is an absolute nightmare for me. Leaves Win7 which is not too bad, but only after tweaking dozens of things to make it resemble XP. This includes turning off this stupid UAC, tame this whole permissions thing, disable Virtual Store (I want to know where my files REALLY are stored). Also getting rid of the built-in ZIP support, installing Classic Startup Menu, make Windows use the DS source filters which I have installed instead of the crappy MS filters, and some more...

After applying all these tweaks I can live with Win7. But MS is actively working to put Win7 into the position WinXP is in now. Just read that their upcoming Office 2019 will need Win10. Once they succeed in making Win7 unusable I will certainly say goodbye to MS. I hope that ReactOS will be ready by then...

[feisty2]

It's ur personal preference, u personally can't let go of the older windows user interface and that doesn't mean the new windows user interface is any worse than the older one, a personal preference is not a valid point to judge the good and bad of an operating system, btw the default start menu in xp is also different from previous windows versions, and I bet user like u surely won't take that default start menu in xp, u have to make that 98 style :)

[manolito]

and I bet user like u surely won't take that default start menu in xp, u have to make that 98 style :)

How in the world did you know that? :devil:

I even have Win98SE still installed on a separate partition because I can't live without the old Delrina Winfax (newer fax software cannot touch it, and at least in Germany the legal situation requires oftentimes to send a fax instead of email).


Cheers
manolito

[TheFluff]

@VS_Fan... multi-threading in Avisynth totally depends on how the coder wrote the filter, 'cause Avisynth itself is single thread.
In other words: whoever is writing a plugin should implement multi-threading, otherwise it's gonna be single-thread.
MT mode it's a way to speed things up by running multiple instances of single-thread filters, but it's not perfect, it doesn't work with every single filter and should not be used with multi-threading filters, 'cause it would lower performances or worse (make AVS crash).
Explaining things you don't actually understand as if you were an authority on the subject to someone who does understand it is generally regarded as rather rude. Your understanding is roughly kinda sorta correct for how Avs-MT worked circa 2010, but not now.

[StainlessS]

Nuttin bettter than WXP with all of that crap turned off, ie in W95/98 mode,
None of them daft transluscent blue menus to appreciate, hit a menu, it apprears, we dont have to 'appreciate it.

Pinterf, if you choose to abandon, I can accept and understand that, so can all others that live in the past, if that is what you want, go for it,

Of course I would personally be disapointed, but, who cares, shit on me, and Well I dont blame you.

Do what you will, to us lowely scoudrels. :) [EDIT: I'm quite sure that we will get by.]

EDIT: My fav was w2K, now that was a blast.(I hated XP, my hate got bigger and shifted focus with appearance of Vista).

[real.finder]

I'm just curious, what's so special about windows xp? there're many windows operating systems predating windows xp, how come some people could upgrade from those even earlier operating systems to xp and then just stopped, why aren't they still using stuff like DOS, oh no, why are they using computers and electricity?

well, many people love xp, included me, in 2003 my dad Pentium 4 pc was have 2 os, 1st Windows ME and 2nd win xp, since back then some people still use win me for some reason, but I used to use xp, and still sometimes in VM nowadays, it's generally nice os

https://www.youtube.com/watch?v=UZas45Zn9zU

[sneaker_ger]

The wiki still says VC2015 but for the new release you need VC2017, correct? Can someone with access (and knowledge) update it?

http://avisynth.nl/index.php/MaskTools2#Download

[raffriff42]

readme for version 2.2.12 says "Visual Studio 2015 Update 3"
Confirmed with Dependency Walker (http://www.dependencywalker.com/).

EDIT sorry, somehow I missed the new release 2.2.13

[sneaker_ger]

New build
[...]
so it finally could be compiled and debugged with VS2017 (with some features disabled).
:confused:

[pinterf]

Visual Studio 2015 and 2017 redistributables are interchangeable, both versions are 14.xx.
For example when you install Visual Studio 2017 Redistributables, it will remove VS2015 version.
@raffriff42: I missed editing that section. At the top of the page I have modified the text to "Requires Visual Studio redistributables (14.xx family)"
https://docs.microsoft.com/en-us/cpp/porting/binary-compat-2015-2017

[ajp_anton]

function testing(clip c,bool info)
{ ... }

sourcefilter
testing(info=true)

error: invalid arguments.
Works if I make the argument "info" optional, or skip the argument name when calling the function.

[StainlessS]

function testing(clip c,bool info)
{ ... }

sourcefilter
testing(info=true)

error: invalid arguments.
Works if I make the argument "info" optional, or skip the argument name when calling the function.

It has always worked that way, non optionals have to be provided in given order and without naming, just giving in function prototype in quotes will accept as optional, but, must use eg x=Default(x,true) internally, else crash when not supplied.

[ajp_anton]

It has always worked that way, non optionals have to be provided in given order and without naming, just giving function prototype in quotes will accept as optional, but, must use eg x=Default(x,true) internally, else crash when not supplied.
May I ask why? Is it intentionally left this way, or can/will it be changed?

And I just realized I posted this in the wrong topic (thought I was in the avs+ topic...). A mod can move these posts if they want.

[StainlessS]

Is it intentionally left this way, or can/will it be changed

1) yes.
2) unlikely.

Non optionals are compulsory, and in compulsory order, so apart from anything else, names in those cases are superfluous.

EDIT: Names are used so as to accept optional defaults, and specify out-of-order optionals where is convienient.

[pinterf]

Fix mt_convolution 10-16 bit mode for SSE2/SSSE3-only processors

Download masktools2 v2.2.14 (https://github.com/pinterf/masktools/releases/tag/2.2.14)

Changelog
**v2.2.14 (20180225)
- Fix: mt_convolution invalid instruction on processors below SSE4.1


Thanks for zub35 for the report

[pinterf]

New release.
Download masktools2 v2.2.16 (https://github.com/pinterf/masktools/releases/tag/2.2.16)

This version is partly an update for handling 32 bit float the same way as Avisynth+ r2728 does.

And there are some quite significant new features.

Special thanks to real.finder for his ideas and testing.
I hope a couple of scripts will benefit from the changes, both quality and speedwise.

Changelog
**v2.2.16 (20180702)
- mt_merge new parameter hint for chroma placement when luma=true and 4:2:0/4:2:2
String 'cplace': possible values "mpeg1" or "mpeg2" (default)
Parameter is effective only for 420 and 422 formats, otherwise ignored.
Default "mpeg1" is using fast 2x2 pixel (1x2 for 4:2:2) averaging when converting a 4:4:4 mask to a 4:2:0 or 4:2:2 format (old behaviour)
420 schema:
+------+------+
| 0.25 | 0.25 |
|------+------|
| 0.25 | 0.25 |
+------+------+

"mpeg2" is using 2x3 (1x3 for 4:2:2) pixel weighted averaging when converting a 4:4:4 mask to a 4:2:0 or 4:2:2 format
420 schema:
------+------+-------+
0.125 | 0.25 | 0.125 |
------|------+-------|
0.125 | 0.25 | 0.125 |
------+------+-------+

- 32 bit float U and V chroma channels are now zero based (+/-0.5 for full scale). Was: 0..1, same as luma
Since internal format changed, use Avisynth+ r2724 or newer for this masktools2 2.2.16.
Affected predefined expression constants when plane is U or V:
cmin and cmax (limited range (16-128)/255 and (240-128)/255 instead of 16/255.0 and 240/255.0
range_max: 0.5 instead of 1.0
new: introduce range_min: -0.5 for float U/V chroma, 0 otherwise
range_half (0.0 instead of 0.5)
(range_size remained 1.0)
- New expression syntax for Lut expressions: autoscale any input (x,y,z,a) bit depths to 8-16 bits for internal
expression use. The primary reason of this feature is the "easy" usage of formerly written 8 bit optimized expressions.

New parameters for lut functions:
String "scale_inputs": "all","allf","int","intf","float","floatf","none", default "none"
and
Boolean "clamp_float": default false, but treated as always true (and thus ignored) when scale_inputs involves a float autoscale.
and
Boolean "use_expr": default 0, calls fast JIT-compiled "Expr" in Avisynth+ for mt_lut, lutxy, lutxyz, lutxyza
0: no Expr, use slow internal realtime calc if needed (as before)
1: call Expr for bits>8 or lutxyza
2: call Expr, when masktools would do its slow realtime calc (see 'realtime' column in the table above)

Extends and replaces experimental clamp_xxxx keywords.

[FranceBB]

Thank you for the new release! :D

[real.finder]

seems both use_expr=1 and use_expr=2 are same (both use_expr=1)

[pinterf]

Decisions for a single 1D mt_lut:
use_expr=1 (when bit depth is more than 8 bits): pass expression to Avs+ Expr filter for 10 or more bits
use_expr=2 (when no LUT is used) this is only for 32 bit float. 8-16 bit uses LUT, for 1D it has enough memory for the table.

Checked for a 16 bit clip.

For short expressions such as a simple "x 2 *" passing it to Expr is even faster than LUT.
For more complex ones 1D LUT is still faster, it depends.

[real.finder]

Decisions for a single 1D mt_lut:
use_expr=1 (when bit depth is more than 8 bits): pass expression to Avs+ Expr filter for 10 or more bits
use_expr=2 (when no LUT is used) this is only for 32 bit float. 8-16 bit uses LUT, for 1D it has enough memory for the table.

Checked for a 16 bit clip.

For short expressions such as a simple "x 2 *" passing it to Expr is even faster than LUT.
For more complex ones 1D LUT is still faster, it depends.

isn't use_expr=2 for use expr all times (8-32 bits) in avs+? even so I note use_expr=2 work with 16 bit now! (at least in the scripts I did edited) and option that use expr all times is needed if it not existing

[pinterf]

isn't use_expr=2 for use expr all times (8-32 bits) in avs+? even so I note use_expr=2 work with 16 bit now! (at least in the scripts I did edited) and option that use expr all times is needed if it not existing

Readme* says:
Possible values:
0: uses lut and internal realtime calculation
1: Expr, when bit depth>=10 or lutxyza
2: When masktools would use realtime calc

E.g. for use_expr=2:
8-16 bit lut is still LUT
8-12 bit lutxy is passed to Expr
14-16 and 32 bit lutxy is passed to Expr

E.g. for use_expr=1:
8 bit lut is still LUT
10-16 and 32 bit lut is Expr
8 bit lutxy is still LUT
10-16 and 32 bit lutxy is passed to Expr

The "feature matrix" in readme shows 'realcalc' bit depth limits for th different lut versions. When realcalc working mode kicks in, Expr is used instead for use_expr=2

*Masktools2 page on avisynth.nl is not yet updated

[real.finder]

I see

so what about use_expr 1 will be When masktools would use realtime calc

2 will be when more than 8 bit

3 always use expr in avs+

edit: or 1 mean always and 2 mean more than 8 and 3 mean if realtime

[pinterf]

What is the point of "always"? Use then Expr directly. It should be even a little faster because Expr is now Invoked and compiled from each GetFrame and not only once, during the filter construction.

[real.finder]

What is the point of "always"? Use then Expr directly. It should be even a little faster because Expr is now Invoked and compiled from each GetFrame and not only once, during the filter construction.

the point is make less lines in scripts, since there are no expr() in normal avs

[pinterf]

And pass even an 8 bit 1D lut to Expr? (I have started updating Expr in Avs+ to be able to work like LUT which is faster for some 1D or 2D cases, but had no time to continue and finish atm.)

[real.finder]

And pass even an 8 bit 1D lut to Expr? (I have started updating Expr in Avs+ to be able to work like LUT which is faster for some 1D or 2D cases, but had no time to continue and finish atm.)

if it faster (Especially with avx2) or even less ram then why not?

and it will make things easier for testing too

[pinterf]

if it faster (Especially with avx2) or even less ram then why not?

and it will make things easier for testing too
It's not perfect :)

[pinterf]

New version, download masktools2 2.2.17 (https://github.com/pinterf/masktools/releases/tag/2.2.17)

**v2.2.17 (20180710)
- mt_convolution: check plane dimensions to exceed convolution horizontal/vertical size
- lut functions: plane order to RGBA from BGRA like in Expr.
expr parameters y-u-v-a naming matches now to r-g-b-a
- for luts: use_expr=3: always send expression(s) to Expr

[pinterf]

New version, download masktools2 2.2.18 (https://github.com/pinterf/masktools/releases/tag/2.2.18)

Thanks tyaniw for the report

**v2.2.18 (20180905)
- mt_merge: fix right side artifacts for non-mod16 width, AVX2 and luma=false (regression in 2.2.16)
- mt_adddiff: fix 32 bit chroma (still used 0.5 centered chroma instead of 0.0)

[FranceBB]

Thank you for the new version.
Works fine on Windows XP. ^_^

[Tempter57]

pinterf
Hi. Where has disappeared in the last versions 2.2.17 & 2.2.18 mt_convolution? Some scripts, for example, BalanceBorders.avs, which use mt_convolution, produce an error

[pinterf]

What is the error message?
From v2.2.17 the width and height (for each affected plane) is checked agains the size of the supplied parameters.
Lack of this kind of check resulted in sometimes crash, or for "luckier" run it only corrupted pixels and/or memory.
E.g. a clip with width=2 will have an error for a 3x3 convolution.
Note that if chroma is also processed, the subsampled size (e.g. width/2 for a YV12 chroma plane) should fulfill the size requirements as well.

[Tempter57]

"mt-convolution:Plane height should be at least the vertical element count"
With masktools2 v.2.2.16 no error
https://i108.fastpic.ru/thumb/2018/1208/d3/c6800a9624fe3ba632c47b67d43bffd3.jpeg (https://fastpic.ru/view/108/2018/1208/c6800a9624fe3ba632c47b67d43bffd3.jpg.html)
function BalanceBorders(clip c, int cTop, int cBottom, int cLeft, int cRight, int "thresh", int "blur")
{
Assert(isYV12(c), "This is not an YV12 clip. Convert color space to YV12 before using BalanceBorders().")

thresh = Default(thresh, 128)
blur = Default(blur, 999)

Assert(blur > 0, "Blur parameter in BalanceBorders() must be > 0")
Assert(thresh > 0, "Thresh parameter in BalanceBorders() must be > 0")

c
cTop >0 ? BalanceTopBorder(cTop, thresh, blur).TurnRight() : last.TurnRight()
cLeft >0 ? BalanceTopBorder(cLeft, thresh, blur).TurnRight() : last.TurnRight()
cBottom >0 ? BalanceTopBorder(cBottom, thresh, blur).TurnRight() : last.TurnRight()
cRight >0 ? BalanceTopBorder(cRight, thresh, blur).TurnRight() : last.TurnRight()
}

function BalanceTopBorder(clip c, int cTop, int "thresh", int "blur")
{
cWidth = c.width
cHeight = c.height
cTop = min(cTop,cHeight-1)
blurWidth = max(4,floor(cWidth/blur))

c2 = c.PointResize(cWidth*2,cHeight*2)

c2.\
Crop(0,cTop*2,cWidth*2,2)
PointResize(cWidth*2,cTop*2)
BilinearResize(blurWidth*2,cTop*2)
mt_convolution("1 1 1","0 1 0",y=3,u=3,v=3)
BilinearResize(cWidth*2,cTop*2)
referenceBlur = last

original = c2.Crop(0,0,cWidth*2,cTop*2)

original
BilinearResize(blurWidth*2,cTop*2)
mt_convolution("1 1 1","0 1 0",y=3,u=3,v=3)
BilinearResize(cWidth*2,cTop*2)
originalBlur = last

balanced = mt_lutXYZ(original,originalBlur,referenceBlur,"z y - x +",y=3,u=3,v=3)
difference = mt_makeDiff(balanced,original,y=3,u=3,v=3)

tp = string(128+thresh)
tm = string(128-thresh)
difference = difference.mt_lut("x "+tp+" > "+tp+" x ?",y=3,u=3,v=3)
difference = difference.mt_lut("x "+tm+" < "+tm+" x ?",y=3,u=3,v=3)

mt_addDiff(original,difference,y=3,u=3,v=3)

StackVertical(last,c2.Crop(0,cTop*2,cWidth*2,(cHeight-cTop)*2)).PointResize(cWidth,cHeight)
}

[DJATOM]

BalanceBorders is just an old script, I suggest you to try https://github.com/sekrit-twc/EdgeFixer. Write me into telegram, if you need x64 build.

[real.finder]

mod for BalanceBorders https://pastebin.com/sbpK1GGC

it work now and it support new color spaces, and some speed up in non plus old avs

[Tempter57]

real.finder & pinterf
Thank You :)

[StainlessS]

Hi P, is there some way to achieve this:-
All may not be lost.
I think that if we can convert encrypted 8 bit clip into 10 bit (with a shift so that bit 0 of encrypted is shifted to bit 1 of 10 bit clip, and both bits 0 and bit 9 of 10 bit clip now 0), and then encode
the 10 bit clip, we may get a better result. However, I have no idea how to convert to 10 bit by only shifting by 1 bit.
(I also have no idea how to do a straight forward convert to 10 bit, so some assistance required here).
EDIT: Reverse procedure restoring back to 8 bit before decrypt would also be necessary.
https://forum.doom9.org/showthread.php?p=1866138#post1866138

something like [EDIT: Below mixed bittage clips, first(K) 10 bit, 2nd(Last) 8 bit, want 10 bit result (same as 1st clip K)]

AviSource("D:\Parade.avi").GreyScale # 8 bit 490x360 YUV12
K=Last.ConvertBits(10).BlankClip # 10 bit 490x360 YUV420P10
Mt_lutxy(K,Last,Expr="y") # Ideally something like Expr="y 2 *" ie shift 8 bit clip over one bit, result 10 bit clip ......0xxxxxxxx0 where x are bits from 8 bit clip.

#info

Produces this
https://i.postimg.cc/PNbHtfkH/Test.jpg (https://postimg.cc/PNbHtfkH)

I'm totally new to non 8 bit stuff, and a bit thick :)

EDIT: This seems to construct the 10 bit encrypt/decrypt key ok

Function MakeRandomKeyClip(clip c,int Seed,int "Matrix") {
Return RandomSource(Length=c.FrameCount,Width=c.width,Height=c.Height,Seed=Seed)
\ .ConvertToPlanarRGB
\ .ConvertBits(10)
\ .ConvertToYUV420(Matrix=Default(Matrix,"rec601"))
}


EDIT: If not possible, perhaps Wonkey_Donkey has a suggestion :)

EDIT: Perhaps making Stacked 10 bit clip possible, then convert to non stacked.

EDIT: Not sure, but suspect we need a 7 bit shift if 10 bit colorspace occupies 10 most significant bits of 16 bit word.

EDIT: Think that the below does as we require.

Function EncryptPrep8To10(clip c,Bool "Shift") {
# Convert from 8 bit to 10 bit (Shift on luma only if it proves useful, doubtful)
c
Shift=Default(Shift,False)
Limiter() # Dump illegal values
StackVertical(Last,Last.Blankclip.Mt_lut("0")) # Stack16 16 bit
ConvertFromStacked(Last,16) # 16 bit Native
(Shift) ? Mt_Lut(Last,Yexpr="x 1 >>u",y=3,u=2,v=2) : NOP
ConvertBits(10)
}

Function DecryptPrep10To8bit(clip c,Bool "Shift") {
# Convert from 10 bit to 8 bit
c
Shift=Default(Shift,False)
ConvertBits(16) # Maybe this dont work as required
(Shift) ? Mt_Lut(Last,Yexpr="x 1 <<u",y=3,u=2,v=2) : NOP
ConvertToStacked
Crop(0,0,0,Height/2)
Limiter() # Dump illegal values
}

SHIFT=True # Experimental (probably of no use)
AMP=True # Amplifiy Difference

AviSource("D:\Parade.avi")
Limiter() # Dont let Limiter inside functions produce difference between in and out clips.

ORG=Last
EncryptPrep8To10(SHIFT)
DecryptPrep10To8bit(SHIFT)
D = Subtract(Last,ORG)
D = (AMP) ? D.Levels(126-20,1,126+20,16,235,coring=false) : D
StackVertical(StackHorizontal(ORG,D),StackHorizontal(Last,D))

[wonkey_monkey]

EDIT: If not possible, perhaps Wonkey_Donkey has a suggestion :)

I'm leaving >8 bit well alone until the day when you can ask three people a question about it and not get three different answers!

[pinterf]

Use Expr when you want to mix together different bit depths.
This one properly shifts luma by one.
AviSource("D:\Parade.avi").GreyScale # 8 bit 490x360 YUV12
K=Last.ConvertBits(10).BlankClip # 10 bit 490x360 YUV420P10
#Mt_lutxy(K,Last,Expr="y") # Ideally something like Expr="y 2 *" ie shift 8 bit clip over one bit, result 10 bit clip ......0xxxxxxxx0 where x are bits from 8 bit clip.
Expr(K,Last,"y 2 *","","")

ConvertBits for YUV works using bit shifts.
http://avisynth.nl/index.php/ConvertBits

Tha fact that masktools mt_lutxy does not check for similar input bit-depth is a bit annoying now (I was surprised on StainlessS' example giving me a vertical bar :) ). It cannot handle properly inputs with different bit depths, unlike Expr.

[pinterf]

EDIT: Not sure, but suspect we need a 7 bit shift if 10 bit colorspace occupies 10 most significant bits of 16 bit word.

Valid values for 10-12-14 bit color spaces always occupy the lower bits in Avisynth+.
http://avisynth.nl/index.php/D_Notation#A_note_on_notation
In other systems there are other formats e.g. P210, P216 (https://docs.microsoft.com/en-us/windows/desktop/medfound/10-bit-and-16-bit-yuv-video-formats) which are always on the 16 bit magnitude, but the 10 bit version having the six LSB set to 0. These formats are used by Avisynth, e.g. these type of fourCC codes can be negotiated by Avisource, but you, as a script writer will never see it because it will be converted to Avisynth's internal YUV422P10 and YUV422P16 format.

[StainlessS]

Thanks very much Pinterf, guess I gots lots and lots to read up on.

Guess I havta try out Expr, I thought that Masktools now supported mixed bittage, I musta mixed it up with Expr().

The fact that masktools mt_lutxy does not check for similar input bit-depth is a bit annoying
I see it as my purpose in life to annoy the hell out of you :)

All of these weird colorspace/bittage formats seem awfully complicated, I think I'll go to bed.

[Dogway]

I might have dull day, but the next doesn't work for me:

mt_lut(mt_polish("abs(-40)"),u=-128,v=-128)

Looks like an issue with mt_polish and abs()

Same with pow(), the next example renders black.

mt_lut("x 1 pow",u=-128,v=-128)

[wonkey_monkey]

What's the result of

subtitle(mt_polish("abs(-40)"))

?

[Dogway]

The result is "a"

and

subtitle(mt_polish("pow(x,1)"))

doesn't output anything

Ok, it uses the ^ operator not the pow() function. The abs() issue still intrigues me.

[pinterf]

Use ^ instead of pow
http://avisynth.nl/index.php/MaskTools2

[Dogway]

Making a vignette function but something is not right.

strength = string(2.0)
w=string(width(last))
h=string(height(last))
a=mt_lutspa( last, mode = "absolute", expr = " x "+w+" 2 / - abs "+w+" 2 / / 255 * "+strength+" ^ 255 / 0 max", chroma = "-128" )
b=mt_lutspa( last, mode = "absolute", expr = " y "+h+" 2 / - abs "+w+" 2 / / 255 * "+strength+" ^ 255 / 0 max", chroma = "-128" )
c=mt_lutxy(a,b,"x y + 2 / 255 - abs")
c

This might be better, normalized to 0,1
strength = string(0.75) # range 0,1
w=string(width(last))
h=string(height(last))
a=mt_lutspa( last, mode = "absolute", expr = " x "+w+" 2 / - abs "+w+" 2 / / 255 * 1.5 "+strength+" 2 / + ^ 255 / 0 max", chroma = "-128" )
b=mt_lutspa( last, mode = "absolute", expr = " y "+h+" 2 / - abs "+w+" 2 / / 255 * 1.5 "+strength+" 2 / + ^ 255 / 0 max", chroma = "-128" )
c=mt_lutxy(a,b,"x y + 255 - abs")
c

[pinterf]

You can find some mt_lutspa examples here:
http://avisynth.nl/index.php/MaskTools2/mt_lutspa

or use sx, sy, sxr and syr in "Expr" (Avisynth+ only)
http://avisynth.nl/index.php/Expr

[Dogway]

Thanks, great examples there. I forgot Expr, still with the old avisynth mindset.
Could you confirm the mt_polish bug with abs()?

[StainlessS]

I just installed Avisynth for this test,

BlankClip(pixel_type="YV12") # added

strength = string(0.75) # range 0,1
w=string(width(last))
h=string(height(last))
a=mt_lutspa( last, mode = "absolute", expr = " x "+w+" 2 / - abs "+w+" 2 / / 255 * 1.5 "+strength+" 2 / + ^ 255 / 0 max", chroma = "-128" )
b=mt_lutspa( last, mode = "absolute", expr = " y "+h+" 2 / - abs "+w+" 2 / / 255 * 1.5 "+strength+" 2 / + ^ 255 / 0 max", chroma = "-128" )
c=mt_lutxy(a,b,"x y + 255 - abs")
c


Gets me this (no idea if correct, have not dissected expr).
https://i.postimg.cc/SsLBVJMc/Dogway.jpg (https://postimages.org/)

Looks like it might be as intended.

EDIT: During testing, suggest use RT_Stats RT_DebugF() to show result expr (and perhaps infix equivalent too [check if RPN parses OK]).

EDIT: eg

BlankClip(pixel_type="YV12")

strength = string(0.75) # range 0,1
w=string(width(last))
h=string(height(last))

RPN_X = " x "+w+" 2 / - abs "+w+" 2 / / 255 * 1.5 "+strength+" 2 / + ^ 255 / 0 max"
RPN_Y = " y "+h+" 2 / - abs "+w+" 2 / / 255 * 1.5 "+strength+" 2 / + ^ 255 / 0 max"

INFIX_X=Mt_Infix(RPN_X)
INFIX_Y=Mt_Infix(RPN_Y)

myName="DoggyDoo: "

RT_DebugF("DBGVIEWCLEAR") # Clear DebugView window (thanx Wonkey Donkey)

RT_DebugF("RPN_X = '%s'",RPN_X,name=myName)
RT_DebugF("RPN_Y = '%s'",RPN_Y,name=myName)

RT_DebugF("INFIX_X = '%s'",INFIX_X,name=myName)
RT_DebugF("INFIX_Y = '%s'",INFIX_Y,name=myName)

a=mt_lutspa( last, mode = "absolute", expr = RPN_X, chroma = "-128" )
b=mt_lutspa( last, mode = "absolute", expr = RPN_Y, chroma = "-128" )
c=mt_lutxy(a,b,"x y + 255 - abs")
c

EDIT: Is above 'w' in RED correct ?


00000074 0.00000000 [2176] DoggyDoo: RPN_X = ' x 640 2 / - abs 640 2 / / 255 * 1.5 0.750000 2 / + ^ 255 / 0 max'
00000075 0.00006196 [2176] DoggyDoo: RPN_Y = ' y 480 2 / - abs 640 2 / / 255 * 1.5 0.750000 2 / + ^ 255 / 0 max'
00000076 0.00012101 [2176] DoggyDoo: INFIX_X = 'max(((((abs((x-(640/2)))/(640/2))*255)^(1.5+(0.750000/2)))/255),0)'
00000077 0.00017681 [2176] DoggyDoo: INFIX_Y = 'max(((((abs((y-(480/2)))/(640/2))*255)^(1.5+(0.750000/2)))/255),0)'

[Dogway]

Useful to grab plain text result of mt_polish() (or infix)
The mask is in PC levels but I ported the functions to avs+ internal Expr and added an option to merge with source in TV levels.
Still a few things might improve since I don't understand well the scale_inputs setting.

function Vignette(clip c, float "Vignette", bool "show") {

strength = string(default(Vignette, 0.7))
show = Default(show, false)

a=expr( c, " sx width 2 / - abs width 2 / / 255 * 1.5 "+strength+" 2 / + ^ 255 / 0 max", "","", scale_inputs = "allf" )
b=expr( c, " sy height 2 / - abs width 2 / / 255 * 1.5 "+strength+" 2 / + ^ 255 / 0 max", "","", scale_inputs = "allf" )

msk=expr(a,b,"x y + 255 - abs", "128","128", scale_inputs = "allf" )
msktv=msk.expr("x 128 - 224 * 255 / 128 +", scale_inputs = "allf" )


show ? msk : expr(msktv,c,"x 16 - y 16 - * 220 / 1 * x 16 - 1 1 - * + 16 + ", "y", "y", scale_inputs = "all" )

}

[StainlessS]

Dogways see RED edit in prev post.

Also, can eg calc WD2 = W / 2 in script instead of masktools. Guess if using relative mode, can just use 0.5 instead.

[Dogway]

The lens is a fixed circle, so it must keep the lens geometry otherwise we get an oval. When shooting square the corners are not black so the maths can be improved, and specially to deal with bitdepth and scale_inputs parameter.

[StainlessS]

Thanks, that clarifies it (thought I had better mention it in case a typo).

Could you confirm the mt_polish bug with abs()?
Is the shape posted as expected ? (ie abs is working OK).

EDIT: I think real.finder is the only scriptor that understands all that bitdepth and scale_input stuff, and he's way too busy to assist here :devil:

[pinterf]

I might have dull day, but the next doesn't work for me:
mt_lut(mt_polish("abs(-40)"),u=-128,v=-128)
Looks like an issue with mt_polish and abs()

Yep, tokens beginning with 'a' and 'y' were not recognized. (abs, atan, ymin, ymax). Fixed on git, no release yet.

[real.finder]

I think I get bug

MessageClip(mt_infix("x range_half - range_half * 112 scaleb / range_half +"))

will give https://i.imgur.com/XUH9rlf.jpg

and aside from that, the float (32bit) chroma scale in Dither_Luma_Rebuild seems broke since the zero-chroma-center transition, I think most if not all scripts that has chroma lut/expr was also affected and they need review

[pinterf]

I think I get bug

MessageClip(mt_infix("x range_half - range_half * 112 scaleb / range_half +"))

will give https://i.imgur.com/XUH9rlf.jpg

and aside from that, the float (32bit) chroma scale in Dither_Luma_Rebuild seems broke since the zero-chroma-center transition, I think most if not all scripts that has chroma lut/expr was also affected and they need review
I think scaleb(112) is the proper syntax here, scaleb works here as a function.
EDIT: I wonder whether mt_infix handles all the newer stuffs properly.
EDIT: Fixed on git, no release yet.

[real.finder]

I think scaleb(112) is the proper syntax here, scaleb works here as a function.
EDIT: I wonder whether mt_infix handles all the newer stuffs properly.
EDIT: Fixed on git, no release yet.

thanks, I send you pm :)

[real.finder]

another case for mt_infix

x range_half - dup * dup y range_half - dup * + / range_max *

seems mt_infix not handle dup, maybe mt_polish too

[pinterf]

Yes, dup is a newer addition, technically it's like reusing an existing short or compex expression using a temporary variable (not 100% but somethong like this)
I think this syntax conversion is beyond my_infix functionality.
var1 = (x - range_half)
var2 = var1 * var1
var3 = y - range_half
(var3 * var3 + var2) / var1 * range_max
The same applyes for the opposite conversion case, repeating expression patterns should be recognized somehow and replaced with dup?

[real.finder]

is there bitdepth scale in mt_lutf, mt_luts, mt_lutsx and mt_lutspa? they should has at least scale_inputs

[pinterf]

New release: v2.2.20

https://github.com/pinterf/masktools/releases/tag/2.2.20

Minor fixes, lut expression syntax additions just like in Avisynth+ 3.5

Now with both Microsoft and ClangCL (LLVM 9.0) versions inside.

Changelog
**v2.2.20 (20200303)
- new "yscalef" and "yscaleb" keywords similar to "scalef" and "scaleb" but scaling is forced to use rules for Y (non-UV) planes
- mt_lutspa: add parameters "scale_inputs", "clamp_float" and "clamp_float_UV"
- new predefined constants in expressions: yrange_min, yrange_half, yrange_max
Unlike range_min, range_half, range_max the y-prefixed versions do not depend on whether the currently
processed plane is luma(Y) or chroma(U/V). They are always returning the values of Y plane.
- new parameter to lut functions: Boolean "clamp_float_UV": default false, as an addition to clamp_float (since v2.2.20).
- New: Parameter "scale_inputs" can now be set to "floatUV" (chroma pre-shift by 0.5 for 32 bit float pixels)
- Fix: mt_motion mask contained out-of range pixels for 10-14 bit inputs
- Fix: mt_edge convolution mode incorrect result on 10-32 bits when normalizer weight is not power of 2
e.g. mode = "1 2 1 0 0 0 -1 -2 -1 15.0" (normalizer: 10th parameter or maximum of (sum_positive/sum_negative))
Note: when processing chroma (u=3,v=3) on 32 bit float clip will result 0..1.0 ranged masks in chroma planes as well.
- Source:
- add LLVM-clangCl to VC project configuration (built-in clang support in VS2019)
- fix LLVM build for VS2019
- silence many warnings
- project configurations: use current SDK version 10.0.18362.0

**v2.2.19 (20190710 - not released)
- Fix: mt_infix to recognize scaleb and scalef
- Fix: mt_infix to recognize ymin, ymax, abs, atan, etc... tokens beginning with 'a' and 'y' were not converted
- Move project to VS2019 v142 toolset, xp builds still at v141_xp
- update current Avisynth+ headers
- update source to use boost 1.70 lib v142 for non-xp builds

[StainlessS]

OOOooooooh, its just like Christmas, thanx :)

[real.finder]

New release: v2.2.20


:thanks: but why "with_dual_filter_signatures" has no Clang? :)

[pinterf]

The project doesn't have such configuration. Way too much version even at present state. I was even considering dropping those dual config versions.

[FranceBB]

Thank you for the new release! And yes, I'm one of those people using the dll "with_dual_filter_signatures".

Move project to VS2019 v142 toolset, xp builds still at v141_xp

Thank you for still building targeting the v141_xp with C++ Redistributable 2019, I appreciate that. :)

[pinterf]

Thank you for the new release! And yes, I'm one of those people using the dll "with_dual_filter_signatures".



Thank you for still building targeting the v141_xp with C++ Redistributable 2019, I appreciate that. :)
You're welcome.

I wonder why are you using dual signature versions?

[l00t]

n00b question here: what is "with_dual_filter_signatures" and how does it differ from the normal versions?

[real.finder]

n00b question here: what is "with_dual_filter_signatures" and how does it differ from the normal versions?

it's here to avoid this kind of problems here (https://forum.doom9.org/showthread.php?p=1800152#post1800152) then here (https://forum.doom9.org/showthread.php?p=1800467) it will happen in any non avs 2.5 color format like Y8 and YV16 not only avs+ HBD

if you has 2.5 masktools it will be used instead of new one in these cases cuz new masktools use float in many parameters since HBD so with "with_dual_filter_signatures" it has both float and int so it's fix the problem

so maybe instead of remove with_dual_filter_signatures it's better to make it the main one :)

from pinterf in PM when I discussed the problem with him

Aaaaah.
Everyone is right.

1.) Avisynth is choosing the best matching parameter list for the call.
In our tests, we were feeding integers for the parameters. The best match was the old masktools2 function.
Try it with 155.0 instead of 155 and feel the difference.

2.) But there is another problem.
When someone wants to use the functions with a parameter list without the names,
e.g. mt_binarize(clip,80,"0 255",3,1,1), it won't work anymore. Parameters Y, U and V and the others are shifted in the parameter sequence.

For example this is the function signature of the new mt_binarize:
"c[threshold]f[upper]b[mode]s[stacked]b[paramscale]s[Y]f[U]f[V]f[chroma]s[offX]i[offY]i[w]i[h]i[sse2]b[sse3]b[ssse3]b[sse4]b[avx]b[avx2]b"

This needs a bit more serious workaround because tp7 made the function signatures automagically assembled from
- filter specific (as seen above, c[threshold]f[upper]b[mode]s[stacked]b[paramscale]s)
- common (starting with Y, U, V, etc)

If I want to keep the compatibility, I have to "flag" the new extra parameters of all filters, thus tp7's filter definition would work from 3 sources:
- filter specific (old, like c[threshold]f[upper]b[mode])
- common
- filter specific (new, like [stacked]b[paramscale]s)

3.) Plus, because of your original problem (best parameter type match), perhaps I have to double the definitions, one for integer only parameters, and one for the float-type.


after that, the fix should be in changes log
- dual interface int and float parameter list (autoload problem)

edit: here it is https://github.com/pinterf/masktools/commit/d67aecaf985a3db229cc47300d1eef22f0c7ae98

edit2: at least with mvtools2 after update limit to float SMDegrain limit parameter seems work fine even if there no dual interface, I think it because it use it as limit=limit

[FranceBB]

You're welcome.

I wonder why are you using dual signature versions?

Although on my machine I have the very latest build of Avisynth+ on both XP and Win10, I sneaked a series of portable installation at work long before they accepted to let me (and my colleagues) use Avisynth officially. It was on C:\Encoding\ and it was using a series of commands to call Avisynth and encode some 8bit MPEG-2 files. It was years ago, way before StainlessS convinced me to switch to Avisynth+, so I don't remember whether I deployed Avisynth 2.5.8 or 2.6.0 to each and every machine along with my collection of plugins; all my workflows were either 8bit or 16bit stacked/interleaved back then. Today, all the installation I officially make within the company are running Avisynth+ but I kinda lost track of the first "sneaky" ones (but I'm pretty sure they are still used by someone somewhere), so I keep my plugins as compatible as possible so that if people copy the plugin folder from another official machine running AVS+ (and I know they do it every once in a while) they don't get weird errors and they don't break workflows, especially 'cause most of them are automatized. It's funny how complicated it is for some company to simply accept that open source software is better and more reliable than crappy and expensive professional one... One day I'll tell you the story of how I "brought" Avisynth into one of the leading TV company and how hard it was for them to accept it...

[real.finder]

Yes, dup is a newer addition, technically it's like reusing an existing short or compex expression using a temporary variable (not 100% but somethong like this)
I think this syntax conversion is beyond my_infix functionality.
var1 = (x - range_half)
var2 = var1 * var1
var3 = y - range_half
(var3 * var3 + var2) / var1 * range_max
The same applyes for the opposite conversion case, repeating expression patterns should be recognized somehow and replaced with dup?

what is the best one? since this problem didn't fix yet :)

also seems the scaling parameter didn't work well in this case with chroma float

ColorBars(width=640, height=480, pixel_type="yv12")
convertbits(32)
mt_binarize(threshold=128, U=3, V=3)

also mt_binarize is like mt_edge that should give 0..1.0 ranged for chroma

edit: seems mt_binarize need 0..1.0 ranged for chroma
ColorBars(width=640, height=480, pixel_type="yv12")
convertbits(32)
mt_lut("x 0.5 +",y=2, U=3, V=3)
mt_binarize(threshold=128, U=3, V=3)
mt_lut("x 0.5 -", y=2,U=3, V=3)
now it's work

so I think it's not a bug since it's should be used with masked clip

[DanilaZabiaka]

Listen, I dont understand your development

WTF is masktools2? It's useful for 60FPS

Can it be processed in real time? Yes, using SVP

Can SVP be free? Yeah, I ported to Uplavnyalka this https://forum.doom9.org/showthread.php?t=180656

What is SVP? Its mvtools+render plugin

And this plugin can be friendly with mvtools as I know

May be you can friend this with svpflow2 and give people simple presets?

Its how ported svp (Uplavnyalka) 'gui' looks like
http://i.prntscr.com/wSiy7yy9TUuEzx0F9A35Vw.png

PotPlayer developer integrated avisynth from ffdshow to this project, why people cant do this too? Or rewrite analogue of render plugin?

[sl1pkn07]

Hi

any plans for make the filter linux avs+ compatible?

greetings

[pinterf]

Hi

any plans for make the filter linux avs+ compatible?

greetings
Yes, and all the others (mvtools, rgtools, etc) as well. I'll start with a simple one for experimenting. In general it takes two phases, first to make the source gcc (and clang) friendly, then the port itself.

[pinterf]

Listen, I dont understand your development

WTF is masktools2? It's useful for 60FPS


Thank you for your kind interest in masktools2.

[DanilaZabiaka]

imho coders code not for users, but just for code. no simple solutions, but complex, NEW, etc. fuh

[pinterf]

masktools functionality and optimization level is everything but not simple. Along with mvtools they are of the most complex filters I met. Feel free to rewrite the whole filter group to a more simple and hyper user friendly one. Spend months of your free time with it. Or better: plan and build your own CPU to reach your magic 60 FPS. But wait, no, hardware engineers work not for users and create too complex things.

[DJATOM]

Don't mind him, apparently Daniil is just unmanner guy who makes ridiculous insults towards forum users :(

[Music Fan]

Hi,
I just re-installed Avisynth+ 32 bit (3.5.1), I selected the masktools2 dll (2.2.20) from the x86 folder, put it in the C:\Program Files (x86)\AviSynth+\plugins+ folder but when I open my script, I get the error message "no function named ylevels".:o
I also tried in the plugins folder, same result.

I had installed this Visual C++ and Visual Studio package ;
https://github.com/abbodi1406/vcredist/releases
I was told (in the AVS+ topic) that it contained everything useful for Avisynth when we don't install the vcredist version, what am I doing wrong ?


edit : that's ok now, I forgot the YLevels_mt avsi :rolleyes:

[real.finder]

I was adding HBD to http://avisynth.nl/index.php/Hysteria

Convertbits(16)
noisymask = mt_edge(thy1=6,thy2=6,mode="cartoon")
cleanmask = mt_edge(thy1=20,thy2=20,mode="cartoon")
mt_hysteresis(cleanmask,noisymask).mt_inflate().Blur(1.0).Blur(1.0).mt_deflate()

these not give same output as 8bit, with 32 it's worst, bug in mt_hysteresis?

[pinterf]

If you say, probably yes :) Should be the same, isn't it?

[real.finder]

If you say, probably yes :) Should be the same, isn't it?

yes, it should, maybe paramscale in mt_hysteresis not work correctly?

[pinterf]

mt_hysteresis fixed.

Download masktools v2.2.21
https://github.com/pinterf/masktools/releases/tag/2.2.21

[real.finder]

mt_hysteresis fixed.

Download masktools v2.2.21
https://github.com/pinterf/masktools/releases/tag/2.2.21

yes, it work now :thanks:

[StainlessS]

Oops, originally posted in Mv Tools thread.

Gavino reported an infix ternary conditional Mt_Polish bug back in 2014, nobody ever seems to have taken a blind bit of notice of it.

2014: https://forum.doom9.org/showthread.php?p=1666933#post1666933
2017: https://forum.doom9.org/showthread.php?p=1814483#post1814483

mt_polish parses nested conditionals incorrectly

For example,
mt_polish("x > 128 ? 255 : x > 64 ? 128 : 0")
produces
x 128 > 255 x 64 > ? 128 0 ?
instead of
x 128 > 255 x 64 > 128 0 ? ?

It can be worked round by putting the inner conditional in parentheses
mt_polish("x > 128 ? 255 : (x > 64 ? 128 : 0)")
but this should not be necessary.

Test script:
BlankClip(pixel_type="YV12")
Subtitle(mt_polish("x > 128 ? 255 : x > 64 ? 128 : 0"))
Subtitle(mt_polish("x > 128 ? 255 : (x > 64 ? 128 : 0)"), y=30)





BlankClip(pixel_type="YV12")

aInfix = "x > 128 ? 255 : x > 64 ? 128 : 0"
aRpn = mt_polish(aInfix)
aInfix2 = mt_Infix(aRpn)

Subtitle("aInfix = "+ aInfix, y=10)
Subtitle("aRpn = "+ aRpn, y=30)
Subtitle("aInfix2 = "+ aInfix2, y=50)
astripped="((x>128) ? 255 : (x>64)) ? 128 : 0"
Subtitle("aInfix2 = "+ astripped + " ##### STRIPPED #####", y=70)

bInfix = "x > 128 ? 255 : (x > 64 ? 128 : 0)"
bRpn = mt_polish(bInfix)
bInfix2 = mt_Infix(bRpn)

Subtitle("bInfix = "+ bInfix, y=100)
Subtitle("bRpn = "+ bRpn, y=120)
Subtitle("bInfix2 = "+ bInfix2, y=140)
bstripped="(x>128) ? 255 : ((x>64) ? 128 : 0)"
Subtitle("bInfix2 = "+ bstripped + " ##### STRIPPED #####", y=160)

return last

https://i.postimg.cc/SRw9d5rx/Ternary.jpg (https://postimages.org/)

I tend to use the 'b' infix version with extra parenthesis, so that it works as Avs script [as per Gavino suggestion fix].

[pinterf]

"A small step for a ternary operator but a giant leap for mt_polish" edition :)
Download masktools2 2.2.23
https://github.com/pinterf/masktools/releases/tag/2.2.23
**v2.2.23 (20200513)
- fix mt_polish for nested ternary operators

**v2.2.22 (20200422)

Add AVX2 to mt_expand and mt_inpand 10-16 bit processing
Add Avisynth+ interface V8 frame property copy support (test)

**v2.2.21 (20200410)

fix mt_hysteresis for 10-32 bits

[StainlessS]

A giant leap indeed, this one aint sitting in my in-box at all, thanks very much, great stuff :)

EDIT:
Lovely [one great big pain in the bum removed].
https://i.postimg.cc/05TsSwKC/Ternary.jpg (https://postimages.org/)

EDIT: @ Big G, finally got it fixed, YippEEEEE!!!

[FranceBB]

Another step forward. Thank you, as always! :)

[`Orum]

I moved my encoding/filtering machine to Linux, and I've been trying to get this filter to run under wine. But, so far, every attempt has ended in failure. Has anyone managed to do so, and if so, how did you manage to get it to work? This is the only filter that I use frequently (as it's used in so many other filters) that doesn't seem to want to work in that environment.

I saw your other comment regarding mvtools, rgtools, etc., which already both seem to run just fine within wine, at least for me. Is this the last filter remaining?

Alternatively, if there's anything I can do to help get it to run under Linux, let me know. I'm no expert at getting things to work under wine (and struggling to get my own filters to play nice with it), but even if it's only testing, I'm more than willing to assist.

[FranceBB]

I moved my encoding/filtering machine to Linux, and I've been trying to get this filter to run under wine. But, so far, every attempt has ended in failure. Has anyone managed to do so, and if so, how did you manage to get it to work? This is the only filter that I use frequently (as it's used in so many other filters) that doesn't seem to want to work in that environment.

I saw your other comment regarding mvtools, rgtools, etc., which already both seem to run just fine within wine, at least for me. Is this the last filter remaining?

Alternatively, if there's anything I can do to help get it to run under Linux, let me know. I'm no expert at getting things to work under wine (and struggling to get my own filters to play nice with it), but even if it's only testing, I'm more than willing to assist.

I have Linux too (Fedora) but instead of Wine I use a Windows VM when I'm too lazy to reboot into Windows.
What I can say, though, is that Wine is expected to have the same compatibility as Windows XP and MaskTools2 Pinterf Mod work absolutely fine on XP so they should work in Wine.
Are you using Avisynth x86 under Wine? Did you try with the XP build of MaskTools2?

[`Orum]

I use a Windows VM when I'm too lazy to reboot into Windows.
Same; that's how I was able to narrow it down to a problem with MT2 specifically.
Wine is expected to have the same compatibility as Windows XP and MaskTools2 Pinterf Mod work absolutely fine on XP so they should work in Wine.
Wine doesn't have 100% perfect compatibility with any version of Windows to my knowledge.
Are you using Avisynth x86 under Wine? Did you try with the XP build of MaskTools2?
I'm using the 64-bit version of AviSynth+, as some of my filters are 64-bit only. I have not yet tested the XP builds, but if that's more compatible I'll definitely give it a shot. Do you know if Wine needs to be configured for XP compatibility as well, or can it remain configured for later versions of Windows?

Edit: Both the XP and Clang builds seem to work well under Wine. The Clang build is about 2% faster than the XP build when tested with lsfmod. Thanks!

[pinterf]

I love when things get resolved without me :)
Wine is not a perfect emulator, once I had to do something (some input local-aware stuff?) differently in Avisynth core because it misbehaved under wine.

[DJATOM]

I still can remember times when I had to split script to 4 parts and render them into y4m on Xeon X3330, as that was my working PC and I didn't allowed to install Windows on it. So I've used Wine + avs 2.5 ST (2.6 ST later) and filtered animes in lossless files (no one ever caught me). Later I've bought own 4 cores PC, but Avs+ evolved to being MT friendly at those times.

[real.finder]

I think I find bug

ColorBars()
ConvertToY8
mt_Convolution("1","50 99 50",U=3,V=3)

it crash

[pinterf]

I think I find bug

ColorBars()
ConvertToY8
mt_Convolution("1","50 99 50",U=3,V=3)

it crash
Check if this (https://drive.google.com/file/d/1PCR118_Vx_Au-IGIAFtKZFd8SkcSrTLb/view?usp=sharing) works (Stupid request, I know that it works, but I still wait a bit longer on your feedback that it functions normally with other filters; you use filters more frequently than I do)

[real.finder]

Check if this (https://drive.google.com/file/d/1PCR118_Vx_Au-IGIAFtKZFd8SkcSrTLb/view?usp=sharing) works (Stupid request, I know that it works, but I still wait a bit longer on your feedback that it functions normally with other filters; you use filters more frequently than I do)

it seems work fine, thanks

[real.finder]

since you didn't release it yet, is this (https://forum.doom9.org/showpost.php?p=1879768&postcount=328) forgotten? also swap

maybe adding support for dupN and swapN like expr will be a good thing since I plan to take some expr/lut improvements from vs ported scripts

[pinterf]

since you didn't release it yet, is this (https://forum.doom9.org/showpost.php?p=1879768&postcount=328) forgotten? also swap
ported scripts
What do you mean? Support them in infix/polish helper functions?


maybe adding support for dupN and swapN like expr will be a good thing since I plan to take some expr/lut improvements from vs ported scripts
Expr already has them. I'd rather continue implementing real lut functions in Avisynth.

[pinterf]

since you didn't release it yet
v24 released (https://github.com/pinterf/masktools/releases)
**v2.2.24 (20200619)
- fix: mt_convolution divbyzero when greyscale + forced U=3,V=3 (process chroma)

[real.finder]

What do you mean? Support them in infix/polish helper functions?

yes


Expr already has them. I'd rather continue implementing real lut functions in Avisynth.

I know Expr has them, but I mean for MaskTools2

[FranceBB]

v24 released (https://github.com/pinterf/masktools/releases)
**v2.2.24 (20200619)
- fix: mt_convolution divbyzero when greyscale + forced U=3,V=3 (process chroma)

It's always good when you wake up, you have your breakfast, you get to work and you find out that Ferenc just published a new version of Masktools :D
Thank you.

[real.finder]

I find another bug

colorbars()
converttoyv12
convertbits(32)
mt_edge(thY1=0,thY2=255,mode="min/max",U=1,V=1)
convertbits(8)

it crash

[GMJCZP]

When I use the x86 Clang version I get the following error message un VDub2:

"Error reading source frame 0: Avisynth read error: CAVIStreamSynth: System exception - Illegal Instruction at 0x0560DAD0"

When I use the x86 only version everything works fine, I am using the neo_f3kdb Clang version without problems

[pinterf]

Sse4.1? Maybe my clang builds need it.

[GMJCZP]

Sse4.1? Maybe my clang builds need it.

I have e5800 CPU, no sse4 instructions capable.

[pinterf]

I see. Then you should use msvc builds. And next time I have to document this prequisite.

[GMJCZP]

Out of curiosity, why does the Clang version of neo_f3kdb work for me, for example?

[pinterf]

Out of curiosity, why does the Clang version of neo_f3kdb work for me, for example?
Because masktools as a whole is sse4.1 compiled when clang (or gcc) is used. The source is full of common sse2/sse4.1 parts which work under msvc for a simple SSE2 build but for clang such source parts must be separated one by one into a sse2 and a sse4.1 version (and apply on them different function attributes sse2 or sse4.1).

This procedure is painful, takes way too much time, yields nothing in 2020-21, and increases source complexity. Plus I though that SSE4.1 is so rare nowadays (even non-AVX2 processors are rare in my opinion - we are talking about video processing).

There are positive examples as well: because of gcc (linux build in mind) I had to do this separation in Avisynth core. sse2, ssse3, sse4.1. I've done this separation for my other easier to touch filters as well.
If I remember well, nor mvtools2 can be compiled with plain sse2 option when clang is used - it is a similar monster like masktools2. Brrr.

[GMJCZP]

I am not an expert in compilations but I am aware that you have had many difficulties in optimizing masktools and mvtools, they are monsters with many heads and I personally appreciate your efforts.

[pinterf]

Compiling with clang/gcc is not any more difficult as with msvc but they are a bit more picky in this 'processor architecture' aspect. Unfortunately the code which was accepted by msvc (and historically old code meant to be Visual c++ only) must be reorganized for these cases which is usually done when a specific code part really have to be touched for some other reasons.

[real.finder]

try https://www.solidfiles.com/v/ZZvV2ezjvvnQn

it has new yfmin, yfmax, cfmin and cfmax keywords

masktools with same new keywords https://www.solidfiles.com/v/vNBW52KnQrwxn

x64 only and for test only

[MysteryX]

It would be great to see filters like this written for both Avisynth and VapourSynth like I did here (https://forum.doom9.org/showthread.php?t=182961).

It's really not that difficult to detach most of the complex code-base from both environments, and then just write function registrations and few platform-specific tasks for each.

Plus taking OUT any of the complexity of Avisynth (like the way it deals with plane indexes) out of the complex code-base makes it simpler.

I'm trying to port some code to VapourSynth; and it seems to be lacking in terms of masks management.

[MysteryX]

I'll update that most of the MaskTools2 functions have equivalents in VapourSynth, and I listed them here (https://forum.doom9.org/showthread.php?t=182989).

Remains the fact that libraries could easily be written to work on both platforms with just a bit of code abstraction.

[Dogway]

I don't know whether this is a bug or not or if this was always the case (I'm trying to port an old script), but for shapes in masktools2 when zero=false it automatically changes to a plus kernel instead of the original kernel and a hole in the center (the 0,0 pixel), is this by design?

[pinterf]

I don't know whether this is a bug or not or if this was always the case (I'm trying to port an old script), but for shapes in masktools2 when zero=false it automatically changes to a plus kernel instead of the original kernel and a hole in the center (the 0,0 pixel), is this by design?
Just looking into the source code I found no reason for this behaviour, I'd need a demo script that proves that.

[kedautinh12]

Hi @pinterf, long time to see you. Can you explain here about arrays for 2 people here :D
https://forum.doom9.org/showthread.php?p=1949689#post1949689

[pinterf]

Hi @pinterf, long time to see you. Can you explain here about arrays for 2 people here :D
https://forum.doom9.org/showthread.php?p=1949689#post1949689
Is this masktools topic or a messageboard?

[Reel.Deel]

Hi pinterf,

I think I found 2 bugs in MaskTools:

1) It seems that mt_merge always processes the chroma channels when luma=true. I don't know if this is the intended behavior but I would think the chroma/u/v parameters would have priority. You can see what I'm talking about by setting luma to true/false in the test script.

2) Access violation when chroma is processed and mask does not have enough planes; with luma=true it does not happen.


base = Blankclip(color=$FFFFFF, pixel_type="RGBP8")
base2 = Blankclip(color_yuv=$FFFFFF, pixel_type="YUV444P8")
over = Colorbars(pixel_type="RGBP8")
over2 = Colorbars(pixel_type="YUV444P8")
mask = Blankclip(color=$000000, pixel_type="RGBP8").Subtitle("AviSynth", size=150, align=5)
masky = mask.ShowRed("Y8")
mask2 = YToUV(masky,masky,masky)

#RGB
#mt_merge(base, over, mask, chroma="copy", luma=true)

#YUV
mt_merge(base2, over2, mask2, chroma="copy", luma=true)

# Avisynth read error: CAVIStreamSynth: System exception - Access Violation at 0x00000000251D8C90
#mt_merge(base2, over2, masky, chroma="process", luma=false)


--------

Is this masktools topic or a messageboard?

Did you not hear the news?.... this is kedautinh's message board now :D

[Dogway]

Just looking into the source code I found no reason for this behaviour, I'd need a demo script that proves that.

Here's a test script. You can change mt_square() with the other shapes as well.

BlankClip(width=640, height=480, pixel_type="YV12")
manyPlus_Pattern("dirac")
mt_luts( last, mode = "max", pixels = mt_square( 3, zero=false ), expr ="y" )

[Dogway]

Maybe someone can help me because I'm unable to properly read the mt_luts() source (https://github.com/pinterf/masktools/blob/16bit/masktools/filters/lut/functions.h). I'm having issues when 2 different clips are supplied, in the case of modes "max", "min" it looks as if the second clip is supplied twice "mt_luts(b,b,.....)", so no difference with "mt_luts(a,b,.....)". EDIT: looks like a known bug (https://forum.doom9.org/showthread.php?p=1637985#post1637985).

In the case of median, I'm unable to reproduce the behaviour.

Taking as an example the following call (square without the center pixel)
MT_Luts(C, C.removegrain(12), mode="med", pixels = "-1 -1-1 1 1 -1 1 1 0 1 1 0 -1 0 0 -1" , expr = "X Y - abs", u=1,v=1)
And this is my failed attempt (tried many combinations though, spent the last 6h on this):
Expr(C, C.removegrain(12),"
y[-1,-1] A^ y[-1,0] B^ y[-1,1] C^ y[0,-1] D^ y[0,1] E^ y[1,-1] F^ y[1,0] G^ y[1,1] H^ y[0,0] Y^ x[0,0] X^

X A - abs A^
X B - abs B^
X C - abs C^
X D - abs D^
X E - abs E^
X F - abs F^
X G - abs G^
X H - abs H^
X Y - abs Y^
X X - abs X^

A B dup1 dup1 min swap2 max
C D dup1 dup1 min swap2 max
E F dup1 dup1 min swap2 max
G H dup1 dup1 min swap2 max
swap5 swap1 swap6 dup1 dup1 min swap2 max
swap5 swap1 swap2 dup1 dup1 min swap2 max
swap6 swap1 swap7 dup1 dup1 min swap2 max
swap3 Y dup1 dup1 min swap2 max
swap7 swap1 swap6 min
swap3 swap1 swap4 dup1 dup1 min swap2 max
swap6 swap1 swap7 dup1 dup1 min swap2 max
swap4 swap1 swap2 max
swap3 swap1 swap5 min
swap3 swap1 swap5 max
swap3 max
swap3 dup1 dup1 min swap2 max
swap3 swap1 swap2 dup1 dup1 min swap2 max
swap1 swap3 min
swap2 max
min","")

[wonkey_monkey]

pixels = "-1 -1-1

Is that correct?

mt_luts is hard to find documentation for...

[Dogway]

Oh my, can't believe it! It was wrong on the source (https://forum.doom9.org/showthread.php?p=1380495#post1380495)already. There are certain strange things on "*.mp4 guy" scripts, not used vars, redundancy, commented code... I'm mostly copying things as they are. Already ported ReCon, blah, MedSharp, and SSSharp.


EDIT: By the way, aside from source code, mt_luts() is also explained at tp7's hideout (https://tp7.github.io/articles/masktools/) for dummies like me.

[pinterf]

Here's a test script. You can change mt_square() with the other shapes as well.

BlankClip(width=640, height=480, pixel_type="YV12")
manyPlus_Pattern("dirac")
mt_luts( last, mode = "max", pixels = mt_square( 3, zero=false ), expr ="y" )
Thanks, you are right. zero=false is erroneously including only those (x,y) coordinates where x or y is zero. Fix is on the way.
That means for zero=false the mt_someshapexx functions were returning the coordinate pairs of a centerless cross - points where x=0 or y=0 but not (0,0) - instead of the correct shape with a hole in the center.

[real.finder]

EDIT: By the way, aside from source code, mt_luts() is also explained at tp7's hideout (https://tp7.github.io/articles/masktools/) for dummies like me.

I don't think you (nor most peoples here) dummy, they said "No one is born a scientist" and it's all about personal interest, language barrier, and other circumstances

it's not hideout, http://avisynth.nl/index.php/MaskTools2#Exernal_Links in fact it was in his website IIRC but then it was down so he move it to github

[pinterf]

Could you please test this build:
https://github.com/pinterf/masktools/releases/tag/2.2.27
Contains some other changes as well:
v2.2.27 (20210909)

- fix zero=false case for shape helper function (mt_rectangle, mt_circle, mt_diamond etc...)
- lut expressions: report obvious script error (unbalanced stack, invalid keyword or variable, etc)
- mt_lut: reuse LUTs across planes if they are the same like in e.g. mt_lutxyz.
- 1D LUT expressions: occupy only the necessary size for 10-14 bit LUT tables (was: buffer was always reserved for 16 bit data)
- mt_merge: error is luma=false, mask is greyscale but clip is not greyscale
- mt_merge new parameter hint for chroma placement when luma=true and 4:2:0: "topleft". "topleft" is a new option for 4:2:0 videos only
- Source syntax update for GCC (20201229)
- CMake build environment, builds on Linux

[Dogway]

Didn't notice the post. Thanks, just tested and works fine although I use my own tools anyway.

[tormento]

Could you please test this build
After updating to latest version, I am having a problem with the following script

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\1_48 Fantozzi\fantozzi.dgi",ct=24,cb=24,cl=0,cr=0)
ConvertBits(16)
Spresso(10,30).STpresso(10,30,4,22,8,49,1,mc=true) #last updates
fmtc_bitdepth (bits=8,dmode=8)
Prefetch(6)

with error

mt_lutxy : Error at position 3: cannot convert to number: 1.000000
(D:/Programmi/Media/AviSynth+/plugins64/STPresso-2020_0507.avsi, line 40)

with STPresso being

# STPresso by Didée, MC by SilaSurfer, r.f mod in 2020.05.07 for HBD
function STPresso(clip clp, float "limit", float "bias", int "RGmode", int "tthr", float "tlimit", float "tbias", float "back", bool "mc", bool "HD", int "radius")
{
sisavs26 = !(VersionNumber() < 2.60)
limit = default( limit, 3 ) # spatial limit: the spatial part won't change a pixel more than this
bias = default( bias, 24 ) # bias: the percentage of the spatial filter that will apply
RGmode = default( RGmode, 4 ) # the spatial filter is RemoveGrain, this is its mode
tthr = default( tthr, 12 ) # temporal threshold for fluxsmooth. Can be set "a good bit bigger" than usually
tlimit = default( tlimit, 3 ) # the temporal filter won't change a pixel more than this
tbias = default( tbias, 49 ) # the percentage of the temporal filter that will apply
back = default( back, 1 ) # after all changes have been calculated, reduce all pixel changes by this value. (Shift "back" towards original value)
mc = default( mc, false ) #
h = clp.height()
ishd = h > 720
HD = default( HD, ishd ) #
radius = default( radius, round(h/540.0) ) #

LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)
BK = string(back)
TLIM1 = (tlimit>0) ? string( round(tlimit*100.0/tbias-1.0) ) : string( round(100.0/tbias) )
TLIM2 = (tlimit<0) ? "1" : string(tlimit)
TBIA = string(tbias)

expr = (limit<0) ? sisavs26 ? "x y - abs "+LIM1+" scalef < x x 1 x y - x y - abs / * - ?" : "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : sisavs26 ? "x y - abs 1 scalef < x x "+LIM1+" scalef + y < x "+LIM2+" scalef + x "+LIM1+" scalef - y > x "+LIM2+" scalef - " \
+ "x 100 scalef "+BIA+" scalef - * y "+BIA+" scalef * + 100 scalef / ? ? ?" \
: "x y - abs 1 < x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"
texpr = (limit<0) ? sisavs26 ? "x y - abs "+TLIM1+" scalef < x x 1 x y - x y - abs / * - ?" : "x y - abs "+TLIM1+" < x x 1 x y - x y - abs / * - ?"
\ : sisavs26 ? "x y - abs 1 scalef < x x "+TLIM1+" scalef + y < x "+TLIM2+" scalef + x "+TLIM1+" scalef - y > x "+TLIM2+" scalef - " \
+ "x 100 scalef "+TBIA+" scalef - * y "+TBIA+" scalef * + 100 scalef / ? ? ?" \
: "x y - abs 1 < x x "+TLIM1+" + y < x "+TLIM2+" + x "+TLIM1+" - y > x "+TLIM2+" - " \
+ "x 100 "+TBIA+" - * y "+TBIA+" * + 100 / ? ? ?"

bzz = HD ? clp.minblur(radius) : clp.removegrain(RGmode)
sisavs26 ? mt_lutxy( clp, bzz, expr, scale_inputs=limit<0 ? "floatf" : "none", use_expr=1, U=3,V=3) : mt_lutxy( clp, bzz, expr, U=3,V=3)
tthr==0 ? last : sisavs26 ? mt_lutxy(last,last.mt_makediff(mt_makediff(bzz,mc ? bzz.STPresso_FluxsmoothTMC(tthr) : bzz.fluxsmootht(tthr),U=3,V=3),U=3,V=3),texpr,scale_inputs=limit<0 ? "floatf" : "none",use_expr=1,U=3,V=3) : mt_lutxy(last,last.mt_makediff(mt_makediff(bzz,mc ? bzz.STPresso_FluxsmoothTMC(tthr) : bzz.fluxsmootht(tthr),U=3,V=3),U=3,V=3),texpr,U=3,V=3)
back==0 ? last : sisavs26 ? mt_lutxy(last,clp,"x "+BK+" scalef + y < x "+BK+" scalef + x "+BK+" scalef - y > x "+BK+" scalef - y ? ?",U=2,V=2) : mt_lutxy(last,clp,"x "+BK+" + y < x "+BK+" + x "+BK+" - y > x "+BK+" - y ? ?",U=2,V=2)
}


Function STPresso_FluxsmoothTMC(clip clp, int "tthr")
{

tthr = default( tthr, 12 )

super = clp.MSuper(pel=2, sharp=1)

bv1 = manalyse(super,isb=true, truemotion=false, delta=1,blksize=16,overlap=8)
fv1 = manalyse(super,Isb=false, truemotion=false, delta=1,blksize=16,overlap=8)

bc1 = mcompensate(clp, super,bv1)
fc1 = mcompensate(clp, super,fv1)

result=interleave(fc1, clp, bc1).FluxSmoothT(tthr).selectevery(3,1)

Return(result)
}

[real.finder]

tormento, it's work fine for me, maybe it's dot vs comma thing again?

[tormento]

tormento, it's work fine for me, maybe it's dot vs comma thing again?
I don't know, rolling back to MaskTools-2.2.26 solved the issue.

There is some mistype or regression somewhere.

EDIT: Jumped back to .27 and now is working fine. Hell, don't know what happened. Sorry guys.

[real.finder]

I don't know, rolling back to MaskTools-2.2.26 solved the issue.

There is some mistype or regression somewhere.

EDIT: Jumped back to .27 and now is working fine. Hell, don't know what happened. Sorry guys.

you may want to try .28 https://github.com/pinterf/masktools/releases

[Blankmedia]

you may want to try .28 https://github.com/pinterf/masktools/releases

I have the same problem than Tormento.

With .28 I get errors with a couple of script.

Example in HaloBuster:

mt_lut : Error at position 15 : cannot convert to number : 0.24

([...]/HaloBuster.avsi, line 19)

line 19 :

mask = mt_lut (mask, "x range_max / 0.24 - 3.2 * 0.0 max 1.0 min range_max *", use_expr=2)

[pinterf]

I have the same problem than Tormento.

With .28 I get errors with a couple of script.

Example in HaloBuster:

mt_lut : Error at position 15 : cannot convert to number : 0.24

([...]/HaloBuster.avsi, line 19)

line 19 :

mask = mt_lut (mask, "x range_max / 0.24 - 3.2 * 0.0 max 1.0 min range_max *", use_expr=2)
What is your input local, are you using "," as decimal separator?
Anyway, masktools always sets decimal separator to "." before parsing the number. I wonder if it got changed somehow.

EDIT: I have ',' in my input local as well but could not reproduce, anyway I have changed that part.
Check this, I hope it works:
https://github.com/pinterf/masktools/releases/tag/2.2.29

[Blankmedia]

What is your input local, are you using "," as decimal separator?
Anyway, masktools always sets decimal separator to "." before parsing the number. I wonder if it got changed somehow.

EDIT: I have ',' in my input local as well but could not reproduce, anyway I have changed that part.
Check this, I hope it works:
https://github.com/pinterf/masktools/releases/tag/2.2.29

I've replace with 2.2.29 and so far both 64 builds work well.

You rock

[Blankmedia]

What is your input local, are you using "," as decimal separator?

I forgot to answer that. Yes I have "," as decimal separator.

[Tempter57]

Check this, I hope it works:
https://github.com/pinterf/masktools/releases/tag/2.2.29
I used script MedSharp2.avsi
Function MedSharp2(Clip C, int "thresh", int "rad", int "mode", bool "lp", bool "hp", bool "nr", float "str", int "kernel")
{

_Thr = Default(thresh, 256)
_Rad = Default(rad, 1)
Mode = Default(mode, 1)
str = Default(str, 1)
THR = string(_Thr)
RAD = string(_Rad)
RAD2 = string(_Rad)

lp = Default(lp, false)
hp = Default(hp, false)
nr = Default(nr, false)
kernel = Default(kernel, 2)


inputclipproperties = c
blank = BlankClip(length=inputclipproperties.framecount, width=inputclipproperties.width, height=inputclipproperties.height, pixel_type="rgb32", fps=inputclipproperties.framerate, fps_denominator=1, audio_rate=inputclipproperties.Audiorate, stereo=true, sixteen_bit=true, color=$828282).converttoyv12()


Lowpass1 = C.NLLV(rad=1).NLLH(rad=1)
Lowpass_Return = lp == true ? _Rad >= 2 ? Lowpass1 : C : C

Lowpass_NR_1 = Lowpass_Return.TMed2(rad=_Rad, thresh=_Thr, Kernel=1)
Lowpass_NR_2 = Lowpass_Return.TMed2(rad=_Rad, thresh=_Thr/4, Kernel=2)

Diff1 = MT_Luts(C, Lowpass_NR_1, mode="med", pixels = "0 "+RAD+" "+RAD+" 0 -"+RAD+" 0 0 -"+RAD+" 0 0" ,\
expr = "X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +", u=1,v=1)
Diff2 = MT_Luts(C, Lowpass_NR_2, mode="med", pixels = "-"+RAD2+" -"+RAD2+"-"+RAD2+" "+RAD2+" "+RAD2+" -"+RAD2+" "+RAD2+" "+RAD2+" 0 "+RAD+" "+RAD+" 0 -"+RAD+" 0 0 -"+RAD ,\
expr = "X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 2 * 1 >= "+THR+" 2 * 0.5 ^ "+THR+" 2 * ? + / - 128 +", u=1,v=1)

mask1 = Mt_Lut(Diff1, " X 128 - 255 * ", u=1, v=1)#.NLLV().NLLH()
#mask2 = Mt_Lut(Diff2, " X 128 - 255 * ", u=1, v=1)#.NLLV().NLLH()

s1 = mt_merge(Diff2, blank, mask1).mt_Lut("X 128 - "+string(str)+" 2 / * 128 +")
s1b = s1.NLLV().NLLH()
s1s = mt_makediff(s1, s1b)
s2 = Diff1.mt_Lut("X 128 - "+string(str)+" * 128 +")
s2b = s1.NLLV().NLLH()
s2s = mt_makediff(s2, s2b)


out = MT_adddiff(C, s1s, u=2, v=2)
out = MT_adddiff(out, s2s, u=2, v=2)
out

Return(Last)
}

# Y'et A'nother H'alo R'educing script
function YAHR(clip clp)
{
b1 = clp.minblur(2).removegrain(11,-1)
b1D = mt_makediff(clp,b1)
w1 = clp.aWarpSharp2(depth=32,blur=2,thresh=128,chroma=3)
w1b1 = w1.minblur(2,1).removegrain(11,-1)
w1b1D = mt_makediff(w1,w1b1)
DD = b1D.repair(w1b1D,13)
DD2 = mt_makediff(b1D,DD)
clp.mt_makediff(DD2,U=2,V=2)
}

Function blurH(clip c, int "rad", Float "CW")
{

Rad = Default(rad, 1)
CW = Default(CW, 0.5)

Center = C
Left = C.PointResize(C.width, C.height, -rad, 0, C.width, C.height)
Right = C.PointResize(C.width, C.height, rad, 0, C.width, C.height)

Average(Center, CW/2, Left, (1-CW)/2, Right, (1-CW)/2, Center, CW/2)

Return(last)
}

Function BlurV(clip c, int "rad", Float "CW")
{

Rad = Default(rad, 1)
CW = Default(CW, 0.5)

Center = C
Down = C.PointResize(C.width, C.height, 0, -rad, C.width, C.height)
Up = C.PointResize(C.width, C.height, 0, rad, C.width, C.height)

Average(Center, CW/2, Down, (1-CW)/2, Up, (1-CW)/2, Center, CW/2)

Return(last)
}

Function Tblur4(Clip c, int "thresh")
{

Thresh = Default(Thresh, 8)
B1 = C.yahr
Mt_LutXY(C, b1, " X Y - abs X Y - abs * X Y - * X Y - abs X Y - abs * "+string(thresh)+" + / 128 +", u=1, v=1)

Mt_AddDiff(last, b1, u=1, v=1)

Return(last)
}

Function NLLV(Clip C, int "rad")
{


Rad = Default(Rad, 1)

B1 = C.BlurV(1*rad, 0.439)
B2 = C.BlurV(3*rad, 0.833)
B3 = C.BlurV(5*rad, 0.934)
B4= C.BlurV(7*rad, 0.983)

B1_D = Mt_Makediff(B1, C, u=1, v=1)
B2_D = Mt_MakeDiff(C, B2, u=1, v=1)
B3_D = Mt_MakeDiff(B3, C, u=1, v=1)
B4_D = Mt_MakeDiff(C, B4, u=1, v=1)

B2_DT = Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 1.915 * X 128 - abs - 0 > \
Y 128 - abs 1.915 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)
B3_DT = Mt_LutXY(B2_DT, B3_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 2.025 * X 128 - abs - 0 > \
Y 128 - abs 2.025 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)
B4_DT = Mt_LutXY(B3_DT, B4_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 2.077 * X 128 - abs - 0 > \
Y 128 - abs 2.077 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)

Mt_AddDiff(B4_DT, C)

Return(last)
}

Function NLLH(Clip C, int "rad")
{


Rad = Default(Rad, 1)

B1 = C.BlurH(1*rad, 0.439)
B2 = C.BlurH(3*rad, 0.833)
B3 = C.BlurH(5*rad, 0.934)
B4= C.BlurH(7*rad, 0.983)

B1_D = Mt_Makediff(B1, C, u=1, v=1)
B2_D = Mt_MakeDiff(C, B2, u=1, v=1)
B3_D = Mt_MakeDiff(B3, C, u=1, v=1)
B4_D = Mt_MakeDiff(C, B4, u=1, v=1)

B2_DT = Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 1.915 * X 128 - abs - 0 > \
Y 128 - abs 1.915 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)
B3_DT = Mt_LutXY(B2_DT, B3_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 2.025 * X 128 - abs - 0 > \
Y 128 - abs 2.025 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)
B4_DT = Mt_LutXY(B3_DT, B4_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 2.077 * X 128 - abs - 0 > \
Y 128 - abs 2.077 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)

Mt_AddDiff(B4_DT, C)

Return(last)
}

function Halomask1(clip c, int "Rad", float "SS")
{

SS = default(ss, 1.5)
Rad = default(rad, 7)
owidth = c.width
oheight = c.height


ssample = spline36resize(c, round((c.width*SS)/4)*4, round((c.height*SS)/4)*4)

mask1 = mt_lutxy(ssample, Xsharpen(ssample, 127, 255), "x y - abs 1 - 63 *", u=1, v=1)
mask2 = (rad >= 1) ? mask1.mt_expand(u=1, v=1) : mask1
mask2 = (rad >= 2) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 3) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 4) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 5) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 6) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 7) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 8) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 9) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 10) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 11) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 12) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 13) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 14) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 15) ? mask2.mt_expand(u=1, v=1) : mask2
mask2 = (rad >= 16) ? mask2.mt_expand(u=1, v=1) : mask2
mask3 = mt_lutXY(mask1, mask2, "y x - 63 - 1.5 *", u=1, v=1).spline36resize(owidth, oheight)

return(mask3)
}



Function NLHH(Clip C)
{


B1 = Average(C.BlurH(1, 0.833), 0.5, C.BlurH(2, 0.833), 0.5)
B2 = C.BlurH(3, 0.750)
B3 = C.BlurH(5, 0.944)
B4= C.BlurH(7, 0.981)

B1_D = Mt_Makediff(c, b1, u=1, v=1)
B2_D = Mt_MakeDiff(b2, c, u=1, v=1)
B3_D = Mt_MakeDiff(c, b3, u=1, v=1)
B4_D = Mt_MakeDiff(b4, c, u=1, v=1)

B2_DT = Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 0.75 * X 128 - abs - 0 > Y 128 - abs 0.75 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)
B3_DT = Mt_LutXY(B2_DT, B3_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 2.25 * X 128 - abs - 0 > Y 128 - abs 2.25 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)
B4_DT = Mt_LutXY(B3_DT, B4_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 6.75 * X 128 - abs - 0 > Y 128 - abs 6.75 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)

Mt_AddDiff(B4_DT, C)

Return(last)
}

Function NLHV(Clip C)
{


B1 = Average(C.BlurV(1, 0.833), 0.5, C.BlurV(2, 0.833), 0.5)
B2 = C.BlurV(3, 0.750)
B3 = C.BlurV(5, 0.944)
B4= C.BlurV(7, 0.981)

B1_D = Mt_Makediff(c, b1, u=1, v=1)
B2_D = Mt_MakeDiff(b2, c, u=1, v=1)
B3_D = Mt_MakeDiff(c, b3, u=1, v=1)
B4_D = Mt_MakeDiff(b4, c, u=1, v=1)

B2_DT = Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 0.75 * X 128 - abs - 0 > Y 128 - abs 0.75 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)
B3_DT = Mt_LutXY(B2_DT, B3_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 2.25 * X 128 - abs - 0 > Y 128 - abs 2.25 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)
B4_DT = Mt_LutXY(B3_DT, B4_D, " X 128 - Y 128 - X 128 - abs Y 128 - abs * 1 + * X 128 - abs Y 128 - abs * Y 128 - abs 6.75 * X 128 - abs - 0 > Y 128 - abs 6.75 * X 128 - abs - 0 ? X 128 - abs 0 > X 128 - abs -1 X 128 - abs 0 > X 128 - 1 ? / ^ 1 ? / 1 * + 1 + / + 128 + ", u=1, v=1)

Mt_AddDiff(B4_DT, C)

Return(last)
}

Function TblurNL(Clip c, float "thresh", float "thresh2", float "thresh3", int "rad")
{


Rad = Default(Rad, 1)

B1 = C.NLLH(1*Rad)
B2 = B1.NLLV(1*Rad)
B3 = B2.NLLH(2*Rad)
B4 = B3.NLLV(2*Rad)
B5 = B4.NLLH(3*Rad)
B6 = B5.NLLV(3*Rad)



Thresh = Default(Thresh, 8)
Thresh2 = Default(Thresh2, Thresh/4)
Thresh3 = Default(Thresh3, Thresh/9)

Thresh1 = Thresh
Thresh2 = Thresh

Thresh3 = Thresh2
Thresh4 = Thresh2

Thresh5 = Thresh3
Thresh6 = Thresh3


Mt_LutXY(b5, b6, " X Y - abs 1 + X Y - abs 1 + * X Y - * X Y - abs 1 + X Y - abs 1 + * "+string(thresh6)+" + / 128 +", u=1, v=1)
Mt_AddDiff(last, b6, u=1, v=1)
b5 = last

Mt_LutXY(b4, b5, " X Y - abs 1 + X Y - abs 1 + * X Y - * X Y - abs 1 + X Y - abs 1 + * "+string(thresh5)+" + / 128 +", u=1, v=1)
Mt_AddDiff(last, b5, u=1, v=12)
b4 = last

Mt_LutXY(b3, b4, " X Y - abs 1 + X Y - abs 1 + * X Y - * X Y - abs 1 + X Y - abs 1 + * "+string(thresh4)+" + / 128 +", u=1, v=1)
Mt_AddDiff(last, b4, u=1, v=1)
b3 = last


Mt_LutXY(b2, b3, " X Y - abs 1 + X Y - abs 1 + * X Y - * X Y - abs 1 + X Y - abs 1 + * "+string(thresh3)+" + / 128 +", u=1, v=1)
Mt_AddDiff(last, b3, u=1, v=1)
b2 = last

Mt_LutXY(b1, b2, " X Y - abs 1 + X Y - abs 1 + * X Y - * X Y - abs 1 + X Y - abs 1 + * "+string(thresh2)+" + / 128 +", u=1, v=1)
Mt_AddDiff(last, b2, u=1, v=1)
b1 = last

Mt_LutXY(C, b1, " X Y - abs 1 + X Y - abs 1 + * X Y - * X Y - abs 1 + X Y - abs 1 + * "+string(thresh1)+" + / 128 +", u=1, v=1)
Mt_AddDiff(last, b1, u=1, v=1)

Return(Last)
}

Function TMed2(Clip C, int "thresh", int "rad", int "mode", int "kernel")
{

_Thr = Default(thresh, 256)
_Rad = Default(rad, 1)
Mode = Default(mode, 1)
THR = string(_Thr)
RAD = string(_Rad)
RAD2 = string(_Rad)
kernel = Default(kernel, 1)

Diff = Mode==0 ? MT_Luts(C, C, mode="med", pixels = "0 "+RAD+" "+RAD+" 0 -"+RAD+" 0 0 -"+RAD ,\
expr = "X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +", u=1,v=1)\
: MT_Luts(C, C, mode="med", pixels = "0 "+RAD+" "+RAD+" 0 -"+RAD+" 0 0 -"+RAD+" 0 0" ,\
expr = "X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +", u=1,v=1)
Diff = Kernel > 1 ? MT_Luts(C, C, mode="med", pixels = "-"+RAD2+" -"+RAD2+"-"+RAD2+" "+RAD2+" "+RAD2+" -"+RAD2+" "+RAD2+" "+RAD2+" 0 "+RAD+" "+RAD+" 0 -"+RAD+" 0 0 -"+RAD ,\
expr = "X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +", u=1,v=1) : Diff



MT_Makediff(C, Diff, u=2, v=2)

Return(Last)
}


Also I receive error with mclean.avsi.
Unfortunately, I receive error on the given version masktools2 х86

[Dogway]

I used script MedSharp2.avsi
Function MedSharp2(Clip C, int "thresh", int "rad", int "mode", bool "lp", bool "hp", bool "nr", float "str", int "kernel")
{

_Thr = Default(thresh, 256)
_Rad = Default(rad, 1)
Mode = Default(mode, 1)
str = Default(str, 1)
THR = string(_Thr)
RAD = string(_Rad)
RAD2 = string(_Rad)

lp = Default(lp, false)
hp = Default(hp, false)
nr = Default(nr, false)
kernel = Default(kernel, 2)


inputclipproperties = c
blank = BlankClip(length=inputclipproperties.framecount, width=inputclipproperties.width, height=inputclipproperties.height, pixel_type="rgb32", fps=inputclipproperties.framerate, fps_denominator=1, audio_rate=inputclipproperties.Audiorate, stereo=true, sixteen_bit=true, color=$828282).converttoyv12()


Lowpass1 = C.NLLV(rad=1).NLLH(rad=1)
Lowpass_Return = lp == true ? _Rad >= 2 ? Lowpass1 : C : C

Lowpass_NR_1 = Lowpass_Return.TMed2(rad=_Rad, thresh=_Thr, Kernel=1)
Lowpass_NR_2 = Lowpass_Return.TMed2(rad=_Rad, thresh=_Thr/4, Kernel=2)

Diff1 = MT_Luts(C, Lowpass_NR_1, mode="med", pixels = "0 "+RAD+" "+RAD+" 0 -"+RAD+" 0 0 -"+RAD+" 0 0" ,\
expr = "X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +", u=1,v=1)
Diff2 = MT_Luts(C, Lowpass_NR_2, mode="med", pixels = "-"+RAD2+" -"+RAD2+"-"+RAD2+" "+RAD2+" "+RAD2+" -"+RAD2+" "+RAD2+" "+RAD2+" 0 "+RAD+" "+RAD+" 0 -"+RAD+" 0 0 -"+RAD ,\
expr = "X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 2 * 1 >= "+THR+" 2 * 0.5 ^ "+THR+" 2 * ? + / - 128 +", u=1,v=1)

mask1 = Mt_Lut(Diff1, " X 128 - 255 * ", u=1, v=1)#.NLLV().NLLH()
#mask2 = Mt_Lut(Diff2, " X 128 - 255 * ", u=1, v=1)#.NLLV().NLLH()

s1 = mt_merge(Diff2, blank, mask1).mt_Lut("X 128 - "+string(str)+" 2 / * 128 +")
s1b = s1.NLLV().NLLH()
s1s = mt_makediff(s1, s1b)
s2 = Diff1.mt_Lut("X 128 - "+string(str)+" * 128 +")
s2b = s1.NLLV().NLLH()
s2s = mt_makediff(s2, s2b)


out = MT_adddiff(C, s1s, u=2, v=2)
out = MT_adddiff(out, s2s, u=2, v=2)
out

Return(Last)
}

The part in red is wrong, requires a space before the minus (-). You can also try my optimized version in SharpenersPack (https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/SharpenersPack.avsi)but it involves other dependencies.

[Tempter57]

Dogway
No, my problem:
mt_lutxy : error at position 114: cannot convert to number: \
(c:\Program Files (x86)\XviD4PSP 5\dlls\AviSynth\plugins\medsharp2.avsi, line 123)
(c:\Program Files (x86)\XviD4PSP 5\dlls\AviSynth\plugins\medsharp2.avsi, line 22)
I used AviSynth + 3.5.1 x86, with masktools 2.2.26 very well

[Dogway]

Ok, sorry, I had that issue when porting it. Can be a regression.

[kedautinh12]

2.2.30
https://github.com/pinterf/masktools/releases

[Blankmedia]

In 2.2.30 with corners=false, the more I multithread it becomes unstable.

masque = mt_hysteresis(mask_flatC, mask_flatN, corners=false)

I have to set Prefetch(threads=5, frames=10).

With

masque = mt_hysteresis(mask_flatC, mask_flatN, corners=True)

I can get to Prefetch(threads=16, frames=24)

I don't know how to be more helpful.

Edit: Scratch all that, I rebooted and it seems to work fine.

[Ceppo]

@pinterf
I did CQTGMC (https://forum.doom9.org/showthread.php?p=1963693#post1963693) and I believe I can get a huge speed up if MFlowInter had a thSAD parameter like MCompensate. If you are interested of course.

[VoodooFX]

@pinterf
I did CQTGMC (https://forum.doom9.org/showthread.php?p=1963693#post1963693) and I believe I can get a huge speed up if MFlowInter had a thSAD parameter like MCompensate. If you are interested of course.

But MFlowInter is not from Masktools2. ;)
Try there: https://github.com/pinterf/mvtools/issues

[Ceppo]

Sorry I was in a hurry :D:D:D:D I got the wrong thread lol