MaskTools2 - pfmod

[pinterf]

MaskTools2, high bit depth support.

Forked from tp7's masktools2 repository.
Thanks to all earlier contributors.

Get the latest release from here (https://github.com/pinterf/masktools/releases)

This fork incorporates tp7's abandoned initial work on 16 bit support (using stacked and interleaved 8 bit hacks back in 2013), which he has never released, but proved to be an invaluable help with my initial steps porting this basic plugin.

All filters are available for all bit depths. Avisynth+ color spaces with alpha (YUVA/RGBA) are not supported. XP version does not support mt_polish.

This release may have benefits for 8 bit users as well, see change log.

As always, report back bugs, regressions. Thank you.

Useful links:

Masktools2 info:
http://avisynth.nl/index.php/MaskTools2

Article by tp7
http://tp7.github.io/articles/masktools/

Project:
https://github.com/pinterf/masktools/tree/16bit/

Original version: tp7's MaskTools 2 repository.
https://github.com/tp7/masktools/

Masktools 2.0.a48 forum:
https://forum.doom9.org/showthread.php?t=98985

[Sparktank]

Amazing stuff! Thanks for the continued work on all the recent projects :)

[real.finder]

Thanks

so we will need edit all old scripts that has lut for > 8 bit now (adding the new syntax)? or not?

[pinterf]

There is no intuitive way deciding whether a constant is scalable or not.
I checked the idea on the examples here: http://avisynth.nl/index.php/MaskTools2/mt_lutspa
At some samples I could not tell for the first sight with 100% confidence which constants are to scale and which are not. An averager /2 is surely not scalable.

[real.finder]

There is no intuitive way deciding whether a constant is scalable or not.
I checked the idea on the examples here: http://avisynth.nl/index.php/MaskTools2/mt_lutspa
At some samples I could not tell for the first sight with 100% confidence which constants are to scale and which are not. An averager /2 is surely not scalable.

The changes log are not clear for me, did you added new bitdepth parameter? and what the default of it? 8 bit?

[pinterf]

The changes log are not clear for me, did you added new bitdepth parameter? and what the default of it? 8 bit?
You don't have to provide it, it is automatically passed when the expression is evaluated. It is used internally in #B and #F but you can check for it using variable name 'bitdepth'

[real.finder]

You don't have to provide it, it is automatically passed when the expression is evaluated. It is used internally in #B and #F but you can check for it using variable name 'bitdepth'

I already get the idea of #B and #F, but my point is for the script that already existing which don't has #B and #F or any thing for bit depth, how will they work in more than 8 bit?

[pinterf]

Existing 8 bit scripts will work as the expression arithmetic says and they will be incorrect on higher bit depths, unless you do only and "x y + 2 /" which means the same (x+y)/2 for all formats. So if you don't take care of the constants, it won't give proper results.

If you subtract N, it will subtract exactly N, even if N=128 and you know that in 8 bit world it means the half range. But in 16 bit this 128 can be a rounder or anything that is meant to be exactly 128.

I think we have no reason to intuitively guess from a constant that it should be scaled or not.

[real.finder]

Existing 8 bit scripts will work as the expression arithmetic says and they will be incorrect on higher bit depths

so, that mean we will need to edit most existing scripts

[real.finder]

another thing, let say that we want to write in 16 bit or float for some reason and in the same time we want it to work in 8 bit or 10 bit etc... too

[yup]

Hi pinterf!

Please explain, all filter from Your rewritten plugin MT_NICE_FILTER?
Where I can find ready to use list? For example QTGMC script use a lot of function and any could declared before use (for better performance).
I think that if filter MT_NICE_FILTER one will be work better if declared like MT_NICE_FILTER, but not default MT_MULTI_INSTANCE.

yup.

[pinterf]

They are auto registering themselves as nice filter.
EDIT:
info added to the first page

[yup]

pinterf!
They are auto registering themselves as nice filter.
And no need additional commands for this? All work from box?

yup.

[DJATOM]

pinterf!

And no need additional commands for this? All work from box?

yup.

Yes, if filter is registering himself, your mt setting will be ignored unless you'll force it (force=true).

[pinterf]

another thing, let say that we want to write in 16 bit or float for some reason and in the same time we want it to work in 8 bit or 10 bit etc... too
We are at the very beginning of these questions, so you'd like an easy way to tell the expression evaluator the default bit depth of the used constants, which is now 8 bits to help the relatively painless transition of the next years.
And the autoscaler would take it into account, and it scaled up and down the numbers if the default bitdepth for scaling is e.g. set to 16 bit instead of 8

EDIT:
maybe the expression string would contain control mnemonics at the beginning, hinting the default bitdepth of the scalable constants if they are not meant to be 8-bit base.
e.g. expr = "i16 x 32768 #B -" where "i16" (in general: i8/i10/i12/i14/i16 for integer formats or f32 for 32 bit float) hints the expression evaluator that the base bit depth of the scalable constants is 16 bits and has to scale it accordingly with the #B or #F operator?

[kgrabs]

Wow, thanks a lot for this! Just having the luts is fantastic, it's really a great thing you've got here

I tried porting over Dehalo_alpha but after about 2 hours it seemed to be a little beyond me. It looks kinda like Dehalo_alpha(darkstr=100, brightstr=100).
I'll post my (incorrect) work if someone feels like figuring it out, and to make it easier the mt_lutxy expressions of the original Dehalo_Alpha, via mt_infix, too:

http://i.imgur.com/Vqx9vhf.jpg

#gives incorrect output
#assumes stack16 input
Function DeHalo_Alpha16(clip input, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss", int "srad", bool "cs", bool "uv")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )
srad = default( srad, 1 ) # srad=3 is DeHalo_Alpha_2BD
cs = default( cs, false )
uv = default( uv, true )

assumeY = IsY(input)

clps = assumeY ? input
\ : input.ConvertToY()
clp = clps.ConvertFromStacked(bits=16)

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr * 256.0)
BRT = string(brightstr * 256.0)
ox = input.width()
oy = input.height()/2
ssx = Round(ox*ss)
ssy = Round(oy*ss)
dsx = Round(ox/rx)
dsy = Round(oy/ry)

halos = clps.dither_resize16(dsx,dsy,kernel="bicubic",U=1,V=1).dither_resize16(ox,oy,1,0,kernel="bicubic",U=1,V=1).ConvertFromStacked(bits=16)
are = mt_lutxy(clp.DHA16_expand(srad),clp.DHA16_inpand(srad),"x y -",U=1,V=1)
ugly = mt_lutxy(halos.DHA16_expand(srad),halos.DHA16_inpand(srad),"x y -",U=1,V=1)
so = mt_lutxy(ugly, are, "y x - y 0.256 + / 65535 * "+LOS+" - y 65535 + 131070 / "+HIS+" + *",U=1,V=1)
lets = mt_merge(halos,clp,so,U=1,V=1)
remove = (ss==1.0) ? clps.dither_repair16(lets.ConvertToStacked(), 1, -1, -1).ConvertFromStacked(bits=16)
\ : clps.dither_resize16(ssx,ssy,kernel="lanczos",taps=3,U=1,V=1).ConvertFromStacked(bits=16)
\ .mt_logic(lets.mt_expand(U=1,V=1).ConvertToStacked().dither_resize16(ssx,ssy,kernel="bicubic",U=1,V=1).ConvertFromStacked(bits=16),"min",U=1,V=1)
\ .mt_logic(lets.mt_inpand(U=1,V=1).ConvertToStacked().dither_resize16(ssx,ssy,kernel="bicubic",U=1,V=1).ConvertFromStacked(bits=16),"max",U=1,V=1)
\ .ConvertToStacked().dither_resize16(ox,oy,kernel="lanczos",taps=3,U=1,V=1).ConvertFromStacked(bits=16)
them = mt_lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=1,V=1).ConvertToStacked()
them = cs ? them.DHA16_contrasharp16(clps) : them
return assumeY ? them : uv ? ytouv(DHA16_utoy(input), DHA16_vtoy(input), them)
\ : them.DHA16_ConvertToX(input)
}

Function DHA16_utoy(clip c){
try {
utoy(c)
} catch ( error_msg ) {
utoy8(c)
}
}
Function DHA16_vtoy(clip c){
try {
vtoy(c)
} catch ( error_msg ) {
vtoy8(c)
}
}
Function DHA16_ConvertToX(clip output, clip input){return Is420(input) ? output.ConvertToYUV420() : Is422(input) ? output.ConvertToYUV422(): Is444(input) ? output.ConvertToYUV444() : output.ConvertToYV411()}
function DHA16_contrasharp16(clip after, clip before){return after.Dither_add16(Dither_sub16(after, after.Dither_RemoveGrain16(mode=11, modeU=-1, modeV=-1), U=1, V=1, dif=true).Dither_Repair16(Dither_sub16(before, after, U=1, V=1, dif=true), mode=13, modeU=-1, modeV=-1), U=1, V=1, dif=true)}
function DHA16_expand(clip c, int expand){return expand > 0 ? DHA16_expand(c.mt_expand(U=1, V=1), expand-1) : c}
function DHA16_inpand(clip c, int inpand){return inpand > 0 ? DHA16_inpand(c.mt_inpand(U=1, V=1), inpand-1) : c}

[real.finder]

I tried porting over Dehalo_alpha but after about 2 hours it seemed to be a little beyond me. It looks kinda like Dehalo_alpha(darkstr=100, brightstr=100).
I'll post my (incorrect) work if someone feels like figuring it out, and to make it easier the mt_lutxy expressions of the original Dehalo_Alpha, via mt_infix, too:

http://i.imgur.com/Vqx9vhf.jpg


I already intend to port dehalo_alpha and other scripts (https://forum.doom9.org/showthread.php?t=174121) but wait for pinterf to made the final decisions, no need to hurry

[real.finder]

We are at the very beginning of these questions, so you'd like an easy way to tell the expression evaluator the default bit depth of the used constants, which is now 8 bits to help the relatively painless transition of the next years.
And the autoscaler would take it into account, and it scaled up and down the numbers if the default bitdepth for scaling is e.g. set to 16 bit instead of 8

EDIT:
maybe the expression string would contain control mnemonics at the beginning, hinting the default bitdepth of the scalable constants if they are not meant to be 8-bit base.
e.g. expr = "i16 x 32768 #B -" where "i16" (in general: i8/i10/i12/i14/i16 for integer formats or f32 for 32 bit float) hints the expression evaluator that the base bit depth of the scalable constants is 16 bits and has to scale it accordingly with the #B or #F operator?

why just not add new exprBit parameter? with 8 bit default

edit: but your one is ok too

[pinterf]

New release, full high bit depth support (10-16, float) for all filters, new flavour in expression syntax.

For details, see first post.

Enjoy Masktools2 v2.2.2 (20170223) (https://github.com/pinterf/masktools/releases/tag/2.2.2)

Change list from v2.2.1


-All filters work in 10,12,14,16 bits and float (except mt_logic which is 8-16 only)
- mt_lutxyza 4D lut available with realtime=false!
(4 GBytes LUT table, slower initial lut table calculation)
Allowed only on x64. Be warned, it would take minutes.
- mt_gradient 10-16 bit / float
- mt_convolution 10-16 bit / float
- mt_motion 10-16 bit / float
- mt_xxpand and mt_xxflate to float
- mt_clamp to float
- mt_merge to float
- mt_binarize to float
- mt_invert to float
- mt_makediff and mt_adddiff to float
- mt_average to float
- Expression syntax supporting bit depth independent expressions:
Added configuration keywords i8, i10, i12, i14, i16 and f32 in order to inform the
expression evaluator about bit depth of the values that are to scale by #B and #F operators.

[real.finder]

ok, just did small port to DeHalo_alpha, I think we need IsVideoFloat() in avs+


# 2017.02.25 1st release
# DeHalo_alpha with high bit support (now only up to 16, no float yet), need masktools 2.2.2, avs25 dropped by now
# modified dehalo_alpha: here with increased search-radius for validation

function DeHalo_alpha_2BD(clip input, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss", bool "cs", int "search_rade", int "search_radi")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )
cs = default( cs, false )
sre = default( search_rade, max(round(max(rx,ry)),3) )
sri = default( search_radi, sre )

ssispmt = Findstr(VersionString(), "AviSynth+") != 0 && Findstr(VersionString(), "r1576") == 0

sislumaonly = ssispmt ? input.isy() : input.isy8()

clp = sislumaonly ? input : ssispmt ? input.converttoy() : input.converttoy8()

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr)
BRT = string(brightstr)
ox = clp.width()
oy = clp.height()

halos = clp.bicubicresize(round(ox/rx),round(oy/ry)).bicubicresize(ox,oy,1,0)
are = mt_lutxy(clp .srdha2e(sre,1),clp .srdha2i(sri,1),"x y -",U=1,V=1)
ugly = mt_lutxy(halos.srdha2e(sre,1),halos.srdha2i(sri,1),"x y -",U=1,V=1)
so = mt_lutxy( ugly, are, "y x - y 0.001 #F + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *" )
lets = mt_merge(halos,clp,so,U=1,V=1)
remove = (ss==1.0) ? clp.repair(lets,1,0)
\ : clp.lanczosresize(round(ox*ss),round(oy*ss))
\ .mt_logic(lets.mt_expand(U=1,V=1).bicubicresize(round(ox*ss),round(oy*ss)),"min",U=2,V=2)
\ .mt_logic(lets.mt_inpand(U=1,V=1).bicubicresize(round(ox*ss),round(oy*ss)),"max",U=2,V=2)
\ .lanczosresize(ox,oy)
them = mt_lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=2,V=2)

bb = cs ? them.removegrain(11) : nop()
xD = cs ? mt_makediff(bb,bb.repair(bb.repair(bb.medianblur(2,-333,-333),1),1)).mt_lut("x range_half - 2.49 #F * range_half +") : nop()
xDD = cs ? mt_lutxy(xD,mt_makediff(clp,them),"x range_half - y range_half - * 0 < range_half x range_half - abs y range_half - abs < x y ? ?") : nop()
them = cs ? them.mt_adddiff(xDD,U=2,V=2) : them

them = sislumaonly ? them : ssispmt ? CombinePlanes(them,input,planes="YUV",sample_clip=input) : ytouv(input.utoy8(),input.vtoy8(),them)

return( them )
}

Function srdha2e(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : srdha2e(i.mt_expand(), dr_rad, count+1)
}

Function srdha2i(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : srdha2i(i.mt_inpand(), dr_rad, count+1)
}

[pinterf]

BitsPerComponent=32? I hope it exists. I see you the script is using mt_logic. Is this the reason for checking float?
Ps.: I imagine how slow it can be with realtime lutxy :)

[real.finder]

BitsPerComponent=32? I hope it exists. I see you the script is using mt_logic. Is this the reason for checking float?
Ps.: I imagine how slow it can be with realtime lutxy :)

BitsPerComponent=32 work, but as there are already IsAudioFloat() I think it will make things more easy and clean

yes it's because mt_logic, and if mt_logic work with Float in future I think the expr need some special edit for Float, so IsVideoFloat() will make that less hard

[pinterf]

min and max. That makes sense in float.
Are there many script that are using explicite threshold parameter value? Such values are treated as-is, no scaling there.

[real.finder]

min and max. That makes sense in float.
Are there many script that are using explicite threshold parameter value? Such values are treated as-is, no scaling there.

didn't check, but I think yes, Didée alone did a lot of functions and most them use lut

[pinterf]

I'm thinking on not the expr in lut-like filters, but on the mt_xxxx direct parameters, e.g. thY1, thC1 in mt_edge, which are subject to scale and it is now the script writer's task to give properly scaled numbers for them.

[real.finder]

I'm thinking on not the expr in lut-like filters, but on the mt_xxxx direct parameters, e.g. thY1, thC1 in mt_edge, which are subject to scale and it is now the script writer's task to give properly scaled numbers for them.

for this, I think it's better to add ParametersBit parameter with 8 (or even better "i8") default

[real.finder]

did another port


# VHSHaloremover with high bit support, need masktools 2.2.2, avs25 dropped by now
# 2017.02.24 1st release
Function VHSHaloremover(clip input, int "xradius", int "yradius", int "darkthr", int "brightthr", float "blurmix")
{
ssispmt = Findstr(VersionString(), "AviSynth+") != 0 && Findstr(VersionString(), "r1576") == 0
sislumaonly = ssispmt ? input.isy() : input.isy8()
sislumaonly ? input : ssispmt ? input.converttoy() : input.converttoy8()
h = Height ()
w = Width ()
xradius = Default(xradius, 8)
yradius = Default(yradius, 1)
darkthr = Default(darkthr, 122)
brightthr = Default(brightthr, 134)
blurmix = Default(blurmix, 0.8)
cy = BicubicResize (w / xradius, h / yradius, 1, 0).BicubicResize (w, h, 1, 0)
cmd = mt_makediff (cy)
cm = cmd.mt_lut ("x "+String(brightthr)+" #F > range_max x "+String(darkthr)+" #F < range_max 0 ? ?")
cr = last.Merge (cy, blurmix)
mt_merge (last, cr, cm, y=3, u=2, v=2)
sislumaonly ? last : ssispmt ? CombinePlanes(last,input,planes="YUV",sample_clip=input) : ytouv(input.utoy8(),input.vtoy8(),last)
}


it should work with float but it gave me wrong result, is it bug in #F? or there are no float scale yet?

[pinterf]

Strange, when I replace
cr = last.Merge (cy, blurmix)
with
mask= last.mt_lut("f32 "+String(blurmix)+" #F")
cr = last.mt_merge (cy, mask, luma=true, y=3, u=3, v=3)
(is it o.k.?)
Then it works.
edit: arrrrgh. There was a bug in float merge (in Avs+ r2420 and before)

[martin53]

I currently look for steps towards a hdracg replacement (http://forum.doom9.org/showthread.php?p=1796949#post1796949) (the former paviko hdr contrast enhancer), since paviko did not release recent sources and cannot be contacted.
And I'd prefer a combination of basic plugin-powered building blocks used in a script function at least for the design phase, when experiments are more valuable than performance, over the closed plugin-does-all approach.

The 1st essential process for the paper hdragc was based on, is a special luma average of surrounding pixels - but restricted to those pixels with not too different luma from the center pixel.

mt_luts() can already yield the average of pixels' environments, but not with the stated restriction.
This could be solved by not using implicit weight=1 for all pixels, but using a weight expression when calculating the mt_luts() "mode" operation.
So mt_luts() then would e.g. have four new parameters string wxpr("1"), string ywxpr("1"), string uwxpr("1"), string vwxpr("1").
For the described task, wxpr would be e.g. mt_polish("y < x - 5 || y > x + 5 ? 0 : 1"), or something smoother like mt_polish("exp(-((x-y)/5)^4)").

Please note that also for modes "min" etc, an integer weight can be useful. E.g. with wxpr=mt_polish("y < x -10 ? 0 : 1") the darkest pixel in the environment can be found that is not darker than 10 increments below the center pixel.

I'm mostly asking pinterf, since you already studied the masktools plugin thoroughly: Do you see my point, and think you might add these parameters, or maybe a simpler/more powerful alternative?

(NOTE: Don't bother with me using the "||" OR operator for the example while masktools expect "|" instead - I know that and additional masktools operators "||", "&&" doing the same as "|" and "&" would have been a fine idea all the years )

[tormento]

ok, just did small port to DeHalo_alpha, I think we need IsVideoFloat() in avs+
What is m4_dham?

[pinterf]

I was so happy when I finally found out how plain mt_luts was working :) O.k., reading and trying to understand.

This would imply a second set lof 2D lut tables (or realtime expressions), where x is the center (current) pixel, and y is the pixel from the neighborhood.
In your idea, when we are e.g. in "average" mode, we are accumulating weighted sums, then this sum is divided by the sum of weights at the end.
And we find out something for the situation where the sum of weights is zero.

Now that I'm typing this post, I understand better the logic behind it.

[ajp_anton]

Would it be possible to add an alias to the ° xor symbol that's more common on keyboards? For example just being able to write "xor".

[pinterf]

More common :)
I think, yes. I used the copy paste method which is only one step from the perfection.

[real.finder]

What is m4_dham?

it was for resize limit in avs25, I forget to remove all of it, just did another update for the script

[pinterf]

Would it be possible to add an alias to the ° xor symbol that's more common on keyboards? For example just being able to write "xor".
Checked the source, you can use the @ sign instead. (and @u, @s)

[tormento]

it was for resize limit in avs25, I forget to remove all of it, just did another update for the script

Tried but did not gave me expected results with 28 days after:

http://i.imgur.com/47tc972t.jpg (http://imgur.com/47tc972)

Is the white shade around contrasted objects the "halo" you meant to remove with this script or it has another name?

[real.finder]

Tried but did not gave me expected results with 28 days after:

http://i.imgur.com/47tc972t.jpg (http://imgur.com/47tc972)

Is the white shade around contrasted objects the "halo" you meant to remove with this script or it has another name?

yes, try play with the parameters, the output should be same with original dehalo_alpha so this will be off topic, unless you see big difference between with 8 bit and more than 8 bit

[martin53]

This would imply a second set lof 2D lut tables (or realtime expressions), where x is the center (current) pixel, and y is the pixel from the neighborhood.
Yes. That's what I guess, too.
In your idea, when we are e.g. in "average" mode, we are accumulating weighted sums
And we find out something for the situation where the sum of weights is zero.
Right. There is already a similar faulty situation today: When the 'pixels' parameter contains no pixel, then there is already 0/0, although for the whole frame. Checked mt_luts() currently returns "x" in this case
colorbars(pixel_type="YV12")
mt_luts(last, last.invert, mode="avg", pixels="")

Thank you for reading and making familiar with the matter

[pinterf]

New release.

Download Masktools2 v2.2.3 (https://github.com/pinterf/masktools/releases/tag/2.2.3)

This version now has full support for popular 8-32 bit formats (though still no YUVA/RGBA), partial AVX/AVX2 for those rare non-SSE2 users :)

There is a new addition to mt_luts for martin53. See first post. Once it is finalized, I hope, martin53 will help in the documentation (and answering all questions), because my trial of describing the new features in English needs polishing :)

Changelog
**v2.2.3 (20170227)**
- mt_logic to 32 bit float (final filter lacking it)
- get CpuInfo from Avisynth (avx/avx2 preparation)
Note: AVX/AVX2 prequisites
- recent Avisynth+ which reports extra CPU flags
- 64 bit OS (but Avisynth can be 32 bits)
- Windows 7 SP1 or later
- mt_merge: 8-16 bit: AVX2, float:AVX
- mt_logic: 8-16 bit: AVX2, float:AVX
- mt_edge: 10-16 bit and 32 bit float: SSE2/SSE4 optimization
- mt_edge: 32 bit float AVX
- new: mt_luts: weight expressions as an addon for then main expression(s) (martin53's idea)
- wexpr
- ywExpr, uwExpr, vwExpr
If the relevant parameter strings exist, the weighting expression is evaluated
for each source/neighborhood pixel values (lut or realtime, depending on the bit depth and the "realtime" parameter).
Then the usual lut result is premultiplied by this weight factor before it gets accumulated.

[martin53]

- new: mt_luts: weight expressions
- wexpr
- ywExpr, uwExpr, vwExpr

:thanks: will soon show usefulness (hopefully) in 'hdragc alternative' thread :)

[pinterf]

:thanks: will soon show usefulness (hopefully) in 'hdragc alternative' thread :)
I hope it will be working the way you were proposing, it worked nicely with fix constants though, wexpr = "0.1", "1.4", "10.8", this latter multiplier was enough to show up nasty overflow errors. Good luck, I'm looking forward to your experiments.

[real.finder]

I made DeHalo_alpha work in float, but I don't think it is the perfect way, cuz we have extra lut for float now to remove the negative pixels that occurred in the mt_lutxy


# 2017.02.27 1st release
# DeHalo_alpha with high bit support, need masktools 2.2.3, avs25 dropped by now
# modified dehalo_alpha: here with increased search-radius for validation

function DeHalo_alpha_2BD(clip input, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss", bool "cs", int "search_rade", int "search_radi")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )
cs = default( cs, false )
sre = default( search_rade, max(round(max(rx,ry)),3) )
sri = default( search_radi, sre )

ssispmt = Findstr(VersionString(), "AviSynth+") != 0 && Findstr(VersionString(), "r1576") == 0

sislumaonly = ssispmt ? input.isy() : input.isy8()

clp = sislumaonly ? input : ssispmt ? input.converttoy() : input.converttoy8()

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr)
BRT = string(brightstr)
ox = clp.width()
oy = clp.height()

halos = clp.bicubicresize(round(ox/rx),round(oy/ry)).bicubicresize(ox,oy,1,0)
are = mt_lutxy(clp .srdha2e(sre,1),clp .srdha2i(sri,1),"x y -",U=1,V=1)
ugly = mt_lutxy(halos.srdha2e(sre,1),halos.srdha2i(sri,1),"x y -",U=1,V=1)
so = clp.BitsPerComponent==32 ? mt_lutxy( ugly, are, "y 255 * x 255 * - y 255 * 0.001 + / 255 * "+LOS+" - y 255 * 256 + 512 / "+HIS+" + * 255 /" ).mt_lut("x 0 < 0 x ?")
\ : mt_lutxy( ugly, are, "y x - y 0.001 + / range_max * "+LOS+" #F - y range_size + 512 #F / "+HIS+" + *" )
lets = mt_merge(halos,clp,so,U=1,V=1)
remove = (ss==1.0) ? clp.repair(lets,1,0)
\ : clp.lanczosresize(round(ox*ss),round(oy*ss))
\ .mt_logic(lets.mt_expand(U=1,V=1).bicubicresize(round(ox*ss),round(oy*ss)),"min",U=2,V=2)
\ .mt_logic(lets.mt_inpand(U=1,V=1).bicubicresize(round(ox*ss),round(oy*ss)),"max",U=2,V=2)
\ .lanczosresize(ox,oy)
them = mt_lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=2,V=2)

bb = cs ? them.removegrain(11) : nop()
xD = cs ? mt_makediff(bb,bb.repair(bb.repair(bb.medianblur(2,-333,-333),1),1)).mt_lut("x range_half - 2.49 #F * range_half +") : nop()
xDD = cs ? mt_lutxy(xD,mt_makediff(clp,them),"x range_half - y range_half - * 0 < range_half x range_half - abs y range_half - abs < x y ? ?") : nop()
them = cs ? them.mt_adddiff(xDD,U=2,V=2) : them

them = sislumaonly ? them : ssispmt ? CombinePlanes(them,input,planes="YUV",sample_clip=input) : ytouv(input.utoy8(),input.vtoy8(),them)

return( them )
}

Function srdha2e(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : srdha2e(i.mt_expand(), dr_rad, count+1)
}

Function srdha2i(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : srdha2i(i.mt_inpand(), dr_rad, count+1)
}

[martin53]

Hi pinterf,

possible thing to do with high bitdepth:
I think the mt_polish() helper function might not yet know the new predefined high bitdepth constants?

E.g. mt_polish("x < 100 ? x : range_max") should return
"x 100 < x range_max ?" but returns only the invalid expression fragment "x 100 < x".

[tormento]

New release.Download Masktools2 v2.2.3 (https://github.com/pinterf/masktools/releases/tag/2.2.3)


github-cloud.s3.amazonaws.com down :(

[martin53]

expression-weighted mt_luts() works well! :thanks: I am struggling a bit with a good working example (is more complicated than I expected), will publish soon.

github-cloud.s3.amazonaws.com down :(
Try again? Works fine here

[pinterf]

Hi pinterf,

possible thing to do with high bitdepth:
I think the mt_polish() helper function might not yet know the new predefined high bitdepth constants?

E.g. mt_polish("x < 100 ? x : range_max") should return
"x 100 < x range_max ?" but returns only the invalid expression fragment "x 100 < x".
Uh-oh. Will look at it.

EDIT:
Thank you, fixed. Added the 'a' letter of mt_lutxyza, names of the scaling helper constants, the #B and #F operators (they should appear as a one operand function e.g. #F(255)).
And there were other missing operators as well, mainly bit-operators (&u, |s, etc...), and the "@" (synonym for the never-found-degree-sign-for-xor).
No release yet.

[real.finder]

hi pinterf, what about add something (option) to clamp to 1.0-0.0 in float in lut?

and what range_size will do in float? will return 1.0?

[pinterf]

hi pinterf, what about add something (option) to clamp to 1.0-0.0 in float in lut?

and what range_size will do in float? will return 1.0?

Range_size is 1.0.

Though it is a basic rule that floats are not clamped, but sometimes, in known situations it is necessary, if we want to mimic the algorithm of the integer color spaces.

I was thinking on the problem, after I looked at your script with if float do_clamped_workaround else do_normal.

One solution can be an extra parameter for all lut functions, another workaround would be to introduce a keyword that can be used inside the expression similar to the i8..i16,f32.
"clamp_f" will only affect and clamp the result to 0..1 when the input clip format is float.

And this propagates another problem, Vapoursynth float format is -0.5..+0.5 for chroma, which should be the normal representation in Avisynth+, but for historical reasons it is 0..1.0 for all planes.

If once in the future it would change then ... I don't know..

And this means that for chroma things the clamping should occur on +/-0.5, and it needs another hinting keyword, clamp_f_chroma.

I wonder how VS users handle the 'let's write full generic scripts' situation, perhaps they don't care it because they don't mix integer and float together, or branch the relevant parts of the scripts?

[real.finder]

clamp_f sound good, and if it (clamp_f) make any x and y etc... from 1.0-0.0 to 255.0-0.0 for process like 8 bit then back to 1.0-0.0 will be better and we don't worry about do something special for float, in other word we will have one expr for both

and that also mean clamp_f will make #F and Range_size etc... keep or convert to like 8 bit expr (255.0-0.0)

[pinterf]

Why do you need that float input autoscaling? The usual #B and #F are working properly for the base float range.