Masktools - disable clipping (e.g. allow luma 235 >>> 240)

[Stormborec]

Is it possible to set up in the Masktools something like coring off / disable clipping?

[foxyshadis]

Masktools2 (the "mt_" version) has no notion of clipping, it won't clip except to 0 and 255. No idea about old original MaskTools, no one's used that in ten years, but it should act the same way.

[Stormborec]

In that case, I don't understand why blue is stuck at "188 " , no matter how much I turn "x y 16 - 0.2 * +" ...

a=blankclip(color=$989898)
b=blankclip(color=$b1b1b1)
c=blankclip(color=$cacaca)
d=blankclip(color=$e4e4e4)

interleave(a,b,c,d).converttoyv12

y = converttoy8()
u = utoy8()
v = vtoy8()
y_scaled = y.bicubicresize(width(u), height(u))
y_adjusted = mt_lut(y,expr="x")
u_adjusted = mt_lutxy(u,y_scaled,expr="x y 16 - 0.2 * +")
v_adjusted = mt_lutxy(v,y_scaled,expr="x y 16 - -0.077 * +")
ytouv(u_adjusted,v_adjusted,y_adjusted)


converttorgb.rgbadjust(1.1,1,0.73772,analyze=true)

[foxyshadis]

Well, in your RGBAdjust, 0.73772 * 255 is 188, so I think you have your answer right there.

As for masktools, the maximum Y is 255, and your chroma is always neutral (128), so the maximum output of "(Y - 16) * 0.2 + 128" is 175.8. If you change 0.2 to 1, or you test with colored blank clips, you can easily get 255. Try:


orignum = stackvertical(y.subluma,StackHorizontal(u.subluma,v.subluma))
adjustednum = stackvertical(y_adjusted.subluma,StackHorizontal(u_adjusted.subluma,v_adjusted.subluma))

function subluma(clip c) {
return ScriptClip(c, "Subtitle(String(AverageLuma))")
}

[Stormborec]

I do not know if I understand exactly ...
But I already know what the problem is:

188 / 0.73772 >>> 255

converttorgb guilelessly cuts to 0 255 ...

[Asmodian]

converttorgb guilelessly cuts to 0 255 ...

If I understand this isn't it as simple as the .rgbadjust(1.1,1,0.73772,analyze=true) converting the 255 output from ConvertToRGB to 188? This is what would have to happen as ConvertToRGB would convert to legal range RGB values (0-255) and 255*0.73772 = 188. You wouldn't expect a >255 value for blue out of ConvertToRGB would you?

[colours]

Adding on a bit to Asmodian's post, Avisynth natively supports only 8-bpc colour formats. With eight bits, a maximum of 256 values are valid, and by convention these are taken to be the integers between 0 and 255, inclusive.

If anywhere in your filter chain a filter produces a value that exceeds these limits, it has to either wrap (https://en.wikipedia.org/wiki/Modular_arithmetic) or saturate (https://en.wikipedia.org/wiki/Saturation_arithmetic). The former is generally not a very useful option in image processing, but either way, your pixel values will always lie between 0 and 255.

Taking a simple example, say you want to multiply every pixel's value by 2, then divide that by 2. If you do both steps within the same filter (e.g. mt_lut("x 2 * 2 /")) you end up with the original frame under some reasonable assumptions. But if you do the steps in different filters (e.g. mt_lut("x 2 *").mt_lut("x 2 /")), the values will have been clipped by the first filter and this will never produce a result greater than 127.

[Stormborec]

This is exactly what I wanted to ask ...
How Avisynth works internally in the chain of filters?
If I have e.g.:
avisource.tweak.coloryuv

is it:
clip(yuv=intiger values)
-> tweak -> clip(yuv=intiger values)
-> coloryuv -> clip(yuv=intiger values)

or:
clip(yuv=intiger values)
-> tweak -> (yuv=float/or something else values)
-> coloryuv -> clip(yuv=intiger values)

[StainlessS]

Tweak(clip clip [, float hue] [, float sat] [, float bright] [, float cont] [, bool coring] [, bool sse] [, float startHue] [, float endHue] [, float
maxSat] [, float minSat] [, float interp] [, bool dither])


http://avisynth.nl/index.php/Tweak


ColorYUV(clip [, float gain_y] [, float off_y] [, float gamma_y] [, float cont_y] [, float gain_u] [, float off_u] [, float gamma_u] [, float
cont_u] [, float gain_v] [, float off_v] [, float gamma_v] [, float cont_v] [, string levels] [, string opt] [, boolean showyuv] [, boolean
analyze] [, boolean autowhite] [, boolean autogain] [, boolean conditional])


http://avisynth.nl/index.php/ColorYUV

Although above ColorYUV parameters given as Float, that is only so that ColorYUV will not complain if you provide Float instead of Int,
Float parameters will accept Integer arguments, whereas if specified as Int parameters it would only accept Int and not float.
So for ColorYUV, args should be Integer values only but could be given as Floats. (eg 128 or 128.0 but not 128.5)

[Wilbert]

I think he asks about the output format of the filters. Every output of a filter is always in YUV [0,255] or RGB [0,255] (integer values for both, so it's the first scenario).

[StainlessS]

OOps, thank you Wilbert.

[feisty2]

uh, wanna avoid domain limit?
0-65535 should be enough to store 0-255 with some over/underflow?
use dither_lut16 on 8bpc clips wisely to do that dirty little work :)
here's an example,
let's separate x * 2 / 2 to x * 2 and x /2 and keep the result free from rounding errors

dither_convert_8_to_16 ()
dither_lut16 ("x 256 /")
dither_lut16 ("x 2 *")
dither_lut16 ("x 2 /")
dither_lut16 ("x 256 *")
ditherpost (mode=-1)


should be identical to

mt_lut ("x 2 * 2 /")


EDIT: example 2: how to deal with both overflow and underflow

dither_convert_8_to_16 ()
dither_lut16 ("x 256 / 128 - 32768 +")
#expressions here
dither_lut16 ("x 32768 - 128 + 256 *")
ditherpost (mode=-1)


EDIT2: you can always overcome domain issues if you can get some tools with domain large enough to work

[Stormborec]

Yes, I meant format, which gives one filter to the next in the chain ...


It follows that it is better to do much as possible in one step?
http://forum.doom9.org/showthread.php?p=1379661#post1379661

So better than coloryuv(cont_u=1...).mt_lutxy(.....) ..... just mt_lutxy(x 128 - 1... * 128 + .....)


I found, that it let me go up to yuv 255,255,255 - this is also retained if I do compresion to xvid avi using virtualdub.

BlankClip(pixel_type="yv12",color=$ffffff)
mt_lut("x 500 +",u=3,v=3)

coloryuv(analyze=true)

[feisty2]

You can do things in several steps, but you need a large domain tool to store under/overflow values
Like I wrote at #12

[Stormborec]

Yes it's clear. I try to do things as much simple as possible - then it's obviously fast and less rounded ..

[Stormborec]

Your advice in one script :) :

function x( clip last, float "ou", float "ov", float "su", float "sv", float "thr", float "str" )

{

ou = default(ou, 0.0)
ov = default(ov, 0.0)
su = default(su, 1.0)
sv = default(sv, 1.0)
thr = default(thr, 0.5)
str = default(str, 1.51)

y = converttoy8()
u = utoy8()
v = vtoy8()
y_scaled = y.bilinearresize(width(u), height(u))
y_adjusted = mt_lutxy(y,y.blur(thr),"x x y - abs 4 / 1 4 / ^ 4 * "+string(str)+" * x y - x y - abs 1.001 + / * +",U=2,V=2)
u_adjusted = mt_lutxy(u,y_scaled,expr="x 128 - "+string(su)+" * 128 + y 16 - "+string(ou)+" * + ")
v_adjusted = mt_lutxy(v,y_scaled,expr="x 128 - "+string(sv)+" * 128 + y 16 - "+string(ov)+" * + ")
ytouv(u_adjusted,v_adjusted,y_adjusted)

return last

}