mt_lut problem / How luma is calculated in YV12?

[zee944]

I have a script in which I make the pixels black below luma 40 and over luma 230 with mt_lut:

LoadPlugin("E:\mt_masktools[2.0a30].dll")

ImageReader("gradient-h.jpg")
ConvertToYV12()
unprocessed=last

mt_lut(last,expr="x 0 >= x 40 <= & 0 x ?",U=2,V=2)  # between luma 0 and 40, make it black
mt_lut(last,expr="x 230 >= 0 x ?",U=2,V=2)  # between luma  230 and 255, make it black
# mt_lut(last,expr="x 40 >= x 90 <= & x 40 - 90 40 - / 90 * x ?",U=2,V=2)
# mt_lut(last,expr="x 180 >= x 230 <= & 180 x 180 - 230 180 - / 180 * - x ?",U=2,V=2)

StackVertical(unprocessed,last)

When I check the result, the cut off point is at luma 29 and 248... instead of the desired 40 and 230. The original image contains only grey shades from [0, 0, 0] to [255, 255, 255]. In the resulting image the darkest pixel is [29, 29, 29], the brightest is [248, 248, 248] in RGB colorspace:



Why? Shouldn't be the cutoff point at [40, 40, 40] and [230, 230, 230]?

How can I fix it?

How luma is converted from RGB to YV12? How an [29, 29, 29] RGB pixel becomes [40, 40, 40] in YV12? Is there a formula for that?

[AVIL]

Hi,

Converttoyv12 makes an implicit range conversion to [16,235] range so your "unprocessed" clip is a processed one.

Use converttoyv12(matrix="PC.601") instead.

More info at:

http://avisynth.org/mediawiki/Convert

Good luck

[zee944]

I thought about that earlier, but I've ruled out. Yes, I do this conversion, but the original range remains. Even if I process the already "processed" image, the cut off points still should be at 40 and 230.

Originally Posted by AVIL

Hi,
Use converttoyv12(matrix="PC.601") instead.

Doesn't help. It just shrinks the greys, and adds extra black and extra white to the two sides of color transition, and the cut off points are still at 29 and 248.

[IanB]

And how exactly are you converting the YV12 back to RGB for viewing?

Unless you override it, all the default conversions do an implicit range conversion from [16,235] range.

You need a ConvertToRGB(matrix="PC.601") at the end of your script or the equivalent else where in your display chain.

[zee944]

You're right... with a ConvertToYV12(matrix="PC.601") at the beginning, and a ConvertToRGB(matrix="PC.601") at the end it works fine now.

I can't say I fully understand its logic, but it works... thanks Ian & AVIL

[IanB]

Given you seem to be working in [0..255] RGB, perhaps you would be better off using :-
RGBLUT (clip, string "Rexpr", string "Gexpr", string "Bexpr", string "AMPFile")

The conversion from RGB to YV12 and back in not lossless.

RGBLUT is in the older 1.5 Masktools.dll

[zee944]

Originally Posted by IanB

Given you seem to be working in [0..255] RGB, perhaps you would be better off using :-
RGBLUT (clip, string "Rexpr", string "Gexpr", string "Bexpr", string "AMPFile")

The conversion from RGB to YV12 and back in not lossless.

RGBLUT is in the older 1.5 Masktools.dll

Mmmm... how do I change the luma with RGBLUT? I have to do weighted R, G, B adjustment for that, or is there more simple way?

Perhaps it'd better stay in the YV12 colorspace all the way. Is there a command to easily adjust the Y, U, V channels? A YUVAdjust() would be the best instead of RGBAdjust()

[Gavino]

Originally Posted by zee944

Is there a command to easily adjust the Y, U, V channels? A YUVAdjust() would be the best instead of RGBAdjust()

ColorYUV

[IanB]

@zee944,

Perhaps you had better explain exactly what you are doing and want to achieve.

You do not seem to have a good grasp of signal scaling, i.e [0..255] vrs [16..235] etc

You cannot process luma, Y, as you are doing without some regard to chroma, UV. This may result in out of gamut combinations.

[zee944]

Originally Posted by IanB

You do not seem to have a good grasp of signal scaling, i.e [0..255] vrs [16..235] etc

True. Theoretically understand the point, in the reality I'm confused every time I meet with it. I always work with DVD movies, with [0...255] RGB32 space.

I thought 'Y' processes luma alone, and 'U' and 'V' sets the colour. Isn't luma independent from 'U' and 'V'?

My basic idea is to separate an image to luma layers, and adjust the colours for each layers seperatadly. Like when you set different colours in Photoshop or PSP for shadows, midtones, and highlights - only with more layers.

[IanB]

Originally Posted by zee944

I thought 'Y' processes luma alone, and 'U' and 'V' sets the colour. Isn't luma independent from 'U' and 'V'?

Yes. but there are gamut limits, i.e. when Y=1.0 (235) or 0.0 (16), U and V must be zero (128) conversely 100% saturated yellow has max Y of 0.886 (210) and saturated blue has a max Y of only 0.114 (41).

Have a wade thru Color conversions for some grounding.

[zee944]

Originally Posted by IanB

Have a wade thru Color conversions for some grounding.

Having a read I'd stay in RGB mode - it's very confusing thinking in YUV after RGB. It seems easier to discover RGBLUT instead.