Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 Doom9's Forum mt_lut problem / How luma is calculated in YV12?
 Register FAQ Calendar Search Today's Posts Mark Forums Read

 1st February 2009, 18:14 #1  |  Link zee944 Registered User   Join Date: Apr 2007 Posts: 240 mt_lut problem / How luma is calculated in YV12? I have a script in which I make the pixels black below luma 40 and over luma 230 with mt_lut: Code: ```LoadPlugin("E:\mt_masktools[2.0a30].dll") ImageReader("gradient-h.jpg") ConvertToYV12() unprocessed=last mt_lut(last,expr="x 0 >= x 40 <= & 0 x ?",U=2,V=2) # between luma 0 and 40, make it black mt_lut(last,expr="x 230 >= 0 x ?",U=2,V=2) # between luma 230 and 255, make it black # mt_lut(last,expr="x 40 >= x 90 <= & x 40 - 90 40 - / 90 * x ?",U=2,V=2) # mt_lut(last,expr="x 180 >= x 230 <= & 180 x 180 - 230 180 - / 180 * - x ?",U=2,V=2) StackVertical(unprocessed,last)``` When I check the result, the cut off point is at luma 29 and 248... instead of the desired 40 and 230. The original image contains only grey shades from [0, 0, 0] to [255, 255, 255]. In the resulting image the darkest pixel is [29, 29, 29], the brightest is [248, 248, 248] in RGB colorspace: Why? Shouldn't be the cutoff point at [40, 40, 40] and [230, 230, 230]? How can I fix it? How luma is converted from RGB to YV12? How an [29, 29, 29] RGB pixel becomes [40, 40, 40] in YV12? Is there a formula for that?
 1st February 2009, 19:01 #2  |  Link AVIL Registered User   Join Date: Nov 2004 Location: Spain Posts: 400 Hi, Converttoyv12 makes an implicit range conversion to [16,235] range so your "unprocessed" clip is a processed one. Use converttoyv12(matrix="PC.601") instead. More info at: http://avisynth.org/mediawiki/Convert Good luck
1st February 2009, 19:58   #3  |  Link
zee944
Registered User

Join Date: Apr 2007
Posts: 240
I thought about that earlier, but I've ruled out. Yes, I do this conversion, but the original range remains. Even if I process the already "processed" image, the cut off points still should be at 40 and 230.

Quote:
 Originally Posted by AVIL Hi, Use converttoyv12(matrix="PC.601") instead.
Doesn't help. It just shrinks the greys, and adds extra black and extra white to the two sides of color transition, and the cut off points are still at 29 and 248.

 1st February 2009, 21:47 #4  |  Link IanB Avisynth Developer   Join Date: Jan 2003 Location: Melbourne, Australia Posts: 3,168 And how exactly are you converting the YV12 back to RGB for viewing? Unless you override it, all the default conversions do an implicit range conversion from [16,235] range. You need a ConvertToRGB(matrix="PC.601") at the end of your script or the equivalent else where in your display chain.
 2nd February 2009, 18:23 #5  |  Link zee944 Registered User   Join Date: Apr 2007 Posts: 240 You're right... with a ConvertToYV12(matrix="PC.601") at the beginning, and a ConvertToRGB(matrix="PC.601") at the end it works fine now. I can't say I fully understand its logic, but it works... thanks Ian & AVIL
 2nd February 2009, 21:50 #6  |  Link IanB Avisynth Developer   Join Date: Jan 2003 Location: Melbourne, Australia Posts: 3,168 Given you seem to be working in [0..255] RGB, perhaps you would be better off using :- RGBLUT (clip, string "Rexpr", string "Gexpr", string "Bexpr", string "AMPFile") The conversion from RGB to YV12 and back in not lossless. RGBLUT is in the older 1.5 Masktools.dll
4th February 2009, 13:02   #7  |  Link
zee944
Registered User

Join Date: Apr 2007
Posts: 240
Quote:
 Originally Posted by IanB Given you seem to be working in [0..255] RGB, perhaps you would be better off using :- RGBLUT (clip, string "Rexpr", string "Gexpr", string "Bexpr", string "AMPFile") The conversion from RGB to YV12 and back in not lossless. RGBLUT is in the older 1.5 Masktools.dll
Mmmm... how do I change the luma with RGBLUT? I have to do weighted R, G, B adjustment for that, or is there more simple way?

Perhaps it'd better stay in the YV12 colorspace all the way. Is there a command to easily adjust the Y, U, V channels? A YUVAdjust() would be the best instead of RGBAdjust()

4th February 2009, 13:44   #8  |  Link
Gavino
Avisynth language lover

Join Date: Dec 2007
Location: Spain
Posts: 3,373
Quote:
 Originally Posted by zee944 Is there a command to easily adjust the Y, U, V channels? A YUVAdjust() would be the best instead of RGBAdjust()
ColorYUV

 4th February 2009, 13:51 #9  |  Link IanB Avisynth Developer   Join Date: Jan 2003 Location: Melbourne, Australia Posts: 3,168 @zee944, Perhaps you had better explain exactly what you are doing and want to achieve. You do not seem to have a good grasp of signal scaling, i.e [0..255] vrs [16..235] etc You cannot process luma, Y, as you are doing without some regard to chroma, UV. This may result in out of gamut combinations.
4th February 2009, 16:08   #10  |  Link
zee944
Registered User

Join Date: Apr 2007
Posts: 240
Quote:
 Originally Posted by IanB You do not seem to have a good grasp of signal scaling, i.e [0..255] vrs [16..235] etc
True. Theoretically understand the point, in the reality I'm confused every time I meet with it. I always work with DVD movies, with [0...255] RGB32 space.

I thought 'Y' processes luma alone, and 'U' and 'V' sets the colour. Isn't luma independent from 'U' and 'V'?

My basic idea is to separate an image to luma layers, and adjust the colours for each layers seperatadly. Like when you set different colours in Photoshop or PSP for shadows, midtones, and highlights - only with more layers.

4th February 2009, 22:22   #11  |  Link
IanB
Avisynth Developer

Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,168
Quote:
 Originally Posted by zee944 I thought 'Y' processes luma alone, and 'U' and 'V' sets the colour. Isn't luma independent from 'U' and 'V'?
Yes. but there are gamut limits, i.e. when Y=1.0 (235) or 0.0 (16), U and V must be zero (128) conversely 100% saturated yellow has max Y of 0.886 (210) and saturated blue has a max Y of only 0.114 (41).

Have a wade thru Color conversions for some grounding.

6th February 2009, 20:30   #12  |  Link
zee944
Registered User

Join Date: Apr 2007
Posts: 240
Quote:
 Originally Posted by IanB Have a wade thru Color conversions for some grounding.
Having a read I'd stay in RGB mode - it's very confusing thinking in YUV after RGB. It seems easier to discover RGBLUT instead.