mt_lut / colorspace conversion problem

[zee944]

I had this source image:

http://i179.photobucket.com/albums/w286/zee944/td_layering_original.jpg

I've separated it to 5 luma layers. First picture holds the luma values between 0 and 51, second between 51 and 102 etc. The last, fifth between 204 and 255. Like this:

http://i179.photobucket.com/albums/w286/zee944/th_td_layering_split.jpg (http://s179.photobucket.com/albums/w286/zee944/?action=view&current=td_layering_split.jpg)

I thought adding them together again with Overlay(Last, <clip>, mode="add", opacity=1.0, pc_range=true) would give my source image back unharmed, but no... it looked like this:

http://i179.photobucket.com/albums/w286/zee944/td_layering_overlayadded.jpg

Eventually I've noticed what I've assumed to be black isn't really black. This is the fourth layer (holding luma values between 153 and 204):

http://i179.photobucket.com/albums/w286/zee944/td_layering_fourthlayer.jpg

This is how I made it (the fourth layer above):
ImageReader("TD_sample.jpg").Crop(0,0,0,-150)
ConvertToYV12(matrix="PC.601")

LumaZeroFrom=0
LumaZeroTo=153.6
mt_lut(last,expr="x "+String(LumaZeroFrom)+" > x "+String(LumaZeroTo)+" <= & 0 x ?",U=2,V=2) # between luma LumaZeroFrom and LumaZeroTo, make all pixels black
LumaZeroFrom=204.8
LumaZeroTo=255
mt_lut(last,expr="x "+String(LumaZeroFrom)+" > x "+String(LumaZeroTo)+" <= & 0 x ?",U=2,V=2) # between luma LumaZeroFrom and LumaZeroTo, make all pixels black
ConvertToRGB(matrix="PC.601")

After pushing up gamma (too see the "black" parts aren't black):

http://i179.photobucket.com/albums/w286/zee944/td_layering_fourthlayer_gammaadded.jpg

I assume these non-black parts makes the end result overbright and oversatured, so totally off.

What can I do to make it right? It probably has something to do with the colorspace conversions I do in the script. But I need RGB. I was thinking about using RGBLut() instead of mt_lut, but I've no idea how to separate the luma layers with that. I know how to calculate the luma value from the R,G,B channels, but don't know how to handle them together to make the same luma separations above.

Please help. :confused:

[scharfis_brain]

R=last.converttorgb(matrix="PC.601").showred().converttoyv12(matrix="PC.601")
G=last.converttorgb(matrix="PC.601").showgreen().converttoyv12(matrix="PC.601")
B=last.converttorgb(matrix="PC.601").showblue().converttoyv12(matrix="PC.601")

R1=R.yourfilter()
G1=G.yourfilter()
B1=B.yourfilter()

sum=MergeRGB(R1.ConverttoRGB(matrix="PC.601"), G1.ConverttoRGB(matrix="PC.601"), B1.ConverttoRGB(matrix="PC.601")).ConverttoYV12(matrix="PC.601")

[zee944]

R=last.converttorgb(matrix="PC.601").showred().converttoyv12(matrix="PC.601")
G=last.converttorgb(matrix="PC.601").showgreen().converttoyv12(matrix="PC.601")
B=last.converttorgb(matrix="PC.601").showblue().converttoyv12(matrix="PC.601")
[...]

I don't understand, what I could use this code for? I need to separate certain luma ranges in RGB, not the RGB channels themselves. :confused:

[scharfis_brain]

you selected luma values but ignored chroma completely.

so either filter on the RGB planes (yourfilter() marks the place where your filter process will be placed)
or you filter it just you to but you apply
mergeluma() to your result, so you preserve the chroma from the input and return the processed luma only

[Didée]

At least to me, "I need to separate certain luma ranges in RGB" is not fully clear.

Anyhow, first step is to greyscale() all 5 layers. Either that's already what you want. Or (if you want to "punch out" on the RGB image) you then use those layers 5 times to mask() the RGB input, and layer pure black onto the masked RGBs.

Your current code has unaltered original chroma in each layer, and Overlay(add) happily sums those 5 up, resulting in saturation increased x 5.

[zee944]

Eeerr.... sorry if my wording is poor.

I don't want to touch the luma levels directly. I want to set the color balance (with not much effect on luma). So my intention was to keep the colours on each luma ranges I separated, and process them with RGBAdjust() seperatedly. This image should show the basic idea of the separation:

http://i179.photobucket.com/albums/w286/zee944/td_layering_split.jpg

It's the same thing when you adjust the color balance of the highlights, midtones, and shadows in PhotoShop, only in AviSynth and with more luma layers than 3.

The method should work by my logic, probably the colorspace conversion (back and forth) ruins it, if not something else I don't think of. On the 3., 4. and 5. layer the actors's face should be totally black! But it's not. Isn't that what leads to the bad result?

Perhaps I could do with 'RGBlut' what I do know with 'mt_lut' to avoid the conversion, but I don't know how... yet.

[scharfis_brain]

On the 3., 4. and 5. layer the actors's face should be totally black! But it's not. Isn't that what leads to the bad result?
That is why you should greyscale the image or process R, G, and B separately!
If the UV Chroma planes are left untouched you'll have no 'real' black even if Luma (Y) is set to black.
That all is YUV magic...

[zee944]

Oh, I see now. It's not because of the color conversion, it's because of the U and V channels. And I understand your first idea now, thanks, already done a script with 5 layers with that, but... although it's similar it's not exactly what I'm looking for. I'd prefer to separate the layers by luma and not by the R, G, B amount.

Is the greyscale method works the way I want? I don't understand how to do that yet. How I make a layer?


# making the fourth layer
LumaZeroFrom=0
LumaZeroTo=153.6
mt_lut(last,expr="x "+String(LumaZeroFrom)+" > x "+String(LumaZeroTo)+" <= & 0 x ?",U=2,V=2) # between luma 0 and 153.6 make all pixels black
LumaZeroFrom=204.8
LumaZeroTo=255
mt_lut(last,expr="x "+String(LumaZeroFrom)+" > x "+String(LumaZeroTo)+" <= & 0 x ?",U=2,V=2) # between luma 204.8 and 255 make all pixels black
GreyScale()
ConvertToRGB(matrix="PC.601")


...and from here? :confused:

[Gavino]

To make the layers the way I think you originally intended them to be, change your method to construct, for each luma range, a binary mask (ie each pixel either 0 or 255) which selects the appropriate pixels of your image. Then merge the original clip (both luma and chroma) with a pure black one using each of these masks to create your layers.

[zee944]

GreyScale()
LumaMaxFrom=180
LumaMaxTo=255
RGBlut(last,Rexpr="x "+String(LumaMaxFrom)+" > x "+String(LumaMaxTo)+" <= & 255 0 ?")
RGBlut(last,Gexpr="x "+String(LumaMaxFrom)+" > x "+String(LumaMaxTo)+" <= & 255 0 ?")
RGBlut(last,Bexpr="x "+String(LumaMaxFrom)+" > x "+String(LumaMaxTo)+" <= & 255 0 ?")
layermask=last


Is it a good binary mask? It makes 255 every pixels over luma 180, and zero every other pixels.

Then merge the original clip (both luma and chroma) with a pure black one using each of these masks to create your layers.

How can I use the mask? Merge(unprocessed,BlankClip(last))...?

And... if there wouldn't be only 0 and 255 in the mask, would the in-between values work as fade in-fade out?

[Gavino]

What I had in mind was using mt_lut (not RGBlut) as you had originally, but setting the output to either 0 or 255 (instead of 0 or x).

Then use mt_merge(BlankClip(src, coloryuv=$008080), src, mask, luma=true, U=3, V=3)
(where src is the original converted to YV12) to get your 'layer'.

The idea is that each layer produced in this way will have the original chroma for non-black pixels, but be properly black where necessary (where the luma was out of range for the layer in question).

[zee944]

Then use mt_merge(BlankClip(src, coloryuv=$008080), src, mask, luma=true, U=3, V=3)
(where src is the original converted to YV12) to get your 'layer'.

Hey... thanks Gavino, it works! :D

But if I'm not mistaken, this method causes problems with pixels which are 255 already. Here's how I do a layer at the moment:

LumaMaxFrom=153.6
LumaMaxTo=204.8
mt_lut(last,expr="x "+String(LumaMaxFrom)+" > x "+String(LumaMaxTo)+" <= & 255 0 ?",U=2,V=2)
mt_merge(BlankClip(unprocessed.ConvertToYV12(matrix="PC.601"),color_yuv=$008080), unprocessed.ConvertToYV12(matrix="PC.601"), last, luma=true, U=3, V=3)
layer4=last


It takes the current (last) clip and makes pixels "255" between luma 153.6 and 204.8. So it checks and modifies the current clip. Is it possible to put the modifications into another clip? That'd hold surely only needed pixels, and no random "255" pixels from the unprocessed clip. It must be possible with MaskTools, but I can't find the appropriate function; mt_lutxy, mt_lutf, mt_luts doesn't seem to be designed for that.

[Gavino]

When creating the mask for each layer with mt_lut, instead of using 'last' (which as you say changes each time), use the original (YV12) source.
orig = unprocessed.ConvertToYV12(matrix="PC.601")
black = BlankClip(orig, color_yuv=$008080)
# for each layer:
LumaMaxFrom=...
LumaMaxTo=...
mt_lut(orig, expr="x "+String(LumaMaxFrom)+" > x "+String(LumaMaxTo)+" <= & 255 0 ?",U=2,V=2)
mt_merge(black, orig, last, luma=true, U=3, V=3)
layerN=last

[zee944]

When creating the mask for each layer with mt_lut, instead of using 'last' (which as you say changes each time), use the original (YV12) source.

My wording was poor... I have no problem to do the layers separately.

What I'd like is to read the input clip and then execute the statement in another clip. If the current pixel is between luma 153 and 204 in clip1, then put '255' to clip2 accordingly.

(And clip2 would start as black, blank clip.)

[Gavino]

Not sure I fully understand what you're trying to do, but let's say you want to set pixels of clip2 to 255 where the corresponding pixels of clip1 are in a certain range. What's more, you want to retain the other pixels of clip2 at each stage rather than simply overwriting it.

Then you could do:
mask = mt_lut(clip1, expr="...") # as seen earlier
clip2 = mt_logic(clip2, mask, mode="or")

[Didée]

@Gavino - correct idea, but not sure if it actually works with "or" in mt_logic. IIRC the doc says that mt_logic with "and" or "or" operation assumes both clips to be 0|255 binary. For all other values the behaviours is undefined.

Since the 2nd clip is 0|255 only, its safe (and correct) to use the "max" mode:

mask = mt_lut(clip1, expr="...") # as seen earlier
clip2 = mt_logic(clip2, mask, mode="max")

[Gavino]

@Gavino - correct idea, but not sure if it actually works with "or" in mt_logic. IIRC the doc says that mt_logic with "and" or "or" operation assumes both clips to be 0|255 binary. For all other values the behaviours is undefined.
The current doc (from MaskTools 2.5) gives examples using pixel values of 11 and 5, so perhaps that was an earlier restriction that no longer applies. But your method is probably clearer anyway.

[zee944]

Thank you Gavino & Didée, it works fine. :D

One more question: if I have one (normal) clip and another (greyscale) clip. The latter holds the values of the desired luma values for the first clip. Both are RGB, how can I merge them?

For instance:
In the first clip a certain pixel is 127, 57, 98 [R, G, B], in the other clip the corresponding pixel is 65, 65, 65 [R, G, B].

If my calculation is correct, 127, 57, 98 has a luma value of approx. 74 or 75. How can it be the desired 65? Mixing these two must be possible with some Overlay or MergeRGB trick, I'm playing around them currently but I have no solution yet.

(It would be even better if I wouldn't need the second pre-made greyscale clip and I could modify luma for certain luma ranges directly within RGBLut. I'd like to avoid colorspace conversion.)

[Gavino]

I have one (normal) clip and another (greyscale) clip. The latter holds the values of the desired luma values for the first clip. Both are RGB, how can I merge them?
If what you want is to create a clip which preserves the color (ie chroma) of the first clip but gives it the luma of the second, convert both to YUV and use MergeLuma(clip1, clip2).
I'd like to avoid colorspace conversion.
I think a form of colorspace conversion is inevitable at some stage since you are trying to apply YUV concepts to RGB.

[zee944]

I'm about to go nuts :angry:

I've tried every single thing we've discussed here; and I'm still fighting against the problem in my first post. If I try to add some fade out / fade in trick to the "layers", when I want to put together the image again, it becomes overbright and oversaturated.

It doesn't matter if I use mt_lut or RGBlut; the result is the same. Even when I avoid the YUV colorspace (although the documentation says Overlay() internally converts to YUV) by replacing MergeLuma() to Overlay(), I've no luck.

This is my current script. It takes apart the image to two layers (pieces) with a fade in-fade out trick, then adds them together again.


ImageReader("TD_sample_03_MET.jpg")
ConvertToRGB32()
unprocessed=Last

unprocessed
GreyScale()
fullmask = BlankClip(last)
FadeInFrom=128
FadeInTo=255
#expr="x "+String(FadeInFrom)+" >= x "+String(FadeInTo)+" <= & x "+String(FadeInFrom)+" - "+String(FadeInTo)+" "+String(FadeInFrom)+" - / 255 * 0 ?" # brightest pixel here is 255 (not x)
expr="x "+String(FadeInFrom)+" >= x "+String(FadeInTo)+" <= & x "+String(FadeInFrom)+" - "+String(FadeInTo)+" "+String(FadeInFrom)+" - / x * 0 ?" # brightest pixel here is as bright as it was originally (not 255)
addtomask=RGBlut(last,Rexpr=expr,Gexpr=expr,Bexpr=expr)
fullmask=Overlay(fullmask, addtomask, mode="add", opacity=1.0, pc_range=true)
MergeLuma(unprocessed.ConvertToYV12(), fullmask.ConvertToYV12()) #...or Overlay(fullmask,unprocessed,mode="chroma")
ConvertToRGB32()
piece1=last

unprocessed
GreyScale()
fullmask = BlankClip(last)
FadeOutFrom=128
FadeOutTo=255
#expr="x "+String(FadeOutFrom)+" >= x "+String(FadeOutTo)+" <= & 255 x "+String(FadeOutFrom)+" - "+String(FadeOutTo)+" "+String(FadeOutFrom)+" - / 255 * - x ?" # brightest pixel here is 255 (not x)
expr="x "+String(FadeOutFrom)+" >= x "+String(FadeOutTo)+" <= & x x "+String(FadeOutFrom)+" - "+String(FadeOutTo)+" "+String(FadeOutFrom)+" - / x * - x ?" # brightest pixel here is as bright as it was originally (not 255)
addtomask=RGBlut(last,Rexpr=expr,Gexpr=expr,Bexpr=expr)
fullmask=Overlay(fullmask, addtomask, mode="add", opacity=1.0, pc_range=true)
MergeLuma(unprocessed.ConvertToYV12(), fullmask.ConvertToYV12()) #...or Overlay(fullmask,unprocessed,mode="chroma")
ConvertToRGB32()
piece2=last

# put together the image again:
BlankClip(unprocessed)
Overlay(piece1,opacity=1.0,mode="add",pc_range=true)
Overlay(piece2,opacity=1.0,mode="add",pc_range=true)
processed=last

StackVertical(unprocessed,processed,piece1,piece2)



And using the mt_merge() trick, this is the closest I can get. It doesn't become oversaturated, which is fine, but something is still uneven, probably because some colorspace conversion error. It's obvious when I use greyscale color bars for testing.


ImageReader("TD_sample_03_MET.jpg")
ConvertToRGB32(matrix="PC.601")
unprocessed=Last

unprocessed
GreyScale()
fullmask = BlankClip(last)
FadeInFrom=128
FadeInTo=255
expr="x "+String(FadeInFrom)+" >= x "+String(FadeInTo)+" <= & x "+String(FadeInFrom)+" - "+String(FadeInTo)+" "+String(FadeInFrom)+" - / 255 * 0 ?" # brightest pixel here is 255 (not x)
#expr="x "+String(FadeInFrom)+" >= x "+String(FadeInTo)+" <= & x "+String(FadeInFrom)+" - "+String(FadeInTo)+" "+String(FadeInFrom)+" - / x * 0 ?" # brightest pixel here is as bright as it was originally (not 255)
addtomask=RGBlut(last,Rexpr=expr,Gexpr=expr,Bexpr=expr)
fullmask=Overlay(fullmask, addtomask, mode="add", opacity=1.0, pc_range=true)
fullmask
mt_merge(BlankClip(unprocessed.ConvertToYV12(), color_yuv=$008080), unprocessed.ConvertToYV12(), fullmask.ConvertToYV12(), luma=true, U=3, V=3)
ConvertToRGB32()
piece1=last

unprocessed
GreyScale()
fullmask = BlankClip(last)
FadeOutFrom=128
FadeOutTo=255
expr="x "+String(FadeOutFrom)+" >= x "+String(FadeOutTo)+" <= & 255 x "+String(FadeOutFrom)+" - "+String(FadeOutTo)+" "+String(FadeOutFrom)+" - / 255 * - 255 ?" # brightest pixel here is 255 (not x)
#expr="x "+String(FadeOutFrom)+" >= x "+String(FadeOutTo)+" <= & x x "+String(FadeOutFrom)+" - "+String(FadeOutTo)+" "+String(FadeOutFrom)+" - / x * - x ?" # brightest pixel here is as bright as it was originally (not 255)
addtomask=RGBlut(last,Rexpr=expr,Gexpr=expr,Bexpr=expr)
fullmask=Overlay(fullmask, addtomask, mode="add", opacity=1.0, pc_range=true)
fullmask
mt_merge(BlankClip(unprocessed.ConvertToYV12(), color_yuv=$008080), unprocessed.ConvertToYV12(), fullmask.ConvertToYV12(), luma=true, U=3, V=3)
ConvertToRGB32()
piece2=last

# putting together the image again:
BlankClip(unprocessed)
ConvertToYV12()
Overlay(piece1,opacity=1.0,mode="add",pc_range=true)
Overlay(piece2,opacity=1.0,mode="add",pc_range=true)
ConvertToRGB32()
processed=last

StackVertical(unprocessed,processed,piece1,piece2)

Result with the first script (oversaturating):

http://i179.photobucket.com/albums/w286/zee944/mt_lut_first_script.jpg

Result with the second script using mt_merge and greyscale bars:

http://i179.photobucket.com/albums/w286/zee944/mt_lut_second_script.jpg

Should I give up? Will it ever be sorted out? :confused:

EDIT: It seems I've fixed the last script. Thanks for everybody.