Levels - Can't Generate True Blackness

[Malinka]

Hi everybody,

here the avisynth noobess goes again. This time I have a problem with the function Levels().

My working example:

levels(clip,235,1,255,0,255)

If I understand the documentation correctly, the line above should consider all pixels of the input that have a luma value <=235 as black. Then, due to the specified output values these pixels ought to be assigened a luma value of 0 (black).

I know that this is not a real life example but, I used it to understand what is going on 'inside' levels. What's bothering me now, is that I expected output frames that contained mostly black but what I got was something quite different (see attached image).

Now, since I cannot understand what is going on here, I would love to get some help from people than wiser than me.

Thank you very much in advance!

[Guest]

What is the color space of your clip? If you convert to RGB it should work as you expect.

Apparently the handling for YUY2 and YV12 is silly or broken as it is not clear that it works on the chroma planes and whether it enforces the YUV valid pixel ranges. E.g., if it is YUY2 or YV12, then the valid luma range is 16 (black) to 235 (white). If RGB it is 0( black) to 255 (white).

Maybe sh0dan will comment on this.

Finally, in your avatar, what is that white fuzzy thing obscuring your face? :)

[sh0dan]

I haven't looked at this for a while, but I seem to remember:

- In YUV mode, the values are scaled from 0-255 to 16-235.
- The color you are seeing is chroma information - in fact your image is probably black, if you use greyscale(). The image is only black, when chroma is 128 (grey).

[Guest]

But doesn't that mean it cannot be usefully applied to YUV formats?

[poptones]

No, it means if you want to use levels to create a mask, you need to use a pixel format suited to working with masks. YUY is not that format.

Levels was designed to return a valid result, so you can apply it at the end of any chain and know you're getting "legal" and correct values. If you want "true blackness" you gotta use a format that allows it. That means RGB or RGB32, and NOT a Y/U format.

[Guest]

It is not usable for the purpose described in the documentation, although it may be for the one you've stated! The documentation for Levels says "You should be able to take the numbers from VirtualDub's Levels dialog and pass them as parameters to the Levels filter and get the same results." That is manifestly not true for non-RGB. The document says nothing about enforcing legal pixel values.

So I agree with what you say about what Levels does. But the document is grossly misleading, as Malinka's experience shows.

[poptones]

"You should be able to take the numbers from VirtualDub's Levels dialog and pass them as parameters to the Levels filter and get the same results." That is manifestly not true for non-RGB.

When virtualdub works natively with yuv formats, and the behavior is different than avisynth, then you'll have a point.

Edit: I do think the color should (perhaps) not be included in the adjustment. Butif you pass bad values out of the yuv filter then you still end up with nonsense like the above.

[Malinka]

Regarding the color space, as you allready figured out, the image is a frame from a YV12 clip.

Regarding the white fuzzy thing, the picture was taken early in the morning after a party, might be the rest of the preceeding night searching its way out. Ah, no I remember, on the image I am actually hanging out, having a smoke. But the cigar didn't fit to avatar size :).

[sh0dan]

@neuron: When the number 255 is entered in the filter args, it is scaled down to 235 - 0 becomes 16, but that's actually the result is closer to RGB, than if they weren't scaled. The main problem is, that

There was a discussion on this a long time ago, but no-one came up with a better algorithm (if that's even possible without converting to RGB).

I would more say, that the settings in YUV mode should be taken from ffdshow.

[kilg0r3]

If levels is not working correctly in YUY formats, is there a workaround that allows to clip some low luma values to black in YV12 mode? I would not like to have to encode my movies in RGB mode. What about LumaFilter in MarcFD's Mpeg2Dec3?

@neuron
What's that white fuzzy thing conceiling Your face? :)

[sh0dan]

@kilg0r3:
IMO it is working correctly - What's your problem exactly?

[kilg0r3]

Probably, I expect it to do something which it is not supposed to do. Let's see.

Supose, I have got In a clip with near black areas (e.g. luma value of say 15), I would like to clip these areas to 0 without preferably affecting the luma value of the other pixels, i.e., all pixels with luma higher than 15 should be left untouched.

The purpose of all this, would be to increase compressability and to avoid artifacts in these near-to-black areas. If however, at luma 0, the chroma information is not 'neutralized', the codec, which is fed with this stream, might still produce some garbage, because there information which it strives to represent (kind of metaphysical explanation :)).

I hope to have made my point clear.

[Guest]

@poptones

My point is simply that the documentation for the Levels filter is misleading. It does not properly explain the purpose and operation of the filter and falsely leads one to believe that the VirtualDub results will always be achieved. This has caused difficulties for Malinka and probably others. It has nothing to do with whether or not VirtualDub supports planar formats.

[poptones]

Supose, I have got In a clip with near black areas (e.g. luma value of say 15), I would like to clip these areas to 0 without preferably affecting the luma value of the other pixels, i.e., all pixels with luma higher than 15 should be left untouched.

That's called "coring" and you can do it, but if you're using a yv format you will be sending illegal values to the codec which will (at best) simply ignore yoour changes and set those "bad" luminance values to a legal value before encoding. At worse it will blindly accept these levels, which will leave you with incredibly ugly black levels when the clip is watched on a decent monitor.

You can make a simple coring filter using clip.colorkeymask($08080808,07) and then layering the clip over an empty clip of blackness - but if you're using yv12 guess what? Most any filter that comes after this is going to set those levels right back to 16>235.

The purpose of all this, would be to increase compressability and to avoid artifacts in these near-to-black areas.

Setting the levels to absolute zero is guaranteed to add artifacts. Try it yourself and see: encode two versions, one with the coring filter as I described (you'll need to convert to RGB and pass it to veedub this way to avoid the levels clipping) and make another without the illegal levels and compare the PSNR with the stats filter. Hell, you probably won't even need to use stats - just compare them in a dark room on a good monitor.

The Levels filter behaves (in this aspect) as it should. I do think the color behavior needs to be fixed (I have the code here on my machine but I'm not set up for CVS right now) but the output level clipping, if changed to allow 0-255 output ranges in yuv will just end up sending illegal data to the other filters, which will surely lead to even more confusion.

So far as getting exact behaviors between the two formats, I still haven;t found a way. A couple of small changes can be made that will introduce a bit more error for nominal values while making these extremes more consistent, but that hardly seems logical. If you want exact RGB behavior, all it takes is converttorgb.levels() - problem solved.

If however, at luma 0, the chroma information is not 'neutralized', the codec, which is fed with this stream, might still produce some garbage, because there information which it strives to represent (kind of metaphysical explanation )

The alternative is it "throws away" that info in the quantizer, which means instead of getting smooth "near black" you get hideous 8x8 blocks of "black" and "near black." Not a big deal if you're trying to make the file sizes small (ie to share the rips on the 'net) but if this is for your own use why on earth would you want this behavior? Mix some noise in there and keep the quantizer "active" and those black levels smooth.

[kilg0r3]

Hum, hum. Seems I know even less about color spaces and stuff than I thought :eek:

I didn't even know that YUY2 and YV12 have a reduced luma range (16-235).

BUT, thank you very much for your detailed explanation.

[sh0dan]

@kilg0r3: You should have a look at Limiter() instead! :)

[kilg0r3]

yeah, yeah, got that :griningsheepishly: