Is this what Levels() and coring do?

[fvisagie]

After reading the Levels documentation (http://avisynth.org/mediawiki/Levels) I'm still uncertain about a few things and I'd be grateful if someone could confirm my understanding, please.

When processing in YV12 mode, these are the points I'm not quite certain about (and how I understand them to work):


with coring=true, chroma is clamped to [16,240], scaled from [16,240] to [0,255], converted by the same formula as luma and scaled back to [16,240]
with coring=false, chroma is converted by the same formula as luma

Is that roughly correct?

Thanks,
Francois

[StainlessS]

If you download "Graffers&Toys.zip" in my sig, and try "Levels_Graf.exe", it will show effects of coring.
Allows visual inspection of effect in graph and on loaded BMP (converted to YUV, when in YUV mode).

EDIT: I've always thought Levels Luma coring worked oddly, (I think it's bugged & never use it (coring),
bugged or not, I dont like the way it works).
Works ok if ouput min=0 and output max=255 otherwise not.

Set I/P min to eg 40, I/P max to eg 220, coring = ticked,
limit=0, Grid=1, RGB unticked, mode=Y,

and play with the O/P min & max sliders to see odd coring effect.
The little bubbles (red circles) shows where the Luma O/P min and max would
be if coring were not selected.

Chroma coring seems to work ok.

EDIT: Perhaps this may assist, lut creator in VB, equivalent to CPP code.

Private Sub Levels_LUT()
Dim i As Integer, divisor As Double, p As Double, pp As Double, q As Double
Dim in_min As Integer, in_max As Integer, out_min As Integer, out_max As Integer
Dim gamma As Double
gamma = (vsb_Gam.Value + 256#) / 256#
If gamma < 0.004 Then gamma = 0.004 ' Approx 1/256, avoid / zero

in_min = hsb_IP_Min.Value
in_max = hsb_IP_Max.Value
out_min = vsb_OP_Min.Value
out_max = vsb_OP_Max.Value
divisor = in_max - in_min
If divisor = 0 Then divisor = 1 ' Avoid / zero
If chk_RGB.Value = vbChecked Then
For i = 0 To 255
p = (i - in_min) / divisor
If p < 0# Then p = 0#
If p > 1# Then p = 1#
p = p ^ (1# / gamma)
p = p * (out_max - out_min) + out_min
p = Round(p)
Debug.Print i; ")", p
If p < 0# Then p = 0#
If p > 255# Then p = 255#
Map(i) = p
Next i
Else
For i = 0 To 255
If chk_Coring.Value = vbChecked Then
p = ((i - 16) * (255# / 219#) - in_min)
Else
p = (i - in_min)
End If
p = p / divisor
If p < 0# Then p = 0#
If p > 1# Then p = 1#
p = p ^ (1# / gamma)
p = p * (out_max - out_min) + out_min

If chk_Coring.Value = vbChecked Then
pp = Round(p * (219# / 255#) + 16#)
If pp < 16# Then pp = 16#
If pp > 235# Then pp = 235#
Else
pp = Round(p)
If pp < 0# Then pp = 0#
If pp > 255# Then pp = 255#
End If
Map(i) = pp
q = Round((i - 128#) * (out_max - out_min) / divisor) + 128
If chk_Coring.Value = vbChecked Then
If q < 16# Then q = 16#
If q > 240# Then q = 240#
Else
If q < 0# Then q = 0#
If q > 255# Then q = 255#
End If
MapChroma(i) = q
Next i
End If
End Sub

[fvisagie]

Thanks, StainlessS. For this work (only normalising luma) I think I'll give Levels() a miss then - too much unwanted stuff thrown in for free.

[StainlessS]

fvisagie, I meant I never use coring, levels without coring=true is fine.
Coring seems to assume both input and output luma are range 16,235, I would prefer
clamping output only. Also, it is intention of original programmer that both input and output min & max,
can be outside the 0->255 range. (from Docs, "However, the input and output parameters can be larger than 255. ",
however fails to mention the less than zero possibility.)
I prefer to think of levels (without gamma & coring thing), as drawing a straight line between coords IPmin,OPmin and IPmax,OPmax
(saturated at those coords) and clamped 0->255, with whatever passes through the 0->255 graph being the result.

EDIT: Modifying 'Limit' in the graffer allows you to use inputs & outputs outside of the 0->255 range, just as you can
as arguments to the levels filter.

[fvisagie]

I prefer to think of levels (without gamma & coring thing), as drawing a straight line between coords IPmin,OPmin and IPmax,OPmax
and clamped 0->255, with whatever passes through the 0->255 graph being the result.

That's how I also have it. Problem is, it does it for chroma also (as I understand). In this case I'd prefer to explicitly address chroma in a separate step, not least because it needs somewhat different treatment than luma.

[StainlessS]

With Chroma, it rotates around the central point 128,128, would seems correct to me, and the slope
is maintained parallel with Luma slope. (Central point 128,128 represents 0,0)

[Gavino]

In this case I'd prefer to explicitly address chroma in a separate step, not least because it needs somewhat different treatment than luma.
In that case, you will probably find ColorYUV (http://avisynth.org/mediawiki/ColorYUV)() more useful than Levels().

[fvisagie]

It slipped my mind that much of this footage contains non-RGB YUV combinations as shown by jagabo's utility

Subtract(last, ConvertToRGB(last).ConvertToYUY2()).Levels(112,1,144,0,255)

to check for illegal YUV combinations. A flat medium grey image means there are no problems.

RGB conversion is unfortunately necessary and Levels does a fair job not only normalising luma but also YUV combinations.

I appreciate your inputs nonetheless. At least now I'm fairly clear on how Levels() works ;).