Levels: same as in VirtualDub or not?

[stickboy]

The Levels documentation (http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/avisynth2/avisynth/docs/filters/levels.html) indicates that the Levels filter should be compatible with the one from VirtualDub, but I'm not getting the same results.

I made the following script:
ColorBars(640, 480)I loaded it in VirtualDub and applied VirtualDub's Levels filter with the following settings: 5, 1.0, 25, 0, 255, luma mode

Then I opened another instance of VirtualDub and loaded the following script:
ColorBars(640, 480) # ColorBars produces an RGB32 clip
ConvertToYUY2() # make sure levels operates on luma only
Levels(5, 1.0, 25, 0, 255)
ConvertToRGB32()The two cases above produce significantly different results.

In RGB mode, however, Levels seems to produce the same results in Avisynth as in VirtualDub for all practical purposes (not exact, but very, very close).

The docs also indicate that Levels should behave the same in RGB mode as in YUY2 mode if gamma == 1.0. However, the following script seems to demonstrate that not to be the case:
a=ColorBars(640, 480)
b=a.ConvertToYUY2().Levels(5, 1.0, 25, 0, 255).ConvertToRGB32()
c=a.Levels(5, 1.0, 25, 0, 255)
Subtract(b, c)The differences are significant enough that I don't think that they're the results of colorspace conversion artifacts, although I could be wrong...

(I'm using Avisynth 2.0.5 (2002-09-18 build from dividee (http://forum.doom9.org/showthread.php?s=&threadid=33742&pagenumber=2)) and VirtualDub 1.4.10.)

[WarpEnterprises]

AFAIK the conversion RGB->YUY2 itself implements a level transform for the luminance from 0...255 to 16...240

[Guest]

235 max for Y, 240 for U and V (just to be picky).

[stickboy]

Okay, therefore what should I be doing to go reliably from VirtualDub values to Avisynth ones in YUY2?

[stickboy]

Am I to assume, then, that if I want to use VirtualDub's Levels dialog to determine settings to use in Avisynth, I'm restricted to using RGB? Therefore, in my script, I'd need to convert to RGB, apply Levels, and then convert back to YUY2? Ick...

[FuPP]

Maybe this one could help (colorYUY2 is an external plugin)?

ColorYUY2(Levels="TV->PC")

I think that will enable you to get same basic levels than using RGB.

[WarpEnterprises]

No, this actually DOES a conversion.
What stickboy wants is a conversion of the settings only.

[stickboy]

I suppose this must just be a stupid question.

What does everyone else do? Do you all:

A. not use Levels
B. use the guess-and-check method for finding appropriate values
C. convert to RGB before using Levels (and similarly using RGB mode in the VirtualDub Levels dialog)
D. use something other than VirtualDub's Levels dialog for generating a histogram and for dynamic previews
E. ...?

(Obviously I don't have a solid grasp of how luminance works; could anyone point me to a good guide?)

[hakko504]

@stickyboy
Luminance is the B/W signal used by television. When color systems were constructed they only added two color signal to the B/W luminance signal. These are called Chrominance signals (UV or CbCr).

In a B/W TV, the luminance signal is directly proportional to the strength of the electron beam that lights up the screen. In new color TV's the luminance/chrominance signal must be converted to RGB before it is fed to the now three electron beams. All three beams are controlled in the same way, but since the electron gun for each color is placed a little bit off center, the three beams will hit different parts of each pixel, in a way that the red beam only hit points of the screen that glow red, etc.
The conversion between YUV and RGB are guided by the following equations: Y = 0.299 R + 0.587 G + 0.114 B U = 0.493 (B - Y) V = 0.877 (R - Y)

I found some of this information here (color formats) (http://www.angelfire.com/id/tagailog/avi.html) but I'm not convinced it's a good guide. You should be able to find better on the 'net.

Edit: I found this M$ link (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwmt/html/YUVFormats.asp) in the AviSynth FAQ (at www.avisynth.org (http://www.avisynth.org/index.php?page=AviSynthFaq))

[WarpEnterprises]

There are two histogram functions available in Avisynth:

Histogram() bulit-in, Y-channel only

Videoscope() e.g. here (http://home.concepts.nl/~hzon00008/bin/VScope11.zip) - more options


Does anybody (meaning neuron2 :) ) know how to get the "intensity" function from the Hue/Saturation/Intensity Filter for VirtualDub in AviSynth??

[stickboy]

WarpEnterprises:

Thanks, but I don't think either of those are what I'm looking for. :( Maybe I'm too used to VirtualDub's, but that's the type of histogram I'd prefer. (Summed values over the range of the clip.)