Should I compress luminance range?

[zambelli]

I notice that nearly every DV camera I work with produces a luminance range that includes values below 16 and above 235. It's always been my understanding that these are non-compliant luminance ranges and that DV cams should not be producing anything outside the "TV scale". Is this correct?

So on to my next question: what's the best way to deal with such DV footage in post production? Should I compress the luminance range to a TV scale (for Avisynth users: ColorYUV(levels="PC->TV")) before proceeding on to other processing? Or should I just leave the footage as is and let each application deal with it in its own way?

[stickboy]

How are you determining that the values are outside of the luminance range? (That is, are you sure that the decoder isn't doing some YUV->RGB conversion that's at fault?)

[zambelli]

I've determined that with AviSynth using both AviSource (MainConcept decoder) and DirectShowSource (MS decoder). I used "ColorYUV(analyze=true)" and watched the Y values go above 235 and below 16.

[Wilbert]

Let me first say that I'm not entirely sure of the correct answer to your questions. But I see it in the following way:


I notice that nearly every DV camera I work with produces a luminance range that includes values below 16 and above 235. It's always been my understanding that these are non-compliant luminance ranges and that DV cams should not be producing anything outside the "TV scale". Is this correct?
I think the answer is no here. YUV values outside [16,235] are illegal for broadcast, but are perfectly acceptable in the camera.

Reading http://www.adamwilt.com/DV-FAQ-editing.html#NLE, I still think it's best to do a YUV [0,255] -> RGB [0,255] (or a YUV [0,255] -> YUV [16,235] scaling if you want to stay in YUV space). Thus


AviSource("dv.avi") # request YUY2 if necessary
ColorYUV(levels="PC->TV")
ConvertToRGB # if necessary


or a little bit better IF you convert to RGB:


AviSource("dv.avi") # request YUY2 if necessary
# YUV [0,255] -> RGB [0,255] using Rec.601 coefficients
ConvertToRGB(matrix="PC.601")


You can also do the following. Look up some frames with are very dark, and compare the following clips:

c1 = AviSource("dv.avi") # request YUY2 if necessary

c2 = AviSource("dv.avi") # request YUY2 if necessary
c2 = c2.ColorYUV(levels="PC->TV")

c1 will be clipped to [16,235] and c2 scaled to [16,235]. If c1 is too dark, and c2 acceptable then you should use c2.

[zambelli]

Wilbert,
Thanks for the well thought out reply. The first method is the one I've been using recently and it's good to know that I haven't been doing the wrong thing. :)

I also verified that the Avia DVD uses a Y range of about 13..233 (close enough, I guess) so 16..235 is definitely the range to shoot for.

[SomeJoe]

A lot of DV cameras will produce individual values that are outside the ITU.601 16-235 range because of noise errors in the CCD pickup, improper white balance, etc. even though they're actually complying with the standard.

Reading individual pixel values or looking at minimum/maximum pixel values won't give you a true representation of the way the camera is encoding the video. The only good way to see is to use a digital YC waveform viewer like is included in Premiere Pro and see that the nominal minimum pixel values are at 7.5 IRE (US NTSC) or 0 IRE (Japan NTSC or PAL) and that the nominal maximum pixel values are around 100 IRE.

[nicco]

The only good way to see is to use a digital YC waveform viewer like is included in Premiere Pro and see that the nominal minimum pixel values are at 7.5 IRE (US NTSC) or 0 IRE (Japan NTSC or PAL) and that the nominal maximum pixel values are around 100 IRE.

can you post a shot?

[SomeJoe]

http://pages.sbcglobal.net/wilsondr/IRE.gif


If you set the Premiere Pro monitor window to show the YC waveform, this is what you get. As you scrub the timeline, you'll see the YC waveform change with each frame of video.

This frame has both white and black areas, and you can see that the white areas are near 100 IRE (Premiere Pro arrives at that figure from approximate luminance values of 235), whereas the black areas are near 7.5 IRE (Premiere Pro arrives at that figure from approximate luminance values of 16). The blacks should not be (and aren't, in this graph) crushed, where you get a flat line at 7.5 IRE. Whites shouldn't be crushed either, although this is generally less of a problem.

Frequently, if I get a piece of video that looks "washed out" or dull/gray in hue, I'll scrub it while watching the waveform monitor and see that blacks are coming in at 15 IRE or that whites are peaking at only 80 IRE. (I've actually pulled poor video like this off of released Hollywood DVDs. :rolleyes: ) I will normally run the video through AVISynth and use Tweak() until I get the IRE values correct. Nails the video to perfect luminance balance every time.

The only problem is that I have to repeatedly encode small sections of video and then load them into Premiere Pro to look at them on the waveform monitor. Since the adjustments to Tweak() are trial-and-error, it can take a while to get the adjustment correct.

I've done a fair bit of C programming, and I halfway want to undertake creating a waveform/vectorscope AVISynth filter, but I don't know enough about AVISynth's APIs to do it. Maybe one day when I have more time I'll start reading in the AVISynth developers forum and try to do it.

Well, I searched the AVISynth forum and there's already a couple AVISynth and VDub waveform & vectorscope plugins done. One is here (http://forum.doom9.org/showthread.php?s=&threadid=72634&highlight=vectorscope), another is here (http://forum.doom9.org/showthread.php?s=&threadid=73552&highlight=waveform+OR+vectorscope). :) :D

[nicco]

Thank yo very much for your reply!
I work in Premiere with PAL-DV sources, so my range should be between 0(black) and 100 (bright).
Right?

[neily]

I have been wrestling with this problem for some time. There seem to be 5 main variables. Firstly is how the camera encodes luminance to a DV AVI value. Secondly is how the PC codec decodes the DV AVI value to either YUV or RGB colourspace. The third is how the encoder for what ever the output format is (DivX, MPEG2 etc) handles the conversion. The forth is how the decoding device, i.e. DVD player or DirectShow filter, decodes the transcode. The fifth is how the output device displays the player signal.

The chain I am mostly interested in is DV camcorder to MPEG2 via CCE to DVD.

The only aspects one has much control over are the choide of DV codec and transcoder settings. The transcoder being CCE, in YUV colour space you have little choice at all as the 0-255 and 16-235 settings produce identical output. In RGB colour space you have a choice between mapping to 0-255 or 16-235, 16-235 giving pretty identical results to YUV.

This means that the only choice comes down to the choice of codec and post-processing between DV decoding and CCE input.

DV codecs fall into 2 main categories, those that output in YUV colour space and those that output to RGB. I use the Matrox DV codec, which outputs to RGB, but has the choice of mapping the DV (YUV) values to either 0-255, 16-255, or 16-235. It's a while since I did my own DV codec tests, but the output does vary widely, and you really need to know the colourspace and mapping they employ.

As mentioned in the DV FAQ, I think that my Sony DV camcorder does DIFFERENTIATE in the luminance range of 16-255. Certainly, using the Matrox codec setting of 0-255, gives an output with a smooth histogram of ~16-255. Using a setting of 16-235 seems to compress or core the high values. Thus to best preserve the luminance range I have settled on using the 16-255 codec mapping.

The problem with this is that this causes a shift in average luminance. Using 16-235 as the base, over an average 15min piece of tape, I am getting 0-255 giving about 3 Y brighter, and 16-255 giving 6 Y darker, using AVISynth as the analysis tool.

Thus when using the DV 16-255 mapping to prevent coring and maximise luminance range, I am also having to use AVISynth ColorYUV to change the gain, offset, and gamma to rectify the overall luminance. I started off using ColorYUV to map the target output to 16-235, but having done a few conversions, 235 just seems too bright. Comparing my encoded DVD's to TV broadcasts and retail DVD's, the whites were just too bright and glaring. I have now settled on trying to map the DV RGB 16-255 output to YUV 18-225, and am using a setting of ColorYUV(gain_y=-12,off_y=-6,gamma_y=64).

As much as I hate playing around with the original data too much, this seems to give the best result for me.

So in answer to your original question, I wouldn't blindly use ColorYUV(levels="PC->TV"), unless you know how the DV codec is handling your source.

I wonder how professionals handle this problem. DV footage is not uncomonly seen on TV. I guess they take a "professional" attitude of just do it so it looks OK, then get on with the next project, rather than worrying about it too much.

[trevlac]

Originally posted by nicco
Thank yo very much for your reply!
I work in Premiere with PAL-DV sources, so my range should be between 0(black) and 100 (bright).
Right?

My understanding is that PAL vs NTSC does not really matter. For DV, 16=black and 235=white.

The trick in the above posted Waveform Monitor seems to be that the 7.5 Setup box is checked. Hence the app equates 16 to IRE 7.5.

[SomeJoe]

Originally posted by trevlac
My understanding is that PAL vs NTSC does not really matter. For DV, 16=black and 235=white.

The trick in the above posted Waveform Monitor seems to be that the 7.5 Setup box is checked. Hence the app equates 16 to IRE 7.5.

That is correct ... if the 7.5 IRE checkbox is checked, Premiere Pro will equate luminance values of 16 = black = 7.5 IRE. If unchecked, it will equate luminance values of 16 = black = 0 IRE.

IRE is an analog measure ... the waveform monitor is attempting to simulate what an analog waveform monitor would produce, but to do so it has to know what the digital->analog conversion hardware would do. In US NTSC systems, properly working D->A conversion hardware (like in a DVD player) would convert a luminance value of 16 to a 7.5 IRE analog signal. In Japan NTSC systems and PAL systems, D->A conversion hardware would convert a luminance value of 16 to a 0 IRE analog signal.

Thus, the 7.5 IRE checkbox tells Premiere how to simulate the analog waveform by assuming you're using US NTSC D->A hardware (checked), or assuming you're using Japan NTSC or PAL D->A hardware (unchecked).

Thus, if you uncheck the box, you should get values from 0 IRE to 100 IRE on a DV signal that was properly recorded by your camera using 16-235 values.

[SomeJoe]

Originally posted by neily
I wonder how professionals handle this problem. DV footage is not uncomonly seen on TV. I guess they take a "professional" attitude of just do it so it looks OK, then get on with the next project, rather than worrying about it too much.

The main problem is the RGB colorspace. If you can use a processing chain that will eliminate any conversions to RGB colorspace, you shouldn't have any problems.

My company produces educational videos on DVD. We film the footage with Sony DVCAM camcorders, and transfer to Premiere Pro for editing. Premiere Pro uses the Main Concept DV codec internally, and keeps all conversions, effects, and transitions in YUV colorspace. Further, it sends output to external plug-ins (like CCE) in YUV colorspace.

Thus, the Sony camcorders record compressed YUV (DV) on tape, that's copied unchanged to the computer, processed for transitions and effects in uncompressed YUV, output in uncompressed YUV to CCE, and encoded to compressed MPEG-2. As long as all items in this chain are ITU.601 compliant (16-235), which they are, there can be no luminance problems.

This seems to be the case with us, as we've looked at our DVD output on analog waveform & vectorscope monitors, and we get very nice 7.5-100 IRE signals.

Be aware that most cameras will record values outside of the 16-235 luminance range. Generally this is a white balance problem or exposure/iris problem on the camera. Thus, though you say that using 16-235 on the Matrox DV codec looks like it crushes whites, it may not be the codec's fault - it might be footage that just has many white values out of range.

[nicco]

e aware that most cameras will record values outside of the 16-235 luminance range. Generally this is a white balance problem or exposure/iris problem on the camera. Thus, though you say that using 16-235 on the Matrox DV codec looks like it crushes whites, it may not be the codec's fault - it might be footage that just has many white values out of range.

This is the case of my sony DCR PC100-E
In some scenes where there is some bright light it burns white, as you can see in the upper right corner of this shot:


I thought this was a problem of something inside my PC (codec, Premiere, monitor contrast,...) but I discovered that it's in the original source too; so it's a problem of the camcoder.

[Wilbert]

Be aware that most cameras will record values outside of the 16-235 luminance range. Generally this is a white balance problem or exposure/iris problem on the camera. Thus, though you say that using 16-235 on the Matrox DV codec looks like it crushes whites, it may not be the codec's fault - it might be footage that just has many white values out of range.
I understand all that. The main problem we were discussing is what do you do in that case. Do you clamp or scale the full luma range to [16,235]?

[SomeJoe]

Originally posted by Wilbert
I understand all that. The main problem we were discussing is what do you do in that case. Do you clamp or scale the full luma range to [16,235]?

If the problem is on the white end, I would go ahead and clamp the values to 235. For instance, in nicco's case above, you can see that there are whites in the upper right corner of the image that are very hot (higher than 100 IRE, outside broadcast allowed range). The detail has been lost in that area of the image anyway, and the rest of the image looks properly luminance balanced. Clamping to 235 in that case won't really lose much, and compressing range would throw luminance balance in the rest of the image off.

(Note, this is assuming that the >235 values we're talking about are Y values in a YUV image. It would be normal to have RGB values >235 after conversion from a YUV image, but that doesn't mean the original YUV image is out of ITU.601 specs).

If the problem is on the black end, that has to be treated more carefully to avoid losing detail. If there are values recorded down to 0, I'd compress luminance range, bringing blacks up to 16. If something was screwy with the original recording or encoding such that the darkest blacks have been recorded at 32 (common problem with US NTSC DV camcorders that have a "setup" switch that gets turned on when it shouldn't be), then I'd use Tweak() to rescale to 16-235.

By the way, Wilbert, I didn't quite mean to intrude on the discussion or hijack the thread. The topic kind of expanded once I started talking about waveform/vectorscope stuff. I hope this answer is helpful.

[nicco]

@SomeJoe
Can you tell me why my YC waveform is different from yours?
I haven't IRE scale on vertical axis.
http://img133.exs.cx/img133/5125/vect2ui.jpg

This YC waveform shot correspond to this picture shot:
http://img63.exs.cx/img63/8651/prov0or.jpg


How can I read it?

[trevlac]

@nicco

I know you are not asking me, and I don't use Premiere ... but maybe this will help.


The proper maximum amplitude for most video test signals is exactly 1 Volt. In television systems, the 1 Volt signal is broken down into two separate members. The lower portion is the sync signal and the upper portion is active video. In NTSC, the sync is displayed from -40 IRE to 0 IRE, and active video is from 0 IRE to 100 IRE. The whole scale, 140 IRE, equals 1 volt. Another way to say that is that 1 IRE division equals about 0.00714 volts. So the sync portion of video is about 0.2857 volts and active video is about 0.7143 volts. In PAL, sync is exactly 0.300 volts and active video is exactly 0.700 volts. The scale for PAL is 143 units with -43 to 0 for sync and 0 to 100 for active video. In PAL systems, one Unit is equal to 0.00699 volts.


To add ... translate. Joe's IRE scale ran from -40 to 100 IRE where the active picture should be between 0 and 100 which should equate to luma pixel values of 16-235. Your voltage scale runs from 0 to 1 volt. Where the active picture for PAL is .3 to 1 volt which would equate to 16-235 luma pixel values.

A little more on your picture ... I assume the borders are in your picture. I can see the thin line at .3 volts. The real picture components seem too high. I would bring the luma down a bit without having the dark parts touch the line. Not much should ever be all the way at the top. Maybe reflection highlights or a light bulb. Bringing it down may also darken the picture more than you want. I would adjust the midtones with gamma if you feel you need more detail. In the end, too much adjusting may not be worth it. To really mess with gamma you need a proper monitor to see the end result.

@Wilbert,

I think the key is not in the adjustment but in the proper identification of the problem.

-- If it is the codec, you should probably fix this and not adjust the video

-- If it is the camera/footage/source, I think it is a case by case basis where you would not use specific constant values for a change. For example, in the posted pic ... on a PC monitor it is too high but bringing it down makes it too dark ... but maybe not on a TV ... so I'd be affraid to muck with the gamma unless I could see it there.

[SomeJoe]

Originally posted by nicco
Can you tell me why my YC waveform is different from yours?
I haven't IRE scale on vertical axis.

Took some looking up, but I found the info.

IRE is apparently an NTSC-derived specification. For PAL, the analog luminance level is measured directly in volts (or millivolts, depending on the waveform monitor. Premiere Pro uses volts.)

1.0 volt = maximum white.
0.3 volt = black.
0 volts = sync pulse.

In your YC waveform monitor for your image, you can see that your whites have been clipped at 1.0 volt. This is similar to when audio is recorded with too high of a gain, and gets clipped. Your blacks are down near 0.37-0.38 volts, slightly above the darkest blacks. This makes sense, as the darkest areas in the image (clothing) aren't pure black, but close to it.

Thus, with the exception of the clipped/crushed whites, the image is not too bad. If white balance on the camera had been adjusted (which adjusts luminance gain), whites wouldn't be clipped. (Assuming the camcorder has a white balance - many don't).

For reference, in Premiere Pro, create a new bars and tone clip and put it on the timeline and take a look at it in the YC waveform monitor. You'll see white at 1.0 volt, black at 0.3 volt. Note that in standard color bars, there is one small bar at the bottom that is blacker-than-black (about 0.27 volts). On a properly adjusted PAL TV set, you should not be able to tell the difference between the blacker-than-black (0.27V) and black (0.3V) bars. If you can, brightness on the set is too high.

[neily]

As I mentioned in an earlier post, it is a while since I did my own DV codec tests. The point is not to make a scientific appraisal, but only to highlight the fact that in deciding how to process DV footage one needs to know a little about the behaviour of the camera and the PC codec. I seem to recall that some of the codecs are available as only DirectShow or VFW filters, so depending on which application is used, a different codec brand may be used.

The image shows a number of luma histograms generated from within AVISynth of the same contrasty frame, with the colourspace output for each codec indicated. The red areas represent the 16-235 boudaries. As the histogram works only within the YUV colour space, the Mainconcept result looks different, as the "luminance" is not being mapped within AVISynth from an RGB range of 0-255 to 16-235. In the Matrox blurb, it describes the "luminance mappings" as from the native DV YUV format to the RGB range indicated.

http://www.yeomanfamily.demon.co.uk/DVCompare/DV-Codec-Comparison.png

It certainly shows all the codecs to be different. It also shows that in the Panasonic and Matrox 16-235 cases, high luminance clipping is going on. The Sony and Matrox 0-255 histograms are similar, suggesting the Sony codec also fails to exploit the lower luminance range effectively. The MainConcept codec looks like it would clip the high end if passed straight to an MPEG2 encoder such as CCE. I'm not sure what is going on with the camera (a standard modern Sony HC30 model), but luminances above 235 seem to be being recorded / differentiated. Its a bit sad to see the bottom end doesn't go any lower than it does. Much of the bottom end tail is probably just noise. It really was a very contrasty image.

Also, depending on the mapping of luminance ranges, average luminance is affected substantially. Tweaking input by clamping just looses detail, but remapping also has a significant effect on average luminance.

As I said, this is not meant to be a scientific appraisal, just a rambling about how tricky this whole luminance business seems for average PC users with average, non-professional equipment.

[trevlac]

@neily

If I may be so bold as to interpret your results ... without testing or really researching the details ... This of course is all about decoding the DV.

Panasonic: The codec takes the 16-235 range of the DV and maps it out to 0-255 RGB, clamping the highs. The camera's white is too high. AVS pushes the range back to 16-235.

Sony: The codec takes to full 0-255 range of the DV and maps it to 0-255 RGB (aka it does not change the range). Going back to YUV AVS does it wrong (because it does not know you have 16-235 RGB) and compresses the range. So blacks are too high.

MainConcept: I suppose there is no mapping here. Is that the range recorded by the cam? The high looks a bit too high.

Matrox ---

16-235: Same as Panasonic.

0-255: Same as Sony.

16-255: This fixes the problem more or less. The codec maps DV 16-255 to RGB 0-255. AVS moves it back to 16-235.

---------------------------------------
My conclusion:
There are 2 flavors you get when asking for RGB from a decoder. 0-255 RGB (Panasonic) or 16-235 RGB (Sony). Matrox seems to allow both plus an extra to help with those blown out whites.


--------------------------------------

Personally, I like the Sony case. When you go to RGB from YCbCr, you don't loose any info due to clamping. But you would have to pull the black down a little. If the white was not so high, you can also tell most of the encoders that you have 16-235 RGB. Then they wont change the range.

[nicco]

@SomeJoe
http://img220.exs.cx/img220/1921/prov210ae.jpg

If white balance on the camera had been adjusted (which adjusts luminance gain), whites wouldn't be clipped. (Assuming the camcorder has a white balance - many don't).

My camcoder has a white balance option, which is "auto" by default, other options are "indoor" and "outdoor". I'll do some tests
:)

[WorBry]

Hi Trevlac,

Your conclusions about the Sony DV Codec are consistent with my own experiences and provide the best explanation I've come across so far for the luminance shift that the Sony DV codec produces.

I capture and edit PAL DV footage from my Sony PAL DCR-PC115E camcorder using Ulead Video Studio 8. For capture and editing I disable the Sony DV codec and use the native Type II DV encoder in VS8 for rendering. If I dont the edited segments (titles, transitions etc) come out lighter (hazier) than the unedited. After rendering the edited video I then enable the Sony DV Codec for further conversions. I particularly like the Sony codec because it appears to eliminate the "color bleed" that I've observed with other DV codecs (Adaptec DVSoft, Panasonic, Mainconcept).

For PC playback I mostly convert the Type II DV sources to DivX or XviD using Virtual Dub and AVISynth Initially, my standard script was:

AVISouce
ConvertToYUY2(interlaced=true)
KernelDeint(order=0,sharp=true)
LanczosResize(720,544)
Undot()

However, the "foggy" luminance effect was getting quite annoying so I decided to investigate further. Sure enough, on histogram analysis of the YUV luma, it was clear that the output was not accurately mapping to 16-235 - i.e. the blacks were too high and the whites too low. I found I could correct this by adjusting the Y offset and gain levels using ColorYUV to bring them back to 16-235 i.e.

ColorYUV(off_y=A,gain_y=B).

Generally, this requires an offset (A) in the range of -16 to -20 and a gain (B) of 16 to 20. This would be consistent with your conclusion that the AVS YUV conversion thinks it is dealing with 0-255, when it is actually receiving upsampled RGB24 16-235, and so compresses the 16-235 range still further.

Now, I also convert my PAL DV's to DVD (for TV playback) using TMPGenc 3.0 Xpress. Again, if I use the native Type II DV as source, with the Sony codec as decompressor, the DVD's have that familiar "hazy fog". To overcome this I've again resorted to pre-correction with AVS using the following source script:

AVISource
ConvertToYUY2(interlaced=true)
ColorYUV(off_y=-20,gain_y=20)

It appears to do the trick, but I have this nagging feeling that there must be a better/proper way of correcting for this with the color adjustment settings in TMPGenc 3.0 Xpress. Does anyone have experience with this who can advise me? Should I "expand" the YUV to 0-255 (which I assume is what YUV Expansion does) and then cut back the "start" and "end" brightness? Any ideas?