Testing the colorspace of an output video

[Gauze]

I'm trying to get a better grasp of colorspaces and what my editor is doing.

My first concern is whether the output color range is 0-255 or 16-235.

Now, I could possibly test this by making a test pattern in Gimp or something but since I have no idea what my tools are doing, I'm not sure this is the best approach.

My second concern is BT.601 vs BT.709. I'm pretty sure both constrict the color range to 0-255 but they also have different Luma coefficients. How would I tell which one would make my output better? Or am I missing something else completely and I shouldn't even worry about it.

My editor is Sony Vegas 10 and I'm either going to output to Debugmode Frameserver which seems to mean I have to do an 8bit project.

[ChiDragon]

Make a test pattern in Gimp? Why reinvent the wheel?

I would buy a Blu-ray test disc (Spears & Munsil, Digital Video Essentials, whatever) and run the known patterns through your workflow. S&M is nice in that it includes printed images showing what the patterns should and should not look like. Maybe DVE does too; I only rented that one.

There are free alternatives too like AVS HD 709 and drmpeg's stuff. Or images somewhere online if you really want a BMP instead of a video. But then you have to ensure your random internet source actually has the correct test pattern and not just some nth-generation replication of the idea.

[Warperus]

1) First of all, I'd like to recommend reading Glenn Chan's article about colorspaces/levels in Sony Vegas 9/10 (not much changed in 11 and 12). You can find it here along with other (mindcrashing) stuff:
http://www.glennchan.info/articles/articles.html

2) BT.601 and BT.709 have identical Luma definition and white point. They have different base colours and different matrix coefficients, but not luma.
E = (1.099 L0.45 – 0.099) for 1.00 ≥ L ≥ 0.018
E = 4.500 L for 0.018 > L ≥ 0
where:
L: luminance of the image 0 ≤ L ≤ 1 for conventional colorimetry
E: corresponding electrical signal.

Unfortunately, it's not the end of story. Computers use sRGB colorspace for pictures/video representation (it's not studio rgb in vegas terms, but quite a different thing, defined by standard IEC 61966-2-1:1999). This sRGB has different luma definition. It has base colors and matrix coefficients similar to BT.709 though. As said in proposition for sRGB difference in luma is not too big. Ony more thing, sRGB proposes levels of 0-255 for R'G'B' components while Bt.601 and BT.709 do not do it - they stick to 16-235.

3) I'd like to stress the point of BT.601, BT.709. These two recomendations are the baseline of television signal representation. They do not directly define how video is represented inside video editor or any other video processing tool. Even more, they do not directly define how file is being stored. Every video format has its own definition of color/luma representation rules (with variants).

H.262 (standard paired with MPEG-2) defines 3 things that can be stored in elementary stream:
* Color primaries (BT.709, Unspecified, BT.470-2 System M, BT.470-2 System B/G, SMPTE 170M, SMPTE 240M)
* Transfer characteristics (BT.709, Unspecified, BT.470-2 System M, BT.470-2 System B/G, SMPTE 170M, SMPTE 240M, Linear)
* Matrix Coefficients (BT.709, Unspecified, FCC, BT.470-2 System B/G, SMPTE 170M, SMPTE 240M)
Levels 0-15 and 236-255 are supposed to be illegal, but apart from 0 they can be present in video stream.
As you can see, there is no direct link to BT.601. It's represented by BT.470-2 System B,G instead.

H.264 defines those 3 things as well. But in addition here we have "full range" flag in elementary stream, that says of 0-255 levels.
MPEG-4 asp uses BT.601 settings.
MPEG-4 avc (H.264) uses BT.709 as default settings.
QuickTime uses it's own settings.

4) Default color primaries/coefficients in H.262 standard published in 1995 are of BT.709. Unfortunately, it was not stressed enough and big part of DVD encodes was done in BT.601. Later H.262 defined "best guess" for "undef" values. Effectively you get SD content in BT.601 if it's not defined specifically.
HDV content is provided mostly in BT.709 settings if not defined otherwise.
H264 video is mostly provided in BT.709 settings.

5) Sony Vegas uses internal codecs for h.264 and mpeg-2, external quicktime library for appripriate video. DV is tricky, there are many settings/possibilities for it. Video in avi is mostly imported through external codecs installed in windows, but for example SonyYUV is Vegas-provided codec. And last thig might be tricky as well, since vfw (used for avi) doesn't report colorimetry. Even more, you can have different YUV->RGB and RGB->YUV filters installed.
For example, default YUV->RGB filter preinstalled in my windows 7 uses (I believe) BT.709. When I installed xvid, its installer added its own YUV->RGB filter, that always used BT.601 rules.

6) Now back to practice. You import some video into Vegas and then export in some format. What you have on timeline is BG'R'A' representation of video (for 8-bit project). There are two possibilities:
a) studio RGB - I believe it to be quantized by BT.709 rules, 16-235 for each of R', G', B' components (' means gamma corrected values as gamma is not linear)
b) computer RGB - I believe it to be quantized by sRGB rules, 0-255 for each of R', G', B' components
You can use Levels filter with presets "Computer RGB to studio RGB" and "Studio RGB to Computer RGB" to convert them back and forth. I believe conversion is not up-to-bit corrent, but for practical use it's more than enough.

7) Many camcoders use 16-255 range to preserve overwhites.
After you import this stuff into Vegas (in 8-bit project), you get 16-255 values for R'G'B' components with common overwhites in 236-255 range and random overdarks in 1-15 range. Latter ones are artifacts and mostly can be ignored unless you prepare your video for TV broadcasting.
Note: If you do, use Broadcast Color filter to ensure levels in video.
As for overwhites you have to correct them in color correction phase or after.
If you use 32-bit with studio gamma (2,2222), you get picture similar to 8-bit project. Even though values are represented as fractions of 1: 0.62-0.92 for studio levels or 0.0-1.0 for computer levels (I might be a bit off in calculations as I don't generally use 32-bit projects). You can use levels filter freely here and most effects work similar to those in 8-bit mode.
If you use 32-bit with linear gamma (1,000), you get a picture linear gamma correction.
* For regular video: Import step clamps overdarks (0-15 YUV (Y'CbCr) values) and they all are 0 in project. It also clamps overwhites (236-255) and they all are 1 in project. You lose distiction between overwhites here! Do not use this mode unless you know what you are doing.
* For computer video: 0-255 is transformed into linear gamma in range 0-1. Note, that 128 will not become 0.5 due to gamma linearity in project and gamma non-linearity in sRGB.
As a rule of thumb, do not use 32-bit project with linear gamma. You don't really need this mode unless you are perfectionist that wants linear light transforms for some effects (you can read about those things in Glenn Chann's page).

8) Preview in Sony Vegas has 2 different windows: small preview and full screen preview:
* Small preview shows everything in computer RGB, without corrections.
* Full screen preview shows picture according to settings. If you turn on correction to studio levels, you'll see it as it's supposed to be in player provided you render using mainconcept avc. If not, you'll see computer levels.
Set up preview device in Vegas setting according to project you are doing and check color correction on full screen preview if you plan to render using internal mpeg-2/avc codec!

8) Import of avi uncompressed RGB does not change picture in 8-bit project. Export to uncompressed RGB does not change picture in 8-bit project. So if you have computer levels in video (0-255 as defines by microsoft sRGB), you'll get it in the resulting file. If you have studio levels in video, you'll get it in output file.
Most avisynth processing tools treat RGB input as sRGB (full range = computer levels = computer RGB), so for imported h264 video from sony camera you want to finish processing in computer levels.
Generally it's bad idea to export using YUV vfw codec. SonyYUV will encode based on studio levels. External codecs will use installed windows RGB->YUV filter and it's not known beforehand what filter will be used (remember my example of xvid affection on import).
If you use debugmode frameserver, stick to RGB too - output will be the same as to avi uncompressed. It's safer to convert colors correctly later in avs script or transcoder settings.

[Gauze]

So... when you say BT.601 and BT.709 have different base colors and different matrix coefficients... what does that mean?

That sounds like something that I wasn't even thinking was possible to be different.

I mean, I guess it doesn't really matter since everything seems like BT.709 but I would still like to know what is going on.

[ChiDragon]

How much background reading have you done on this? Here (http://www.poynton.com/notes/colour_and_gamma/ColorFAQ.html#RTFToC9) is the closest link that I can get to answering your question, but I don't know what basis of knowledge you have to draw from.

[fvisagie]

I'm trying to get a better grasp of colorspaces and what my editor is doing.

My first concern is whether the output color range is 0-255 or 16-235.

Assuming you're working in YUV -

to get a rough idea of a frame's colour values,
Blur(1.0).ColorYUV(analyze=true)

to get a rough idea of how many represent out-of-gamut RGB values (i.e. will be clipped in RGB conversion),
#-------------------------------------------
# ShowIllegalYUV 1.0.0
# Visualises illegal YUV combinations by displaying them in the output; legal values are displayed as a uniform medium grey
#
# Useful when
#
# Checking whether clip contains illegal YUV value combinations; or
# Checking whether YUV correction has desired effect
#
# ShowIllegalYUV(clip c, bool "il", string "mx")
#
# <c> input clip
# <il> = true/false, specifies whether input clip is interlaced, default true
# <mx> = "601"/"709", specifies whether to use Rec.601 or Rec.709 conversion matrix coefficients, default "601"
#
# Requirements
#
# input clip must be in YV12
#

function ShowIllegalYUV(clip c, bool "il", string "mx") {
il = Default(il, true)
mx = Default(mx, "601")
return(Subtract(c, ConvertToRGB(c, interlaced=il, matrix="Rec"+mx).ConvertToYV12(interlaced=il, matrix="Rec"+mx)).Levels(112,1,144,0,255))
}

(both of the above ideas from jagabo)

and, to get a more exact (and much slower) idea of illegal YUV combinations, ShowBadRGB (http://forum.doom9.org/showthread.php?p=1405496#post1405496) by Gavino and Worbry.

Cheers,
Francois