Conversion between RGB and 4:4:4 YUV

[trevlac]

I was shocked to read the following from Microsoft.


Color Space and Chroma Sampling Rate Conversions
This section provides guidelines for converting between YUV and RGB, and for converting between some different YUV formats. We consider two RGB encoding schemes in this section: 8-bit computer RGB, also known as sRGB or "full-scale" RGB, and studio video RGB, or "RGB with head-room and toe-room." These are defined as follows:


Computer RGB uses 8 bits for each sample of red, green, and blue. Black is represented by R = G = B = 0, and white is represented by R = G = B = 255.

Studio video RGB uses some number of bits N for each sample of red, green, and blue, where N is 8 or more. Studio video RGB uses a different scaling factor than computer RGB, and it has an offset. Black is represented by R = G = B = 16*2N-8, and white is represented by R = G = B = 235*2N-8. However, actual values may fall outside this range.



Studio video RGB is the preferred RGB definition for video in Windows ...
Full Link (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwmt/html/YUVFormats.asp)

Does this not mean that for video source, there is no such thing as TV->PC ? YUY2->RGB should not map to 0-255?

This does not seem practical for watching source on a PC. They also seem to advocate using spec 709, which may make sense for watching on a monitor, but not for 601 compliant TV watching.

Anyone know what I am talking about? Am I confused? :confused:

[sh0dan]

All AviSynth RGB<->YUV converters default to converting YUV range 16-235 to RGB 0<->255 range and vice versa. The vast majority (Vdub, Xvid, DivX, HuffYUV) also assume 601 range. The majority of DVD's are encoded in 601 range.

The graphic cards also (mostly) show 601 range in overlay mode, but some can be changed to PC-format.

So in essense - 601 range is the way to go - also for PC editing/watching.

There are however no limitations in AviSynth - almost no filters automatically kill PC-range material.

[Si]

Your quote != your conclusion therefore you are confused :p

MS are just stating a set of definitions.

I define turquoise as blue but then again I'm blue/green colourblind :)

Most YUV filters around here assume Y=16-235 apart from the obvious one's that do level conversions.

All RGB ones assume R,G and B = 0-255 (minus any exceptions that prove a rule :) )

regards
Simon

[trevlac]

Originally posted by siwalters
Your quote != your conclusion therefore you are confused :p

I may well be confused, but my conclusions may have not been clear. :D
Forget the 709 vs 601 thing. It's all about the nominal range. Black = 0 or black = 16. For Studio RGB, black = 16. Microsoft says studio RGB is what should be used for video.


Most YUV filters around here assume Y=16-235 apart from the obvious one's that do level conversions.

Assume what? That there is nothing in the 0-15 and 236-255 part?


All RGB ones assume R,G and B = 0-255

If black = 0 then this is not studio RGB. It only really matters if you do a color space conversion, or if you try to set the levels etc.




Originally posted by sh0dan

All AviSynth RGB<->YUV converters default to converting YUV range 16-235 to RGB 0<->255 range and vice versa. The vast majority (Vdub, Xvid, DivX, HuffYUV) also assume 601 range. The majority of DVD's are encoded in 601 range.

I'm not clear what you mean by 601 range for RGB. Black=0? When I look at 601, there are no calculations that convert from 16-235 to 0-255. In fact there are things like : "EY = 0.299Er+ 0.587 EG + 0.114 EB" for luma.


The graphic cards also (mostly) show 601 range in overlay mode, but some can be changed to PC-format.

So in essense - 601 range is the way to go - also for PC editing/watching.

If this means black = 16 Great !


There are however no limitations in AviSynth - almost no filters automatically kill PC-range material.

But the converters do change the range, don't they?

Help...
Treat me like I'm an idiot. I feel like we're talking about different things.


Does the above MS link not say:

in 8bit studio RGB black=16 and white=235
there is head room and toe room (aka 0-15 and 236-255 have data)
"Studio video RGB is the preferred RGB definition for video in Windows"


So, if you map TV->PC you are not using studio RGB.


Edit It is also interesting to note that Poynton discusses studio RGB here (http://www.poynton.com/notes/colour_and_gamma/ColorFAQ.html#RTFToC31). I believe his equations do no mapping. I believe they also have no 'rounding errors' for YUV->studio RGB luma conversion. It would be interesting to run all possible values to an 8bit int and then convert back to see how many work with no rounding loss.

[Si]

Sorry - you haven't understood my turquoise=blue statement :)

Microsoft can say what they they think black is

I can say what I think black is.

NTSC analog engineers can say what they think black is

etc.

Here (that is in Avisynth filters) black is R=0, G=0, B=0 (if working in RGB colourspace) white is R=G=B=255

Black is Y=16, U=128, V=128 in YUV colourspaces (YUY2 and YV12). White == Y=235 U=V=240.

99% of all Avisynth filters follow this plan e.g. a lot of YUV filters will make sure their output is in the 16-235/240 range if they change pixel values.

AFAIK nobody uses the gaps (0-16 or 235/240-255) for anything.

Its just the convention used here.

There are filters to change from one convention to another so its never a problem for anyone that want's to do something different.

On a personal note, I don't remember MS definitions saying what they say now a few years ago. Maybe 2 legs good, 4 legs bad now :D

regards

Simon

[trevlac]

@siwalters

Ok. Now I see what you are saying.

Thanks

My orignial question/post/surprise was that Microsoft is saying RGB video under windows should follow a different convention than is currently used. Seems like a big switch.

Now the next question is why the change? Why is studio RGB preffered over computer RGB? Especially if 0-15 and 236-255 have no use.

I suspect it is because conversions from/to YUY2/RGB(16-235) suffer fewer rounding losses, there is head/toe room if the equipment/source is off a bit, and analog source does not really run in the range of 0-100IRE.

If I come across any practical examples, I'll post. Past that it is truely just definitions and speculation.

Cheers

[scmccarthy]

conversions from/to YUY2/RGB(16-235) suffer fewer rounding losses This is my suspicion too. My feeling is that if I were converting from YV12 to RGB and back to YV12 internally within a filter (because the filter only works for RGB), The best way to go would be to use RGB(16-235).

If we look closely at the math involved in a YV12->RGB->YV12 conversion, we can see that the best 8-bit, interger math solution to minimize round off is to use RGB(16-235).

Regarding Simon Walters comment:AFAIK nobody uses the gaps (0-16 or 235/240-255) for anything. I stuck the following note in my monochrome.cpp file: // Note that my variable names are borrowed from the Levels filter.
// Originally, I assumed that Y could not go higher than 240.
// By experiment, I discovered Y can be treated like the unsigned char it is.
// Like unsigned char, its limits are 0..255 inclusive. I like to learn by experience as well as theory. This might only work becuase Avisynth limits Y for me, but if it works, it works.

Stephen

[Le Furet]

Sorry, I may be drunk, but I can't understand how a loss of precision (you auomatically lose precision when coding using 235-16+1=220 different values rather than 256) could result in a better conversion :confused:

[trevlac]

Originally posted by Le Furet
Sorry, I may be drunk, but I can't understand how a loss of precision (you auomatically lose precision when coding using 235-16+1=220 different values rather than 256) could result in a better conversion :confused:

If you start with YUY2 which has a nominal range of 16-235, going to 0-255 will not gain you any xtra values. If it did, then 1 input color would equal >1 possible output color, so xtra range must have gaps or be errors. A generallized luma formula for conversion with remapping is:

Y = (.299R * 256/220 + .587G * 256/220 + .114B * 256/220) + 16

In addition, conversion without remapping is done by the following formula for Luma. (per ITU-601)

Y = .299R + .587G + .114B

and back

R = Y
G = Y
B = Y

So you see, there is no loss for luma due to rounding if you do not change the range. Try any number thru the luma calc. No rounding.

Finally, seeing that most output ends up as some form of YCbCr, any xtra tonality gained in RGB will eventually be lost. Do doubt, any interium editing steps would be much better in RGB888.

This (http://www.poynton.com/notes/colour_and_gamma/ColorFAQ.html#RTFToC31) seems to be the most precise formula I can find for conversion without remapping. It unfortunately gives RGB a range of 0-219. Without the 16 dc offset changes, I'd guess you could use it for 16-235 to 16-235 conversions.

[scmccarthy]

@trevlac,

Yes, this is what I meant.

Except, we could call it RGB(220) instead of RGB(15-235) or RGB(0-219).

The range from zero to 219 is probably easier to deal with inside a filter, so that is the way to go. The idea is to only use this for intermediate processing to avoid loss of precision. Only the multipliers will affect the precision.

Stephen

[trevlac]

@scmccarthy

'studio RGB' seems to be what a number of sources use. RGB(220) would be more clear.

Also, there are other issues with conversion.

For example, one needs to upsample the 4:2:2 (or 4:2:0) CbCr data to 4:4:4. Most converters probably just double the pixels, which of course is fast, but distorts the color info. It would be more appropriate to actually resample the data. A weight median method that takes vertical and horizontal into consideration, might be good. Interlace is a problem especially for 4:2:0. If you really wanted to be fancy, you would take the sharpness of the luma and apply it to the chroma. (I'm not sure how this is done, but I'd love to know. I have seen references to doing this.)

PIC mjpeg does not remap the range. If you store the data as yuy2 and ask for RGB, it gives you studio RGB. Most people seem to consider this a bug. I don't know what PIC does with the color.

[sh0dan]

Where is this leading?

Since I've implemented most of the internal colorspace conversion, I might be able to help, but since it isn't clear what you are trying to find out/achieve, I find it hard to help you.

"Studio RGB" / "RGB(220)"just sounds like a bad idea that should be lost in space. I can't see where it would be any advantage to use it.

All internal converters use interpolation. Yes 4:2:0 interlacing is a special darling to do. It is implemented as a 75/25 weighed average, as suggested by this page (http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html).

[trevlac]

I'm sorry. I did not mean to imply that the internal converters are low quality. My remarks were towards people doing conversions inside their own filters, which is what the other poster brought up.

The original intent of my post was to get thoughts on Microsoft's comments (that seem to imply) that RGB(220) is better for video. Simon made it clear to me that, that is not the AVS standard.

If you feel this is useless speculation, I will keep to myself.


Only reason to use RGB(220) is if you are going to convert. Pluses of RGB(220) over RGB(256) would be:


Little (if any practical) rounding loss if you use poyntons formulas
Preserves any data outside the nominal range. Techniques like PC->TV (on YUV, to get at the data in RGB) will crush the range


On the minus side is breaking the standard practice and viewing.

I don't wish to campaign for any changes. I was just wondering what people were doing. Especially seeing that that MS article says the standard practice is not preferred.

[sh0dan]

I'm just trying to figure out how to help! :cool:

In any case, if you've got PC-range YUV you can still maintain the range, by using the rec709 range matrix - ConvertToRGB(matrix="rec709"). There is no rec709 matrix to convert from RGB, however. I have not seen anything assuming PC-range YUV input - CCE for instance will just convert it to 601, IIRC.

I never hope there will be anyone actually assuming studio RGB - this will make yet another problem. I'd personally wish, that Rec601 was never thought up, and everything was just Rec709. YUV could certaintly use the extra precision. But this is not how it was supposed to be. :(

So, to conlude - I'm sorry if I'm a bit agressive about killing off any talks about "studio RGB", as the 601 vs 709 have already created enough problems.

[trevlac]

Originally posted by sh0dan
I'm just trying to figure out how to help! :cool:
.....

So, to conlude - I'm sorry if I'm a bit agressive about killing off any talks about "studio RGB", as the 601 vs 709 have already created enough problems.

I hear ya, and I'm with ya. As a rule, know what you've got and know what you are doing. You can probably do whatever you want with the tools that are already available. :)

[MfA]

Originally posted by trevlac

R = Y
G = Y
B = Y


Hmmm?

Anyway ... the color transform isnt invertible whichever range you pick for RGB, the more precision in RGB the smaller the errors if you go YUV->RGB->YUV.

[scmccarthy]

@trevlac&@sh0dan

Perhaps it would help if you understood what the work 'studio' in studio RGB(220) means. A studio is a place where the images are worked on prior to release to the public. It is not for viewing, but strictly useful only for processing. It is a useful alternative to 4:4:4YUV only. The Color FAQ quoted by trevlac is my most trusted source of info other than BT.601, BT.709, etc. As Charles Poynton put it:Studio R'G'B' signals use the same 219 excursion as the luma component of Y'CBCR. @sh0dan

trevlac is right in pointing out that I suggest using RGB220 for intenal use of a filter only. Take a look at a few of the filters that still have not been converted from RGB to YV12 and use RGB220 to convert them to a YV12 filter.

@trevlacAlso, there are other issues with conversion. Most of these issues don't apply. The xvid converters avisynth uses don't just double the pixels. (Sh0dan did in fact just double the pixels when avisynth2.5 alpha first came out just to save time, but he had a todo note by the code// TODO: Interpolate UV downwards. There is a lot of attention given to get the color conversions right in avisynth.

Convert.cpp has been an inspiration to me of good coding and good coding style. It is how I learned how to write a filter. If you have any questions, investigating convert.cpp might be very interesting and helpful.

Stephen

[trevlac]

I was inspired by MfA's "Humm?", I wondered how it worked out if I used Poynton's formulas.

Seeing that it would be a stretch to call me a programmer, I certainly may have made a mistake, or the sample may not be practical.



#include <stdio.h>
#include <math.h>

void main() {

int good, bad, outOfGamut, Y2, U2, V2, r, g, b;
double R,G,B;

good=bad=outOfGamut=0;

for(int Y=0;Y<256;Y++) {
for(int U=0;U<256;U++) {
for(int V=0;V<256;V++) {

//** Calc RGB as doubles **
R = Y + (350.901/256)*(V-128);
G = Y -(86.132/256)*(U-128) - (178.738/256)*(V-128);
B = Y + (443.506/256)*(U-128);

//** Check for valid values
if(R>255 | R<0 |G>255 | G<0 |B>255 | B<0) {
outOfGamut++;
continue;
}

//** Turn Double into Int
r = (int)(R + 0.5);
g = (int)(G + 0.5);
b = (int)(B + 0.5);

//** Go back to YUV
Y2 = (int) (((76.544*r + 150.272*g + 29.184*b) / 256) + 0.5);
U2 = (int) (((-44.182*r -86.740*g + 130.922*b) /256) + 128.5);
V2 = (int) (((130.922*r - 109.631*g - 21.291*b) / 256) + 128.5);

//** Compare
if(V2==V && Y2==Y && U2==U) { good++; } else { bad++;}
}
}
}


printf("good: %d\n bad %d\n out %d\n"
, good, bad, outOfGamut);
}


Results
bad: 218
good: 4098313
outOfGamut: 12678685

218 rounding errors (all in U) with 4million conversions with no error.


If I check for outOfGamut on the int values, I get the following:
31066, 4164554, 12581596

Interesting ...... no?

[scmccarthy]

@trevlac

Don't get yourself off track and read too much in being out of gamut.

YUV is a larger color space than RGB and there are some legal values in YUV that are illegal in RGB. Color spaces are interesting in that they are all tristimulus values, but they do not cover that exact same territory in their entirety. That is a completely different issue than whether it is useful to have a version of RGB with head and toe room like YUV has or whether there is an advantage in not stretching the 220 luma values of Y into the 256 luma values of RGB in order to retain sharpness.

Take a harder look at the whole idea behind color spaces in general before diving into the particuliars.

I am sorry if I misunderstood you. But I think that otherwise this is some good advise for you. This is a good exercise, but you also should step back and look at the big picture.

Stephen

[sh0dan]

Originally posted by trevlac

Results
bad: 218
good: 4098313
outOfGamut: 12678685


Getting more than 3 times as many out-of-Gamut values than good values indicate an error somewhere, unless I'm misreading your intentions.

[trevlac]

@scmccarthy, @sh0dan,

I don't think a high outOfGamut is a problem. According to Poynton, only 25% of the colors should map. Intuatively, half of the U and V values produce negative results, because you subtract 128.

I'm not sure why my outOfGamut number is lower than Poynton's. I will try to run the same test with Poynton's PC formula. It should be easy to run for any formula.


@scmccarthy,

I don't think there has to be new code for RGB(220). It is more of a concept of what is inside the RGB data you have. I did not intend to get off track. I just thought it might be interesting to see what the round off issue really is.

Thanks for the comments on colorspaces. I have been learning a bunch about them recently. Even wrote a vdub waveform monitor/histogram/etc tool. YCbCr covers a wider color space than RGB. In fact, it also covers a wider space than is 'legal' for YUV/YIQ. And RGB covers colors that are not 'legal' in YUV/YIQ. In practical terms, one can create a menu on a pc (RGB), convert it to YCbCr with no problem, and still have it clipped because the PAL/NTSC signal would be above/below 100/0IRE.

I find this whole colorspace thing to be 1 big puzzel. Especially when you throw in 601, 709, linear RGB, yada yada. :D

[sh0dan]

For reference you can look at the internal converters (http://cvs.sourceforge.net/viewcvs.py/avisynth2/avisynth/src/sources/?sortby=date&only_with_tag=MAIN) (mostly ASM), or the recently done Any CS -> YUV 4:4:4 here (http://cvs.sourceforge.net/viewcvs.py/avisynth2/avisynth/src/filters/overlay/444convert.cpp?sortby=date&only_with_tag=MAIN) (all C++).

You might notice that YUY2 and YV12 -> YUV 4:4:4 doesn't do interpolation of the chroma components. I actually started by implementing that, but dropped it again, since I realized that it would only blur the chroma when it was converted back. However, YUV 4:4:4 -> 4:2:2 and 4:2:0 averages the chroma in the pixels inside the area they cover. This produces untouched chroma in areas that hasn't changed, and still avoids chroma aliasing.

[trevlac]

@sh0dan,

"Convert444NonCCIRFromRGB" & "Convert444NonCCIRToRGB" excellent. I will have to study it/play with it to know. Most educational.

Thanks

:D

[Bidoche]

After reading this discussion, I am tempted to actually make (in 3.0) the distinction between PC and TV range.

That is to say that, instead of having just one YV12 ColorSpace, you have YV12PC and YV12TV. Same thing for others YUV ColorSpaces.

This will enforce filters writers and/or script users to care about it.
If one had to insert a ConvertToTVRange or AssumeTVRange before a filter...


So, what do you think ?

[hanfrunz]

mmmh i think this not a good idea, because these colorspaces are the same. The differnce is just that you define an overhead in the "TV" scale. But values >240 are possible but defined as illegal. Maybe a statusbit 0=TV 1=PC would be better. So newer filters could check this bit and old ones just don't care...
I think only filters which do colorspaceconversions do need to know if it is TV or PC scale.

hanfrunz

[sh0dan]

@Bidoche: My designer philosophy is KISS, so no. Only a smaller portion of the filters need to care about PC/TV range, usually involving some kind of RGB conversion.

[trevlac]

@Bidoche

I don't know if I get a vote, but ... :)

- YUY2 and YV12 should always be TV range
- RGB can be both TV or PC
- Only conversions from/to RGB should ever be concerned about this
- Don't know if it's possible to make it clear to all users

[sh0dan]

Originally posted by trevlac
- YUY2 and YV12 should always be TV range
- RGB can be both TV or PC

I agree on your two other points, but IMO the two statements above should be switched around.

YUV can be 0->255 AND 16->235. RGB can only be 0->255.

[trevlac]

Originally posted by sh0dan
I agree on your two other points, but IMO the two statements above should be switched around.

YUV can be 0->255 AND 16->235. RGB can only be 0->255.

Strange.... let me try again.

YCbCr has a full range of 0-255, but the nominal range of YCbCr is defined as 16-235 where black is 16 and white is 235.

Bottom of page 17 here (http://www.video-demystified.com/mm/tutor/ch03.pdf). Just really talks about what is on page 6 here (http://bs.hhi.de/users/karl/specs/0601-5.pdf).

I have never seen a reference to YCbCr having black as 0 and white as 255.

RGB on the other hand seems to be a strange animal. The "Color Space and Chroma Sampling Rate Conversions" section of the microsoft link (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwmt/html/yuvformats.asp) was what made me post.



Computer RGB uses 8 bits for each sample of red, green, and blue. Black is represented by R = G = B = 0, and white is represented by R = G = B = 255.
Studio video RGB uses some number of bits N for each sample of red, green, and blue, where N is 8 or more. Studio video RGB uses a different scaling factor than computer RGB, and it has an offset. Black is represented by R = G = B = 16*2N-8, and white is represented by R = G = B = 235*2N-8. However, actual values may fall outside this range.

The second is a messy way to say it (because they are trying to account for > 8 bit), but it works out to a 16-235 range. All RGB.

----------
Your position may be that using RGB 16-235 just adds to the confusion, but I can't see how you support 0-255 YCbCr.

[Bidoche]

Originally posted by trevlac
I have never seen a reference to YCbCr having black as 0 and white as 255.Now you have seen one, made by sh0dan just the post above yours...

[trevlac]

Originally posted by Bidoche
Now you have seen one, made by sh0dan just the post above yours...
Well, either the senior members are joking with the newbies (without smilies) or this newbie is just lost....

I was refering to a standard of some sort. In the mean time, someone did point out to me that jpeg stills use YCbCr with a nominal range of 0-255. Perhaps some PC video is based upon the same. :)

No offence intended. I am clearly out of my league here. Shodan was kind enough to point me to some great stuff in avs. I'll try to hold off on the speculations. :)

[Bidoche]

I guess I could have added a smiley, but my point there was that we don't need it to be a standard to use it as we please.