Could we make a colour coefficients/levels sticky ?

[madhatter300871]

Hi

After MUCH reading I think I have correctly made the following assumptions, and wonder if it would be possible/worthwhile having a sticky made by the more knowledgeable on here regarding colour reproduction.

Just something that can be referred to as a starting point, a good rule of thumb. I know every case cannot be dealt with as different videos are manipulated in different ways.

I know most of the problems that I have encountered with playback "quality" has been down to the Rec.601/Rec.709 coefficients and incorrectly mapped levels, and after much reading and finally doing some very easy level tweaking and being aware of when to use colormatrix in an avisynth script my final movie playback has in the main been much, much better.

Provided I have a good source to start with, here are some points that I have come to assume can be used as a good general rule of thumb:-

SOURCE
--------
1. Blu-ray (h264 & VC1) HD video usually uses Rec.709 coefficients and 0-255 levels.

2. Xvid/DivX/MPEG2 SD video usually uses Rec.601 coefficients and 16-235 levels.

ENCODING
----------
1. Encoders do not expect a certain set of coefficients per-say, so avisynth "colormatrix" does not need to be used simply because a certain encoder is being used.

2. Rec.601/Rec.709 only needs to be considered when converting from one colour space to another, usually YV12 to RGB.

3. avisynth "colourmatrix" filter should be used when converting from SD to HD and vice versa. This is only because the final playback device will use either Rec.709 or Rec.601 coefficients when converting the YV12 data to RGB.

DECODING
----------
1. When playing back on a PC, using overlay mixer will map video from 16-235 to 0-255 levels.

2. When playing back video on a PC, VMR mixer will NOT map 16-235 to 0-255 levels.

3. ffdshow can map levels using the "levels" option.

4. For final output onto a digital screen (be it LCD monitor, plasma, projector) a conversion to RGB is always done at some point by some process.

5. A vertical resolution of 720 and above is considered to be HD.

6. A vertical resolution of 719 and below is considered to be SD.

7. Point 5 & 6 are open to much debate, with different renderers and different players assuming different values... so testing could be done by a user on a per system basis, find out what works for you.

I would genuinely appreciate your comments on my assumptions.

Many thanks.

[Wilbert]

Could we make a colour coefficients/levels sticky ?
http://avisynth.org/mediawiki/Luminance_levels and http://avisynth.org/mediawiki/Colorimetry

[madhatter300871]

Hi Wilbert

Yes, those two pages where amongst the final few pages that I read, indeed it was those two pages that ultimately brought my reading to a close.

So do you have any comments on the points that I make ?

Is a sticky not a good idea regarding this, a "quick-reference" if you will that will enable people to get a better grasp of how colour information is stored and subsequently played back by a PC ?

I for one had to do a great deal of reading in order to come to the conclusions that I did with some confidence, and I am not yet totally convinced my conclusions are correct.

Would be great if the more knowledgeable, such as yourself, could confirm my points, or change them as appropriate.

What do you think ?

[hello_hello]

Blu-ray (h264 & VC1) HD video usually uses Rec.709 coefficients and 0-255 levels.

Are you sure about Bluray using 0-255 levels? I thought pretty much all video uses 16-235? Maybe I'm missing something, but if I play the DVD and BluRay version of the same movie on my PC, they seem to display using the same luminance levels.

Encoders do not expect a certain set of coefficients per-say, so avisynth "colormatrix" does not need to be used simply because a certain encoder is being used.

The way I understand it both XviD and X264 use Rec.601 when encoding RGB, but that's the only time where you could say they "expect" a particular colorspace.

When playing back on a PC, using overlay mixer will map video from 16-235 to 0-255 levels.
When playing back video on a PC, VMR mixer will NOT map 16-235 to 0-255 levels.

Both renderers display video using exactly the same levels on my PC (XP). Neither remaps to PC levels. I have my nvidia card set to remap video levels so they always display correctly regardless of which media player or renderer I use.

[madhatter300871]

Hello hello_hello !

I'm not sure that blu-ray uses 0-255 levels, no. I was looking for clarification of this.

Well the bluray version and DVD version would display the same luminance as they are decoded correctly, that doesn't prove that bluray is or isn't encoded full range. They would only display differently if they where decoded incorrectly, using either the wrong range or the wrong coefficients. Wouldn't they ?

Are you sure X264 uses Rec.601 to encode at HD ? I thought the encoder just encoded what it was given, and it's up to the decoder/renderer to use the right coefficients when converting to RGB for final display.

How do you know your render is not remapping ?

I was hoping we could convince the guys here to make a sticky regarding this, I'm sure a good solid understanding would allow us all to be more informed when doing encodes.

Thanks.

[hello_hello]

I'm not sure that blu-ray uses 0-255 levels, no. I was looking for clarification of this.

When I read your post, I actually looked for clarification that it does, but couldn't find any.
Reading this: http://avisynth.org/mediawiki/Luminance_levels#What_are_luminance_levels.3F
It appears what is usually referred to as the colorspace (R.601, R.709) is actually different standards for encoding luminance, but the range of levels which can be used (16-235, 0-255) is independent of each. Best as I can tell 16-235 is used for just about all video.
In fact if you fiddle around in ffdshow's Output/RGB Conversion options, the tooltip for standard levels (16-235) says "nearly all video uses this".
Maybe when it comes to digital or HD outputs such as those of a BluRay player it's the output levels which are expanded (as needs to be done when displaying video on a PC monitor which uses 0-255).

Well the bluray version and DVD version would display the same luminance as they are decoded correctly, that doesn't prove that bluray is or isn't encoded full range. They would only display differently if they where decoded incorrectly, using either the wrong range or the wrong coefficients. Wouldn't they ?

I have my video card set to expand all video to PC (0-255) levels. I assume if I was playing a BluRay video which uses 0-255 levels and the video player decoded it using those levels, I'd have to disable the setting in my video card or the video would look really dark.
Kind of like when playing video with standard 16-235 levels. If I use the TV to PC levels shader in MPC-HC to expand the levels to 0-255 as well (with the video card set to also do the same), the video displays obviously way too dark.

Are you sure X264 uses Rec.601 to encode at HD ? I thought the encoder just encoded what it was given, and it's up to the decoder/renderer to use the right coefficients when converting to RGB for final display.

Nah..... I'm not sure of anything any more.
Maybe I misunderstood this when I read it, and maybe I need to understand the exact definition of "raw" video, but from the x264 wiki regarding the input colorspace switch: http://mewiki.project357.com/wiki/X264_Settings#input-csp

"Tell x264 what colourspace your raw video input is with this switch. Supported colourspaces are listed in x264 --fullhelp.
Note that while RGB colourspaces are listed, the video is converted to YUV using the bt601 (ie, "SD") matrix before encoding."

I've read posters offering the advice that XviD expects R.601, and the definition of "expects" seems to be when encoding RGB video it converts to YUV using R.601, and I took the above to mean the same thing.... but maybe not. I wonder what the definition of "raw" video input is in the above example.

What I do know is that when re-encoding video (DVD, BluRay, AVI, MKV.... you get the idea...) with one of the usual encoder GUIs, the colorspace used for the encoded video is always the same as the colorspace of the source video, so yeah the encoder just encodes what it's given.
Getting it to decode/playback correctly is of course another story... as you said the colorspace used by a PC when playing video is dependant on it's resolution (and I assume most standalone players work on the same principle) so you need to convert from R.709 to R.601 (or the other way around) according to the source and encoded video's resolutions.
I've never had to change the levels (16-235 etc) when encoding, regardless of the definition.

Generally things should be predictable, R.601 for SD and R.709 for HD, but thanks to Windows being dumb there's a grey/wrong area. Any 1080p encode should display fine (R.709) but for those 720p encodes where the image has been cropped (1280x544 type resolutions) even though they're HD Windows displays them using R.601 and therefore the wrong colors. When playing those types of encodes with MPC-HC, I use the R.601 to R.709 shader to fix the colors on playback.

I was hoping we could convince the guys here to make a sticky regarding this, I'm sure a good solid understanding would allow us all to be more informed when doing encodes.

In the mean time this post is a summation of several discussions. http://forum.doom9.org/showthread.php?t=133982#post1090068

I'm still trying to get my head around the whole thing myself.

[Wilbert]

and it's up to the decoder/renderer to use the right coefficients when converting to RGB for final display.
Right. The decoder/renderer might or might not choose to obey the info in the header (which can be set for x264, but not for Xvid). From the wiki of hello_hello:

Video Usability Info

These options set a flag in the output stream that can be read by the decoding application and possibly acted on. It's worth noting that most of these options in most scenarios are pointless, and are usually ignored by software decoders.

(...)

Default: undef

Set what color primaries for converting to RGB.

Possible Values:

* undef
* bt709
* bt470m
* bt470bg
* smpte170m
* smpte240m
* film

(...)



It appears what is usually referred to as the colorspace (R.601, R.709) is actually different standards for encoding luminance, but the range of levels which can be used (16-235, 0-255) is independent of each.
Right.

Nah..... I'm not sure of anything any more.
Maybe I misunderstood this when I read it, and maybe I need to understand the exact definition of "raw" video
Raw means raw: uncompressed video without header.

"Tell x264 what colourspace your raw video input is with this switch. Supported colourspaces are listed in x264 --fullhelp.
Note that while RGB colourspaces are listed, the video is converted to YUV using the bt601 (ie, "SD") matrix before encoding."

I've read posters offering the advice that XviD expects R.601, and the definition of "expects" seems to be when encoding RGB video it converts to YUV using R.601, and I took the above to mean the same thing.... but maybe not. I wonder what the definition of "raw" video input is in the above example.


If it's raw you need to tell the encoder what you supply to it. With --fullhelp you will see: i420, yv12, i422, i444, rgb and a bunch of other input formats. If you choose a rgb format, it will do "the video is converted to YUV using the bt601 (ie, "SD") matrix before encoding.".

[madhatter300871]

OK, that Q & A was a big help.... I didn't see that !! Do I need glasses ? Anyway, I now assume the following :-

SOURCE
--------
1. Blu-ray (h264 & VC1) HD video usually uses Rec.709 coefficients and 16-235 levels.

2. Xvid/DivX/MPEG2 video usually uses Rec.601 coefficients and 16-235 levels.

ENCODING
----------
1. HCenc flags the output as Rec.709, so it really should be given Rec.709 material.

2. DivX/XviD/MPEG4 ASP codecs expect Rec.601, so really should be given Rec.601 material.

3. x264 can be fed with Rec.601 or Rec.709, choose depending on resolution.

4. avisynth "colourmatrix" filter should be used when converting from SD to HD and vice versa. This is only because the final playback device will use either Rec.709 or Rec.601 coefficients when converting the YV12 data to RGB.

DECODING
----------
1. When playing back on a PC, the decoder/render can map levels to 0-255 if configured to do so.

2. The decoder/renderer MIGHT obey headers, MIGHT not. The only thing you can really do is use your eye and decide if you need to do a Rec.709/Rec.601 conversion on playback on a per film basis.

3. ffdshow can map levels using the "levels" option.

4. For final output onto a digital screen (be it LCD monitor, plasma, projector) a conversion to RGB is always done at some point by some process.

5. A vertical resolution of 720 and above is considered to be HD.

6. A vertical resolution of 719 and below is considered to be SD.

Point 5 & 6 are open to much debate, with different renderers and different players assuming different values... so testing could be done by a user on a per system basis, find out what works for you.

Question 1 :- Does XviD REALLY require Rec.601 ? Well if thats the case and I encode to HD resolutions with XviD wont it then playback using the wrong coefficients ? So on playback I would have to do a 601 to 709 conversion either with MPC or avisynth or whatever method I choose ? Is this really true ?

Question 2 :- What happens if I present the XviD encoder with Rec.709 video ? I know I have to use my eye, but I am interested in the more academic answer.

Question 3 :- Are my assumptions close to right ?

Thanks

[Wilbert]

Question 1 :- Does XviD REALLY require Rec.601 ?
XviD itself doesn't require anything.

Well if thats the case and I encode to HD resolutions with XviD wont it then playback using the wrong coefficients ? So on playback I would have to do a 601 to 709 conversion either with MPC or avisynth or whatever method I choose ? Is this really true ?
If it's feeded Rec.601, then yes.

Question 2 :- What happens if I present the XviD encoder with Rec.709 video ?
Then you have to correct it upon playback, or you have to live with it. Wrong levels upon playback is far more ennoying than wrong color coeffients, at least for me.

[madhatter300871]

Wilbert, thanks.

But the threads mentioned :-

http://forum.doom9.org/showthread.php?t=133982#post1090068

and :-

http://www.afterdawn.com/guides/archive/basic_dvd_authoring_project_part_3a_-_preparing_ntsc_page_5.cfm

...says that DivX/XviD/MPEG4 AVC expects Rec.601. I don't think this is actually true, do you ?

[hello_hello]

2. DivX/XviD/MPEG4 ASP codecs expect Rec.601, so really should be given Rec.601 material.

I think the "expects" thing only applies to encoding RGB, and XviD uses Rec.601 for the conversion. If I understand it correctly the same applies to x264.
So if I'm understanding it correctly now, XviD should be no different to x264 when encoding DVDs or BluRay etc. It just encodes the YUV video "as is". So if you encode R.709 video using either XviD or x264 it should encode using the original colors. Whether the encode plays with the correct colors is another matter. When it comes to a PC at least, it seems the color choice is always made by the renderer according to definition and oblivious to any colorimetry info in the video stream.

Which is actually fairly easy to confirm. If you open a video (either XviD or x264) with MPC-HC, maximise the window and use ffdshow to decode it, you can use ffdshow to resize the video "on the fly". The actual video display size won't change because the window is maximised, however if you use ffdshow to resize between SD and HD, you can see the colors change as you disable and enable the resizer.

If there's a media player/decoder/renderer for the PC which actually pays attention to any colorimetry info in the video stream, I'd like an introduction.

Whether the same applies to standalone devices or not, I don't know. I'd assume if there's no colorimetry info or they ignore it, they'd also choose the colors based on definition. It'd be kind of silly for a device to use R.601 simply because the video is XviD etc.

x264 can be fed with Rec.601 or Rec.709, choose depending on resolution.

I'm still inclined to think the same applies to XviD.

The decoder/renderer MIGHT obey headers, MIGHT not. The only thing you can really do is use your eye and decide if you need to do a Rec.709/Rec.601 conversion on playback on a per film basis.

It's probably safe to assume HD is R.709 and SD is R.601, but for h264 video at least, MediaInfo should be able to tell you which colorimetry is used, if the information is present. When it comes to mpeg video there's DGIndex.

Question 1 :- Does XviD REALLY require Rec.601 ? Well if thats the case and I encode to HD resolutions with XviD wont it then playback using the wrong coefficients ? So on playback I would have to do a 601 to 709 conversion either with MPC or avisynth or whatever method I choose ? Is this really true ?

With the exception of encoding RGB, I don't think XviD "expects" anything. To the best of my knowledge XviD doesn't write any color info to the video stream, so the colors used on playback would be totally up to the playback device.

Question 2 :- What happens if I present the XviD encoder with Rec.709 video ?

It'll just encode it. If there's no RGB conversion taking place, I'd assume the encoder simply re-encodes the YUV video completely oblivious to which colorimetry is supposed to be used when displaying it. I'd assume x264 is exactly the same.
If you encode a R.709 HD video using either encoder it should still display using R.709. If you resize down to SD it'll display using R.601 so you need to color convert while encoding.

One thing from your second link which confuses me:
http://www.afterdawn.com/guides/archive/basic_dvd_authoring_project_part_3a_-_preparing_ntsc_page_5.cfm

"Unlike other encoders, which actually check for colorimetry information in the video stream, CCE assumes the input video uses Rec.601....."

I'd like to know which encoders check for colorimetry info and in which type of video streams.
Most encoder GUIs use AVISynth and the way I understand it the original YUV video will be processed from start to finish as YUV so there's no need to worry about which color method is used (unless you change the resolution). It seems to me those guides are a bit misleading, or maybe they assume YUV to RGB conversion will be taking place (when using a VirtualDub filter or something similar) bringing about the need to worry about what the encoder might "expect".

This page:
http://www.afterdawn.com/guides/archive/basic_dvd_authoring_project_part_3a_-_preparing_ntsc_page_2.cfm
says DVD compliance requires the use of R.709, which I'm certain is totally incorrect, and is probably why it keeps mentioning color converting when using the CCE encoder which it says "expects" R.601. Why the CCE encoder should have to "expect" anything when working with YUV video I don't know, but maybe the guide just assumes RGB conversion will be taking place before encoding, or the person who wrote it doesn't really understand the whole thing properly either.

While I was doing some more reading I bumped into this old thread:
http://forum.doom9.org/showthread.php?t=131169

Wilbert,
Do you still work on the premise DVDs should be R.709?
I assume DGIndex defaulted to R.709 in the early days (if there's no colorimetry info in the video stream) but that was later changed to R.601, so I guess the general consensus now is that DVDs are usually R.601?

[Wilbert]

I assume DGIndex defaulted to R.709 in the early days (if there's no colorimetry info in the video stream) but that was later changed to R.601, so I guess the general consensus now is that DVDs are usually R.601?
Yes, that's correct.

"Unlike other encoders, which actually check for colorimetry information in the video stream, CCE assumes the input video uses Rec.601....."

I'd like to know which encoders check for colorimetry info and in which type of video streams.
I'm not aware of such encoders when provided compressed video. If you feed them AviSynth scripts, there's nothing to check since it delivers uncompressed video without header.

but for h264 video at least, MediaInfo should be able to tell you which colorimetry is used, if the information is present. When it comes to mpeg video there's DGIndex.
Provided the flags in the headers are set correctly.

If there's a media player/decoder/renderer for the PC which actually pays attention to any colorimetry info in the video stream, I'd like an introduction.
Perhaps players like Zoomplayer of BSplayer (sorry i forgot the precise name), but you will have to look that up.

I think the "expects" thing only applies to encoding RGB, and XviD uses Rec.601 for the conversion. If I understand it correctly the same applies to x264.
True. For x264 you can choose how the conversion is done.

@madhatter300871,

You need to differentiate between coders and specs. MPEG ASP or MPEG AVC (or MPEG2 for that matter) doesn't say anything about how the YCbCr samples should have been recreated (at least regarding colorimetry).

[hello_hello]

Wilbert,
Thanks for the info. One more question (or two) if I may..... just trying to make sure I understand the process.

If I'm encoding a R.709 video via AVISynth, and if AVISynth is feeding the encoder uncompressed video without any header.....
I know I start with a R.709 source and end up with a R.709 encode (at least in respect to both requiring R.709 conversion to RGB in order to display with the correct/same colors) but I'm just trying to understand what happens in between.

My understanding is that AVISynth uses R.601 "internally", but I assume that's only if you're using a plugin which requires RGB, and if it's feeding the encoder uncompressed video am I correct in assuming it generally stays as YUV from start to finish? Therefore there'd not be any color converting taking place and the uncompressed video being fed to the encoder would still be using the same colorimetry as the source? Is that correct?

Not that it's ever likely to effect me, but what if I happened to be using an AVISynth or VirtualDub plugin which requires RGB? Even if the source is R.709 I assume the video is converted to RGB using R.601, and at some stage in the chain it'll be converted back to YUV using R.601 (even if it's by the encoder), so I guess I'm asking if that is how it works, whether two wrongs make a right? Logically it seems if video is converted using the same incorrect colorimetry twice, then the colorimetry used for converting is irrelevant because you'd just end up back where you started, but I'm just curious if my logic there is correct.

I think I've probably confused myself in the past by thinking of uncompressed video and RGB as being one and the same, probably thanks to VirtualDub and the fact it outputs uncompressed video as RGB, but now it's finally sunk into my thick brain that uncompressed video doesn't have to be RGB, and can be YUV etc, I seem to be finally getting my head around this. At least a little bit....

Well I guess the abundance of freely available misinformation on the internet doesn't help either.
Thanks again for responding in this thread Wilbert. It's nice to have someone who seems to know what they're talking about provide real answers to questions asked for a change.

[madhatter300871]

Hello_Hello, yes it seems to be getting clearer for me as well. I think internet mis-information is what threw me down the wrong track also.

So, I think I now have the following :-

SOURCE
--------
1. Blu-ray (h264 & VC1) HD video usually uses Rec.709 coefficients and 16-235 levels.

2. Xvid/DivX/MPEG2 video usually uses Rec.601 coefficients and 16-235 levels.

ENCODING
----------
1. HCenc flags the output as Rec.709, so it really should be given Rec.709 material.

2. DivX, XviD (ASP) and X264 (AVC) codecs do not require to be fed a certain colorimetry source, they simply encode what they are given.

3. If an Avisynth filter is used that requires a colourspace change to RGB it will use Rec.601 coefficients for the conversion.

4. Virtualdub filters work in the RGB32 colourspace, and Rec.601 coefficients are used for the conversion.

5. avisynth "colourmatrix" filter should be used when converting from SD to HD and vice versa. This is only because the final playback device will use either Rec.709 or Rec.601 coefficients when converting the YV12 data to RGB.

6. During the encoding phase, we only need to be aware what coefficients are used if doing an RGB colourspace conversion somewhere in the chain or when performing a resolution change.

DECODING
----------
1. When playing back on a PC, the decoder/renderer can map levels to 0-255 if configured to do so.

2. The decoder/renderer probably doesn't obeyheaders. If in doubt about the colours in the output and coefficients are suspected the only thing you can really do is use your eye and decide if you need to do a Rec.709/Rec.601 conversion on playback on a per film basis.

3. ffdshow can map levels using the "levels" option.

4. For final output onto a digital screen (be it LCD monitor, plasma, projector) a conversion to RGB is always done at some point by some process.

5. A vertical resolution of 720 and above is considered to be HD.

6. A vertical resolution of 719 and below is considered to be SD.

7. If Rec.709 is decoded using Rec.601 coefficients, expect colours to be darker.

8. If Rec.601 is decoded using Rec.709 coefficients, expect the colours to be lighter.

Point 5 & 6 are open to much debate, with different renderers and different players assuming different values... so testing could be done by a user on a per system basis, find out what works for you.

Question 1 :- As Hello_Hello asked, if we perform a colourspace change to RGB on a Rec.709 source, do some processing and convert back to YUV, using Rec.601 coefficients each time does the output have the same colour or have we converted to Rec.601 twice and hence made the colours much darker ?

Thanks

[Wilbert]

If I'm encoding a R.709 video via AVISynth, and if AVISynth is feeding the encoder uncompressed video without any header.....
I know I start with a R.709 source and end up with a R.709 encode
Yes sure.

My understanding is that AVISynth uses R.601 "internally"
Nope. If you do a color conversion in AviSynth (because a plugin or an encoder requires a certain colorspace) you can select the color conversion coeffients that should be used: http://avisynth.org/mediawiki/Convert

and if it's feeding the encoder uncompressed video am I correct in assuming it generally stays as YUV from start to finish?
Yes, if your source is YCbCr and there is no conversion to RGB somewhere. Indeed uncompressed video can be YCbCr. Even Virtualdub can save to uncompressed YCbCr nowadays.

Logically it seems if video is converted using the same incorrect colorimetry twice, then the colorimetry used for converting is irrelevant because you'd just end up back where you started, but I'm just curious if my logic there is correct.
Yes, your logic is correct here.

@madhatter300871,

1. Blu-ray (h264 & VC1) HD video usually uses Rec.709 coefficients and 16-235 levels.
Like is said above, h264 and VC1 are standards and don't require anything, so above is not true. At most you can say that encoders "expect" this or that. When encoding with x264 for example you can specify the colorimetry and levels, so it doesn't "expect" anything either.

2. Xvid/DivX/MPEG2 video usually uses Rec.601 coefficients and 16-235 levels.
Ditto. That's only the case if you feed the Xvid/DivX encoders with RGB. MPEG2 is a standard, so it doesn't expect anything.

2. DivX, XviD (ASP) and X264 (AVC) codecs do not require to be fed a certain colorimetry source, they simply encode what they are given.
If you feed them YCbCr then yes, they encode what is given. If you feed them RGB it will convert to YCbCr in some way. Btw, not all encoders take RGB (QuEnc requires YCbCr for example).

. If an Avisynth filter is used that requires a colourspace change to RGB it will use Rec.601 coefficients for the conversion.
Nope. IF an AviSynth filter performs a colorspace conversion, then it depends on the filter how it is performed and if you can choose how to. However usually you do the colorspace conversion using the standard AviSynth functions where you can select the colorimetry (see second answer).

4. Virtualdub filters work in the RGB32 colourspace, and Rec.601 coefficients are used for the conversion.
That used to be true in the old days. Nowadays filters can work in YCbCr if they are programmed that way. There are standard filters for Rec.709<->Rec.601 conversions. You willl have to read the documentation of Virtualdub to find out all the possibilities.

The rest is more or less true.

[hello_hello]

I guess I've confused the issue or added to the misinformation available by bringing up the topic of AVISynth and VirtualDub filters.

For the record I probably assumed AVISyth uses R.601 "internally" after reading something like this: http://avisynth.org.ru/docs/english/externalfilters/colormatrix.htm
"Background information
There are several ways to convert a YUV stream to RGB. The most well known one, uses Rec.601 coefficients. It is for example used in the color conversion routines of AviSynth, VirtualDub and XviD/DivX."


That used to be true in the old days. Nowadays filters can work in YCbCr if they are programmed that way. There are standard filters for Rec.709<->Rec.601 conversions. You willl have to read the documentation of Virtualdub to find out all the possibilities.

I really only use VirtualDubMod for basic remuxing etc and I guess I should break that habit and move to using VirtualDub again instead.

For the purposes of this discussion the subject of VirtualDub filters is probably not relevant and just confusing the issue and as you said most filters these days probably aren't "RBG only" any more.

[madhatter300871]

H264 and VC1 are standards, OK. But what I suppose I'm getting at here is that the concept of levels and colorimetry must exists mustn't it ? So is it not a good assumption that a commercially available bluray disc with HD video content should "usually" be decoded to RGB using Rec.709 colorimetry and luminance levels 16-235 as it has "probably" been encoded this way ? And likewise commercially available DVD with SD video content should "usually" be decoded with Rec.601 and 16-235 ?

Thanks for pointing out that I can choose which coefficients to use during an avisynth colourspace conversion ..... heaven knows how I missed that, I have read the avisynth docs and could slap myself for not seeing that !

[hello_hello]

Here'd be my summary of all the average person (like myself who encodes using GUIs such as MeGUI) needs to worry about when it comes to colorimetry.

HD video generally requires R.709 color coefficients when decoding for playback.
SD video generally requires R.601 color coefficients when decoding for playback.

When encoding from HD to SD resolutions R.709 to R.601 color conversion should be used.

Windows renderers display SD video correctly using R.601
Windows renderers display Full HD video correctly using R.709.
In the case of cropped 720p encodes... anything with a vertical resolution of less than roughly 678 pixels.... Windows will display them incorrectly using R.601 instead of R.709. This can be corrected on playback using MPC-HC's BT.601 to BT.709 shader.

(A note on the above.... I haven't worked out the exact cut-off point where Windows switches from R.709 to R.601 but it's not 720 vertical lines as seems to be the consensus. It's slightly less than 688 vertical lines and I suspect it might change a little according to the video's width as well. I do know for sure... 100% positive... that my 1280x688 720p encodes display using R.709 as they should, while 1280x544 720p encodes display incorrectly using R.601)

For those videos which don't follow the general HD/SD rules.....

When converting DVDs/mpeg2 video most encoder GUIs have a color correction option which automatically converts from R.709 to R.601 if necessary using DGIndex and the colormatrix AVISynth plugin. If the colorimetry info isn't present R.601 should be assumed (at least for DVDs) and the color correction option will have no effect.
When converting h264 HD video MediaInfo should tell you which colorimetry is used if the information is present in the video stream, otherwise assume R.709.

A note regarding MeGUI and it's automatic color correction option for mpeg2 video (and the same probably applies to other encoder GUIs such as AutoGK which automatically correct the colors in the same way)....
Automatic color correction assumes encoding from SD to SD. When encoding HD mpeg2 video (not resizing down to SD) the automatic color correction will either do nothing or convert to R.601 incorrectly, so it should be disabled when encoding from HD mpeg2 to HD. Maybe one day MeGUI will clever enough to enable/disable the color correction option according to the output resolution.

[Groucho2004]

I haven't worked out the exact cut-off point where Windows switches from R.709 to R.601 but it's not 720 vertical lines as seems to be the consensus.

Slightly confused. Which layer/component of Windows would switch color spaces, even bypassing the display driver?

[madhatter300871]

Hello_Hello.

Did you mean your 1280x544 encodes display using Rec.601, which is incorrect, and they should really be using Rec.709 ?

Sorry for being confused by that, I'm having one of those days !

Also, for your summary, what is your position on 16-235 versus 0-255 ? I have my card outputting 16-235 levels and my projector set to 16-235 input (I can set my projector to accept 16-235 or 0-255). Should this not also be set to match what the video is encoded at, which I am beginning to think is 16-235 99% of the time. And if I do set my card to output 0-255 what is it actually doing ? Is it :-

A. Re-mapping colours, so video level 16 is changed to video level 0, video level 235 is changed to video level 255 and everything in between is scaled accordingly ?

or

B. Video level 16 no longer represents black, it represents a slightly lighter black (well, level 16, as 0 is now black) and 235 no longer represents white, it represents a slightly darker white (well, level 235 as 255 is now white). The net result being blackest black and whitest white are no longer displayed as levels 0-15 and 236-255 are not actually present in the video stream.

Does that make sense ?

[Groucho2004]

Found this (http://www.codecguide.com/faq_display_issues.htm) FAQ which seems quite useful.

[hello_hello]

Slightly confused. Which layer/component of Windows would switch color spaces, even bypassing the display driver?

The renderer??
I don't know exactly how it all works, but I do know when playing video with MPC-HC and the WMR9 renderer, there's a cut-off point where the color space changes. I thought it was pretty well accepted as fact.
There's no conversion to RGB anywhere else in the chain which I'm aware of (in my case) so I assume it's the renderer which makes the colorimetry choice.

Are you saying the color space used doesn't change on your PC according to the video definition?

In my case I generally use MPC-HC and it's internal filters for watching video, but when using ffdshow to do the decoding (no conversion to RGB taking place) it's easy enough to test. I posted some info earlier post explaining how to maximise the MPC-HC window, use ffdshow to resize the video from a HD resolution to a SD resolution (or the other way around) and how you can then watch the colorimetry being used change as you enable/disable the ffdshow resizer.

[Groucho2004]

I don't know exactly how it all works

Me neither. I simply found the term "Windows" too simplistic as a reference.

[hello_hello]

Hello_Hello.

Did you mean your 1280x544 encodes display using Rec.601, which is incorrect, and they should really be using Rec.709 ?

Yes. The windows renderer, or whichever part of the chain makes the colorimetry decisions, changes the colorimetry according to the video's vertical resolution. The accepted cutoff point (it's mentioned in numerous threads in this forum) seems to be 720 vertical lines. Less than 720 vertical lines and R.601 is used.
I'm not disagreeing as to whether or not this happens, just where the actual cutoff point is.

It'd seem reasonable to assume 720p is high definition (because it is) but in the case of encodes where the black bars have been cropped you've got less than 720 vertical lines so Windows uses R.601. ffdshow (if you use it to decode and convert to RGB) is far more likely to get it right as it basis colorimetry on the video's width as well. With ffdshow, anything over 1024 pixels wide is olso considered to be HD regardless of vertical resolution, so basically anything with a resolution greater than that of an anamorphic DVD will be converted to RGB using R.709.


Also, for your summary, what is your position on 16-235 versus 0-255 ?

When it comes to encoding, you never have to think about it, which is why I left it out.
All DVD/Bluray video uses 16-235. That's the range of levels used and has nothing to do with whether the video is R.709 or R.601 as both can either use standard/TV levels or full range/PC levels. They always use standard levels.

I have my card outputting 16-235 levels and my projector set to 16-235 input (I can set my projector to accept 16-235 or 0-255). Should this not also be set to match what the video is encoded at, which I am beginning to think is 16-235 99% of the time.

I don't think it really matters. If your video card is set to "expand" the levels to 0-255 and the display device is set for an input of 0-255, the video should display in exactly the same way as it would if the video card wasn't expanding the levels and the display device input was set for 16-235.

Re-mapping colours, so video level 16 is changed to video level 0, video level 235 is changed to video level 255 and everything in between is scaled accordingly ?

The above. Except it's not re-mapping colors as such it's remapping or "expanding" the luminance levels from 16-235 to 0-255.
The way I understand it the difference between R.601 and R.709 is they're just different standards for storing luminance levels so they've not actually got anything to do with colors as such, except as they're slightly different standards if you encode video using one and decode using the other it effects the luminance with which colors are displayed, some more than others, or something like that.... but either standard can use the 16-235 range of levels or the 0-255 range of levels, however virtually all video uses 16-235.

Is the same video card also driving your PC's display? If it is, can you set the output for the PC display and the projector independently?
If you can't it might be better to set the card to expand the levels to 0-255 and set the projector's input the same. Your PC monitor should be getting 0-255 levels for video so if you use 0-255 for everything then everything should display correctly.

[hello_hello]

Did you mean your 1280x544 encodes display using Rec.601, which is incorrect, and they should really be using Rec.709 ?

On that subject again, I did briefly consider converting my cropped 720p encodes to R.601 where applicable so they'd display correctly on my PC, which is my main playback device.
But then I considered the likelihood of the average standalone player being more clever than Windows which would mean those color converted encodes would display incorrectly using a standalone player.... so in the end I decided against color converting and I just use the MPC-HC shader to convert to R.709 on playback. I've just got to remember to enable/disable it manually.

1280x688 encodes definitely display using R.709 even though they fall below 720 vertical lines. 1280x544 encodes display using R.709. Wherever the cutoff point is between those resolutions probably doesn't matter too much as I'm not sure I'd have any encodes with a vertical resolution between 544 and 688 anyway.

[madhatter300871]

I have a dedicated PC to drive the projector, so I dont bother connecting it to a monitor.

Are you suggesting that video played back onto an LCB panel (monitor/LCD TV/LCD projector etc) should always be played back full scale ?

Im not too bothered about my desktop not being quite right colour wise, just want to do all I can to get video play back as faithfully reproduced as possible. Is there any benefit to playing back video at PC levels ?

I can't use the MPC-HC pixel shader option as I am using DXVA to playback my x264 encodes, and they don't work when DXVA is being used.

[hello_hello]

Are you suggesting that video played back onto an LCB panel (monitor/LCD TV/LCD projector etc) should always be played back full scale ?

It probably depends on the input being used and the levels it expects. That's why I wondered earlier if the "Bluray using 0-255 levels" thing, which I've heard mentioned before, has more to do with the input/output of the TV/Player than is has to do with the levels used for the video itself.

Is there any benefit to playing back video at PC levels ?

I don't know for sure, but I doubt it.

I can't use the MPC-HC pixel shader option as I am using DXVA to playback my x264 encodes, and they don't work when DXVA is being used.

Well I am too and I do..... ;)
Which renderer are you using? You need to use either the WMR9 or EVR renderer and have 3D surfaces enabled... or whatever that option is called.... I think it defaults to 2D.
I'm using DXVA for playback and the MPC-HC shaders work fine.

[madhatter300871]

Pixel shaders work when using DXVA ? Well again .... it seems I have this wrong. I must admit I did so much tweaking and testing to get to where I am now that I must have just plain been confused about this. Great .... I'll try that again tonight.

So my latest summary :-

SOURCE
----------
1. Blu-ray (h264 & VC1) HD video usually uses Rec.709 coefficients and 16-235 levels.

2. Xvid/DivX/MPEG2 video usually uses Rec.601 coefficients and 16-235 levels.

ENCODING
----------
1. HCenc flags the output as Rec.709, so it really should be given Rec.709 material.

2. DivX, XviD (ASP) and X264 (AVC) codecs do not require to be fed a certain colorimetry source, they simply encode what they are given.

3. When performing colourspace conversions with avisynth, you can choose which coefficients to use and so you need to be aware what your source uses.

4. If you convert using the wrong coefficients, and convert back again using the same wrong coefficients, you end up with the correct colours still.

5. avisynth "colourmatrix" filter should be used when converting from SD to HD and vice versa. This is only because the final playback device will use either Rec.709 or Rec.601 coefficients when converting the YV12 data to RGB.

6. During the encoding phase, we only need to be aware what coefficients are used if doing an RGB colourspace conversion somewhere in the chain or when performing a resolution change.

DECODING
----------
1. When playing back on a PC, the decoder/renderer can map levels to 0-255 if configured to do so. Output levels should be set to match display input levels.

2. The decoder/renderer probably doesn't obey headers. If in doubt about the colours in the output and coefficients are suspected the only thing you can really do is use your eye and decide if you need to do a Rec.709/Rec.601 conversion on playback on a per film basis.

3. ffdshow can map levels using the "levels" option.

4. For final output onto a digital screen (be it LCD monitor, plasma, projector) a conversion to RGB is always done at some point by some process.

5. A vertical resolution of 720 and above is considered to be HD.

6. A vertical resolution of 719 and below is considered to be SD.

7. If Rec.709 is decoded using Rec.601 coefficients, expect colours to be darker.

8. If Rec.601 is decoded using Rec.709 coefficients, expect the colours to be lighter.

Point 5 & 6 are open to much debate, with different renderers and different players assuming different values... so testing could be done by a user on a per system basis, find out what works for you. Hello_Hello suggest 688 vertical resolution seems to be his trigger point.

[J_Darnley]

7. If Rec.709 is decoded using Rec.601 coefficients, expect colours to be darker.

8. If Rec.601 is decoded using Rec.709 coefficients, expect the colours to be lighter.

Haven't you got this confused with levels?

[madhatter300871]

Hi.

Nope, not got it confused with levels. If you decode your YUV data to RGB using incorrect coefficients, colours can appear dark or washed out.

Using incorrect levels also screws up the final decoded image.

[hello_hello]

7. If Rec.709 is decoded using Rec.601 coefficients, expect colours to be darker.

8. If Rec.601 is decoded using Rec.709 coefficients, expect the colours to be lighter.

We're probably debating terminology now, but I don't think the "lighter" or "darker" description is very accurate even though I've read those terms used as the effect of incorrect colorimetry. In fact even AutoGK's help files use the term "darker" to describe encodes of R.709 video without the use of color correction.

After a lot of experimenting though (if you get MPC-HC's shaders working it'll be easy to use the R.601 to R.709 shader to look at the effect it has) I wouldn't use the terms darker or lighter. It seems to be mostly reds and greens which are effected but depending on the video the difference between the two color standards is sometimes barely noticeable, if at all, while sometimes it's really obvious.
If memory serves me correctly, reds tend to get darker while greens tend to get brighter, or reds get brighter while greens get darker, depending which colorimetry is used.

Hello_Hello suggest 688 vertical resolution seems to be his trigger point.

It's slightly less than 688 because 1280x688 encodes display correctly using R.709. I'd be astounded if my trigger point is any different to that of anyone else's PC. I'd be willing to bet everyone's 1280x544 encodes display using incorrect (R.601) colorimetry when played on a PC but most people probably don't notice. I know I didn't notice myself for a long time. It wasn't until I tried to work out why some SD encodes taken from HD sources seemed not to need color correction when compared to the source while others did. As I usually make SD encodes using 720p encodes as the source (because I use a second PC for the SD encodes and it's much quicker to transfer/encode a 3GB file than it is a 40GB one) I backtracked further and started comparing both encodes to the original Bluray video. It was then I discovered some of the 720p encodes were displaying using the wrong colorimetry. From there I worked out the resolution thing....
Of course once I worked it out and finished banging my head against the desk I thought it only fair if I couldn't remain blissfully unaware of the wrong colorimetry being used when playing 1280x544 encodes, I should share my misery with others. ;)

[leeperry]

5. A vertical resolution of 720 and above is considered to be HD.

6. A vertical resolution of 719 and below is considered to be SD.
more like x<1025=SD, x>1024=HD, coz all the <1.78AR 1280*___ stuff out there is HD.

7. If Rec.709 is decoded using Rec.601 coefficients, expect colours to be darker.

8. If Rec.601 is decoded using Rec.709 coefficients, expect the colours to be lighter.
601>709 looks greenish and 709>601 looks washed out: http://thumbnails39.imagebam.com/13677/18b89a136763540.jpg (http://www.imagebam.com/image/18b89a136763540)

[madhatter300871]

It is usually only noticeable with reds and greens, but in my case it is never "hardly noticeable", it's quite distracting for me. I do concede that while red gets darker green gets lighter and vice versa ... never noticed that before as it is usually the reds that jump out to my eye, and the general feeling of washed out colours.

Well i think one thing is for sure, it is important that I pay attention to colorimetry of the source and desired colorimetry of the encode depending on what resolution my encode ends up at.

I don't think its the horizontal resolution that the media playback chain looks at when decided if it should use Rec.709 or Rec.601 when playing back video, I am still pretty certain it's vertical resolution. Using FFdshow you can set horizontal/vertical parameters to watch out for but I don't think renderers do that. Im not an expert though, it's just been my observations.

[hello_hello]

It is usually only noticeable with reds and greens, but in my case it is never "hardly noticeable", it's quite distracting for me.

I guess you don't play any 1280x544 encodes then? ;)

I don't think its the horizontal resolution that the media playback chain looks at when decided if it should use Rec.709 or Rec.601 when playing back video, I am still pretty certain it's vertical resolution. Using FFdshow you can set horizontal/vertical parameters to watch out for but I don't think renderers do that. Im not an expert though, it's just been my observations.

I did a little more playing around using ffdshow to resize the video and now I'm sure horizontal size comes into it.

I started with a 1280x544 encode and played it in a maximised window. I used ffdshow to increase the width while maintaining the aspect ratio (so the hight would automatically increase accordingly). I could reliably reproduce a switch from R.601 to R.709 colorimetry at a width of 1360, which made the resolution 1360x578.
Next I reduced the height to 576 while leaving the width the same and it switched back to R.601. From there I increased the width while leaving the height at 576 but no matter how wide I made the video it didn't switch to R.709 until I increased the height to 578. So 578 pixels seems to be the minimum height for R.709.

I then tried a second encode, this time with a resolution of 1280x720. While maintaining the aspect ratio I could reliably produce a change to R.601 at 1198x672. 1200x674 would always display using R.709.
This time I left the height at 672 and increased the width. A 2 pixel increase to 1200 would get the video to display using R.709.

So from the above I've tentatively concluded that Windows will always display using R.601 if the width is less than 1200, regardless of the height, and it'll always display using R.601 if the height is less than 578, regardless of the width.

Or to put it another way, for video to display using R.709, width must be equal to or greater than 1200 AND height must be equal to or greater than 578.
At least that's how it appears to be working for me.

I did however, stumble across a bit of a solution. If you resize video using ffdshow but select the option to maintain the aspect ratio, ffdshow will add black bars to the video where applicable. So if you're playing a 1280x544 encode for example, and you resize to 1280x720 while selecting the "keep aspect ratio" option, ffdshow will output 1280x720 video with black bars top and bottom and the renderer will happily display it using R.709.

For my display I've now got it set to resize everything over a width of 1150 and a height of 520 to 1280x720, so pretty much anything over DVD definition should now display using R.709 (seems you need to disable the "process pixel ratio internally" option though or it messes with anamorphic video).

The only downside to using ffdshow to automatically resize is you can't use DXVA for decoding as well..... sigh. Myself, I'd prefer the correct colors though over worrying about whether it's the CPU of the GPU decoding the video.

Edit: I should add, I'm using XP. Whether the newer renderers in newer versions of Windows work from a different rule book, I can't say.

[madhatter300871]

I do play 1280x544 encodes yes, and others. What I was saying was that it is never "hardly noticeable", meaning that is is always "very noticeable". I perhaps should have written it better :)