Color variations between source and x264 output in QuickTime Player and others

[chaynik]

Image on the left is uncompressed 8-bit 4:2:2 and on the right is the output of x264. While the encoding quality is excellent, notice that it is somewhat paler than the source, especially visible in the tan colors of the table. The same video encoded with Apple's Compressor has colors looking exactly like the original. VLC also plays back the file with pale colors.

On the Windows side of things, QT and VLC behave the same, with only the CoreAVC decoder playing back the brighter more vibrant colors. I thought at first it was an x264 issue, but CoreAVC's correct behavior makes me wonder...

Thanks in advance!

[Dark Shikari]

TV vs PC luma levels, probably. Make sure you signal the luma range of the source correctly and that your decoder obeys that signal.

[ToS_Maverick]

and also keep the 601 vs 709 colorspace in mind!

whats the colorspace of your source?

[Blue_MiSfit]

Also, do you see the horribly upsampled chroma on the uncompressed version (badly aliased red on the flower/stem edge)?

~MiSfit

[LoRd_MuldeR]

Originally Posted by Dark Shikari

Make sure you signal the luma range of the source correctly and that your decoder obeys that signal.

If your decoder doesn't do the YUV -> RGB conversion in software, but outputs YUV data to the renderer, then you must rely on the renderer to use the correct levels.

Especially VMR7, VMR9 and EVR are known to have problems. Haali's renderer has options to set the desired levels. No idea about QuickTime's renderer...

[CruNcher]

Originally Posted by Blue_MiSfit

Also, do you see the horribly upsampled chroma on the uncompressed version (badly aliased red on the flower/stem edge)?

~MiSfit

yep the bad up sampling is very visible compared vs the output decode, actually you see it everywhere in the RED also in the background

[Mr VacBob]

QT obeys the colorspace atoms ('colr') in the mov/mp4 file if they're present, but will otherwise make one up that results in different YUV conversion than other players.

I'm not sure if the official H264 decoder respects h264 color signalling, but it might.

[Sharktooth]

craptime(tm) does it in his own (wrong) way... always.
drop it.

[chaynik]

Thanks for all the replies, guys. I suspected this had something to do with PC vs TV luma ranges. The question is, should I correct these prior to encoding (by using ColorYUV in AviSynth for example)?

As far as 601 vs 709 differences, how would I be able to tell? My source is a DigiBeta ingested over SDI directly into 2vuy/UYVY.

I realize many have their gripes with QuickTime, but my encodes are to be played back in software that uses QT for rendering, so I do not have a choice. Plus, QT is not the only one with these inconsistencies, VLC is the same. I guess the real question is, which is the "correct" way to display this image.. The one on the left or the right? Would I be able to tell by using something standarized, such as SMPTE colorbars?

As far as the QT colorspace atoms, how would I be able to edit those or even find out their value? I tried various VUI settings in x264, but they seemed to have no effect. Does QT only obey the 'colr' atoms and not the VUIs?

[chaynik]

Anyone?

[Dark Shikari]

The best way to resolve it is to figure out what Quicktime assumes the video to be (colormatrix and lumarange wise) and convert your source into that before x264 encodes it.

[Blue_MiSfit]

Exactly.

I find that QuickTime expects PC range luma (full). So, I convert it to that before encoding anything targeted at QuickTime.

Not sure about the matrix, I have a hard time deciding which one is correct in most cases.

~MiSfit

[BlackSharkfr]

Check in your video driver settings what the hardware YUV->RGB conversion setting is. For me it always defaults to 16-235 (limited range) everytime i restart the computer (nvidia control center issue)
Which means no matter what the decoder did, when the gpu grabs the image in YUV form it always converts it to the 16-235 range.

I have two solutions on my side :
Solution 1 : switch the range to 0-255 (full range) in your display driver settings (check in the video colour settings)
If you have an nvidia gpu, this setting is in the control center under Video & TV / video colour settings in the advanced tab (rough translation from my localized driver)
I don't have an Ati gpu so I can't say where this option is located in the Catalyst control center.
EDIT : i made a clean driver reinstallation and no longer have the following bug
problem on my side : it switches back to limited range everytime i restart my computer, i can't figure out why.


Solution 2 : check in your player and codec settings and make sure they all do the YUV->RGB conversion internally.
some players let you change this (VLC), others don't (MPC)
In this case you have to make sure your codecs internally output to RGB instead of one of the YUV outputs (YV12, YUY2, etc...), FFDshow allows you to do this and CoreAVC as well.
And i don't know about quicktime, i don't use it.

[Lenchik]

Originally Posted by BlackSharkfr

make sure your codecs internally output to RGB instead of one of the YUV outputs (YV12, YUY2, etc...), FFDshow allows you to do this

How to turn this option on in ffdshow?

[poisondeathray]

Originally Posted by Lenchik

How to turn this option on in ffdshow?

In the output options, uncheckmark YV12, YUY2, YVYU, UYUV, etc...

Checkmark "high quality YV12 to RBG conversion" if you don't want that bad sampling (looks like checkers in reds, similar to the left hand picture in the 1st post) - this take more CPU usage, so if you have a slow PC, maybe not a good idea

In the RGB conversion subheading, select your YCbCr specification (e.g. BT.601 vs BT.709), and contrast (e.g. full range vs standard)

Don't forget to turn it "off" and reset it if you are doing encoding using ffdshow to decode the source, or you might get unwanted filtering and several colorspace conversions (quality loss)

[Blue_MiSfit]

Or use Haali renderer and have full control of these options at that level. I find it much easier to do this instead of digging through ffdshow.

Note that QuickTime does _not_ let you change these things. Yet another reason it sucks on Windows

~MiSfit

[BlackSharkfr]

I made i clean video driver reinstallation and the nvidia control pannel now works correctly and remembers my luma range settings.
I finally have the full range again like back in the days of the old 84.xx drivers

[chaynik]

Originally Posted by Mr VacBob

QT obeys the colorspace atoms ('colr') in the mov/mp4 file if they're present, but will otherwise make one up that results in different YUV conversion than other players.

I'm not sure if the official H264 decoder respects h264 color signalling, but it might.

Are there any tools that can edit the colr atom so I could set it to the correct range? That would seem like the best solution!

[refulgentis]

Originally Posted by chaynik

Are there any tools that can edit the colr atom so I could set it to the correct range? That would seem like the best solution!

mp4tracks in the new libmp4v2 will do this: http://code.google.com/p/mp4v2/

[chaynik]

Originally Posted by refulgentis

mp4tracks in the new libmp4v2 will do this: http://code.google.com/p/mp4v2/

Thank you for the suggestion! Is there a compiled version of this available anywhere? I unfortunately have no idea how to build something like this...