Some thoughts on a new colorspace.

[General Lee D. Mented]

I've been thinking about colorspaces and some ideas have come to me.

I'm bothered by how primitive most colorspaces are. Most people will agree chroma data is less important than luma so scaling the chroma down is still probably a good idea. However, just scaling it by 2 on one or both axis is very... crude to me. Also, YUV has problems with pure red or pure blue objects, because in these cases you only have one differential channel active and it can lead to some annoying artifacts.

So, since cpu power is much cheaper these days it may now be feasible to create a colorspace with some improvements. I'm thinking about things like arbitrary chroma size, that's downsampled with a bicubic resample instead of a crude 2:1 or 4:1 scale.

Also, I think instead of a YUV space that's luma, differential red, differential blue, a space that's luma, differential magenta, differential cyan would produce better results. The worst case scenarios of pure magenta and pure cyan are far less common in natural images than pure red and pure blue and thus you'd be less likely to see chroma subsampling artifacts.

I know that adding a colorspace would mean all sorts of issues like conversion functions etc, but I think the improvement in image quality may be worth it.

Any thoughts?

[JJ]

Hi,

aren't cyan and magenta just (white - red) and (white - green) respectively? I believe you'll end up with negated chroma values, but I'm not sure about it.
Also note that many video sources (DVDs) have already subsampled chroma, so changing the color space might instead of improving the image quality just introduce new artifacts.

Regards,
JJ

[Joe Fenton]

Primary additive colors: Red, Green, and Blue
Secondary additive colors: Magenta, Yellow, and Cyan

Primary subtractive colors: Magenta, Yellow, and Cyan
Secondary subtractive colors: Red, Green, and Blue.

Look at any color wheel and you'll see the relationship. People base the color spaces for VIDEO on differentials of Red and Blue because they are the primary colors for additive colors (TVs and monitors). Painters and printers use subtractive colors.

I doubt you'll gain anything by using Magenta and Cyan in place of Red and Blue.

BTW, differentials of Red and Blue are used instead of red/blue and green because the eye is less sensitive to red and blue than to green. You want full bandwidth for green. If you give colors reduced bandwidth, pick the colors that the eye is less sensitive to - red and blue. Now that would be an interesting experiment... instead of Luma/Red-diff/Blue-diff, maybe someone should try just plain Green/Red/Blue. Wonder how that would look. Anyone ever try that? No color conversions needed, just scale the red and blue.

[Joe Fenton]

Oh, just as an aside to my last comment, most plasma flat panel displays use twice the number of green pixels compared to blue and red pixels. This cuts down the number of cells that need to be printed. Besides, the eye will never pick that up at normal viewing distances.

A typical plasma cell layout is
g r
b g

Repeat across the entire panel. So if it's okay for flat panels, maybe it would be okay for a video codec.

[mf]

Originally posted by Joe Fenton
BTW, differentials of Red and Blue are used instead of red/blue and green because the eye is less sensitive to red and blue than to green. You want full bandwidth for green. If you give colors reduced bandwidth, pick the colors that the eye is less sensitive to - red and blue.
I will have to disagree. Chroma subsampling artifacts, to me, are much more visible on saturated red and blue, than on green.
@Gldm - I've been thinking about a HSV colorspace where Lightness and Hue aren't subsampled, and where only saturation is subsampled.

[MfA]

Compressibility is as much an issue as perceptual uniformity.

For instance Marcus Nadenau (http://dewww.epfl.ch/~nadenau/Research/publication.htm) has compared YUV against CIE Lab (and the opponent color space) in JPEG-2000 and YUV came out the clear victor.

Some links you might want to check out :

This page (http://www.compuphase.com/cmetric.htm) on a metric to determine colour distances.

This paper (http://www.site.uottawa.ca/~edubois/projects/chromavq.ps) on removing non-linearities from the RGB color components (resulting from gamma correction and the capture devices themselves) before converting it to a luminance+chrominance colour space and compression through vector quantization. More even chromatic components to me seem helpfull for any kind of compression though.

[General Lee D. Mented]

Originally posted by mf
I will have to disagree. Chroma subsampling artifacts, to me, are much more visible on saturated red and blue, than on green.
@Gldm - I've been thinking about a HSV colorspace where Lightness and Hue aren't subsampled, and where only saturation is subsampled.


Well if you can get a function for say RGB->HSV or 4:4:4 YUV->HSV, then it shouldn't be too hard. The problem will be finding an input source that's not already chroma-subsampled, unless you start scaling things down to 1/4 res and going from there.

No problems using any colorspace in WARP though, it absolutely does not care and will return exactly what space the input was.

[Joe Fenton]

Originally posted by mf
I will have to disagree. Chroma subsampling artifacts, to me, are much more visible on saturated red and blue, than on green.


Maybe on a still shot where your nose is against the screen. If you've ever run across a blue sign at night, you'll notice (in my experience and anyone I've ever asked) that those signs are virtually unreadable. So light levels also play a huge role in the perception of blue and red. Not to mention, green isn't subsampled, per se, so of course you wouldn't notice artifacts on saturated green.

Originally posted by mf
@Gldm - I've been thinking about a HSV colorspace where Lightness and Hue aren't subsampled, and where only saturation is subsampled.

Now that sounds like a good idea.

[General Lee D. Mented]

Been looking at some deconstructed HSV channels of images online. It seems to me that HSV will potentially compress better than YUV, at least with my code. The H channel has lots of flat runs that the wavelet transform will compress very highly. Saturation has somewhat more detail but probably not as much as either U or V does, and if it is subsampled or preferably bicubic scaled, it should also produce some compression savings. As for the value channel, it seems pretty much identical to the regular luma Y channel.

[mf]

I've been testing out some things on HSL. Both the Hue and Saturation channels seem to be very smoothable. And since I've always been told wavelets smooth the image, that would mean they compress very well (or at least, it would be usable as a preprocessor, to lower the size of those planes, like subsampling does for MPEG). Since I don't have matlab or similar tools, I wasn't able to test YUV subsampled, so I'll just keep it at a visual test. I simply separated the HSL channels, smoothed them (quite heavily I may add) with a smart smoother, and reconstructed the RGB image. It may be nice to note that the RGB image which has undergone HS smoothing has a lower PNG filesize (I compressed the files with pngcrush -l 9 -brute) than the original, while there is almost no visible difference. This is lossless PNG compression, and I think Gldm's wavelets will prove MUCH more effective.

http://mf.creations.nl/HSLtest/RGB-origcrushed.png
http://mf.creations.nl/HSLtest/RGB-HSLcrushed.png

Of course these still aren't real results on wavelet compression of HSL images, but I think they can give an indication of what kind of distortion to expect (or at least of how we can preprocess almost invisibly). Also, afaik HSL is a reversible colorspace (no roundoff problems with back-and-forth converting to RGB), correct me if I'm wrong, but that'd be a nice plus. Oh, and the original was a JPEG, so the H and S channels were already pretty screwed up. I think a clean source will smooth better.