Why is subsampling chroma better than just assigning less bits to CRCb in 4:4:4?

[Electron.Rotoscope]

I was having a discussion with someone about the difference between 4:2:0 and 4:4:4 and trying to explain that 4:2:0 became the standard for non-interlaced content on the web because luma should have a higher proportion of the bandwidth than chroma etc etc, but it got me thinking:

If I have an original image that's say 1920x1080, why is it better to record the Cr and Cb parts of the image at 960x540 instead of just assigning fewer bits to the chroma per frame and keeping it 1920x1080? Obviously it wouldn't work in RGB 4:4:4, but why not YCbCr 4:4:4? Is this just a holdout from the past when decode was still a bottleneck and it's easier to decode 4:2:0 because fewer pixels?

I suppose this is a bigger question I've never really understood: when is it better to resample the image vs encoding at low bitrate but the original resolution? I've seen sometimes people take a UHD video stream and make it HD before encoding it because they were using a very low bitrate, but I've never understood where that tradeoff should be made. Is that also an ease of decode thing?

[Sunspark]

I can't answer this, but I can contribute an observation. Bandwidth of terrestrial antenna broadcast likely plays a role here. While 720p broadcast is common, 1080 is generally only ever the interlaced version and not progressive.

Low bitrate at a high resolution will be a blurry blocky image. The resampling from UHD 4:2:0 to HD can work well for chroma because when you have 4 pixels and you're downscaling by an integer multiple, you can keep the single pixel. You can actually end up with 4:4:4 at the lower resolution if you do it properly. I don't think anyone does this though.

[Electron.Rotoscope]

Low bitrate at a high resolution will be a blurry blocky image.

I guess this goes to the core of my question. I guess I just assumed that at sane bitrates (not, like, 1 kbps for HD) that a modern compression algorithm would be able to at least do as good a job as just rescaling something to be lower resolution.

Obviously this is more academic than practical, clearly the whole world almost universally uses 4:2:0... but this a more practical question as well: if I have a UHD source and I'm viewing it on a UHD screen, at what bitrate should I look at downscaling to HD because getting an HD image and applying 2x zoom during playback will look better? It's just not a question I've ever considered before.

[FranceBB]

Encoding in 4:4:4 means that you would still need to spend a fair bit of bitrate to encode the chroma and, even if you were to apply a quantization pass that treated luma and chroma differently, it would still not be worth it.
If you go to page 18 of one of my papers (https://view.publitas.com/p222-8308/francesco-bucciantini-utilizzo-delle-trasformate-nella-codifica-delle-immagini-fisse-in-movimento-ed-audio/page/18-19) (it's in Italian but I guess if someone wants to read it, it should be possible to translate it with Google Translate) you're gonna see that it mentions 3 types of visions: photopic, mesopic and scotopic (thanks to cones and rods). In the photopic vision (lots of light), only cones (which are able to capture both luma and chroma) are used. In the mesopic vision (medium light), using only cones wouldn't be enough, so rods are employed, however, those are only able to see luma and not chroma. In the scotopic vision (low light), only rods are active and the vision is mainly in black and white with little to no perception of colors. In other words, we're much more sensible to luma changes than we are to chroma changes, so much so that downscaling and also compressing chroma still isn't the end of the world. For this reason, spending so much bitrate on the chroma wouldn't make sense, so downscaling it to half the resolution and then having the end device resize it actually made sense back in the days. Nowadays resizers have become so good that chroma artifacts have become not so easy to spot and in a day and age where bandwidth is highly constrained and expensive (think about hotbird for satellite), it still makes sense to air in 4:2:0.

[Frank62]

Very interesting, never read this that detailled, but back to the first question: The point is, that chroma resizing to a quarter resolution and back to original can't hardly be seen if done with a modern (normal) good resizer. A lower bitrate could be seen because of artefacts, especially in high motion scenes, so color would be spatially separated from luma.

Ok... But wouldn't it be possible to do both? Lower resolution after downsizing the chroma would only need a lower bitrate than the original, to keep spatially consistent with luma, wouldn't it?

[Electron.Rotoscope]

In other words, we're much more sensible to luma changes than we are to chroma changes

I don't think anyone's disputing that more bandwidth should be spent on luma than on chroma, I'm just asking why this is done with a scaler/resampler rather than in the encoder.

It looks like this has come up before and my earlier searches just didn't find it: https://forum.doom9.org/showthread.php?t=173857

And if I'm interpreting it right, it looks like my instinct is correct that x264 does a better job of reducing the bandwidth assigned to the chroma than a resampler can

lowering chroma resolution is a very inefficient way of saving bits on chroma

[huhn]

because it isn't

if you are just taking compression into account you are gaining nothing at all with modern(old now codecs).

downscaling is one of the worst compression algorithm available and that's what chroma sub sampling is nothing else.

no you don't need to spend more on chroma the --chroma-qp-offset is made for that.

with the same bandwidth you usually win a 3mbit video is 3 mbit if 444 or 420 does not matter so a satellite send 3 mbit.

the bad is the codecs need to learn how to deal with that so the luma get's at least the same bandwidth. the decode image is double the size of 420. barely any hardware decoder can deal with 444 and this is why it is still not the norm and may never be "we always did 420" 420 needs less bandwidth after decoding meaning cheaper chips.

[Asmodian]

Is this just a holdout from the past when decode was still a bottleneck and it's easier to decode 4:2:0 because fewer pixels?

It is a holdout from way before that!

It is due to bandwidth constraints, raw video bandwidth, way back in the day. Any 'modern' compression method would be better than 4:2:0, but we still have 4:2:0 on UHD bluray. :(

But it still saves bandwidth between decode and display and the display device does not need more compute to convert to RGB from YCbCr 4:2:0 compared to 4:4:4.

[Electron.Rotoscope]

420 needs less bandwidth after decoding meaning cheaper chips.

I guess that answers my question! Standards acting like standards I guess!

In good news, least one of the companies that makes livestreaming hardware boxes is starting to support 4:4:4 H.264 encodes and in-app decodes as of this month, so it looks like that switch is finally slowly coming into the real world. Maybe subsampling will finally get phased out

[Emulgator]

Given the choice of having full-res, but blocky chroma, XOR half-res but clean chroma...
I would suggest a test.
(Do not copy-paste, this is no script, just a hint)
Take a pristine 4k Source
y=Grayscale.Resize(width/2,height/2).
Encode with your desired full bitrate. (this will be Y as greyscale in FHD)
u=UtoY() and encode with your desired starving bitrate (this will be U as greyscale in FHD)
v=VtoY() and encode with your desired starving bitrate (this will be V as greyscale in FHD)
Decode them all using your favourite source filter and YToUV(y,u,v)

Spending the same bitrate I guess there will be more chroma blocks with 4:4:4 as if encoded as 4:2:0.
(Any encoder will analyse chroma, if fed 4:2:0 finding only lower chroma frequencies and will assign less bits OOTB, job done)

[huhn]

encoder don't block anymore at any reasonable bit rate so this is a no brainer.

and yes there is a point where downscaling is better at a certain bit rate. and sub sampling may no will win if you go beyond that.

the real issue currently is getting the same bit rate to the luma channel CRF can do that but the encode will be bigger if not a proper offset is found.

the cases where chroma matter are usually already gonna just from sub sampling alone the encode doesn't matter anymore at that point.

there is no need to bit rate starve anything these days even a youtube video at 4:4:4 will for the majority of test hands down win. using something where chroma matters in the first place.

you can just do the test with an RGB image with proper chroma placing and such there is a lot out there to do test encodes.