Enhanced Compression Model (ECM) / H.267: VVC successor

[birdie]

Source 1.

Source 2.

Fraunhofer's reference implementation: https://vcgit.hhi.fraunhofer.de/ecm/ECM

[modus-ms325c]

VVC hasn't even picked up momentum and they're already making a "sequel"?

[birdie]

Originally Posted by modus-ms325c

VVC hasn't even picked up momentum and they're already making a "sequel"?

Hasn't been good enough

[FranceBB]

Originally Posted by modus-ms325c

VVC hasn't even picked up momentum

It will, give it time.
It will finally give TV manufacturers an excuse to bring 8K to customers.

[hajj_3]

Originally Posted by FranceBB

It will, give it time.
It will finally give TV manufacturers an excuse to bring 8K to customers.

Nope. EU regulations have an energy efficiency limit for tvs. It isn't possible for 8k tvs to be energy efficient which is why 8k tvs have disappeared. Manufacturers are less likely to make them if some of the wealthiest areas of the world can't buy them.

Originally Posted by modus-ms325c

VVC hasn't even picked up momentum and they're already making a "sequel"?

the mpeg roadmap shows it being ratified at the end of 2025/beginning of 2026, by that time vvc will be 5.5yrs old.

[ksec]

Why create another thread when we already have one?

[benwaggoner]

Originally Posted by modus-ms325c

VVC hasn't even picked up momentum and they're already making a "sequel"?

It's standard for the new codec R&D to start relatively soon after the prior standard has been completed. There's always a bunch of leftover tools that didn't make the cut to take another run at. Generally each new generation allows ~10x more encode compute and ~2x more decode compute, so techniques that didn't make sense for one generation can make sense for the following one.

[benwaggoner]

Originally Posted by hajj_3

Nope. EU regulations have an energy efficiency limit for tvs. It isn't possible for 8k tvs to be energy efficient which is why 8k tvs have disappeared. Manufacturers are less likely to make them if some of the wealthiest areas of the world can't buy them.

Moreover, the industry hasn't been able to demonstrate that 8K moving image content offers a better visual experience than a 4K downscale of the same content. An 8K panel gives 12-16 RGB sub pixels per YUV content pixel, which offers some theoretical benefit, but not a practical one at reasonable viewing distances for natural image content. 8K does look better for computer monitors running productivity apps and such.

The minimum pixel size of 2nd gen quantum dot OLED and microLED are also still too big to make an 8K TV out of it at an acceptable living room size. So the displays the offer the best image quality for movie content on a calibrated display in a darker viewing environment aren't available in 8K either. Of course, this is a temporary situation. I expect EU-legal 8K TVs to become feasible in the next few years.

That said, my primary TV/gaming monitor is an 85" Samsung Q900C, and it is a great display and experience. Nice to be in the USA where our power restrictions aren't as limited (and I use it in a darker room, so ambient light adaptation keeps power draw lower).

[kurkosdr]

Originally Posted by hajj_3

Nope. EU regulations have an energy efficiency limit for tvs. It isn't possible for 8k tvs to be energy efficient which is why 8k tvs have disappeared. Manufacturers are less likely to make them if some of the wealthiest areas of the world can't buy them.

It's worth mentioning that the EU doesn't impose a hard limit on TV power consumption in general, it only imposes a hard limit on the default mode (typically called "natural" or similar by the manufacturer), and nothing prevents you from switching your TV to "dynamic" or "standard" mode which has no power limits (that is, after you select "yes" on the silly warning about power consumption being higher in those modes, which of course it is). Most TVs urge you to pick a mode during installation nowadays, so you realistically can't be accidentally stuck in the default mode on a modern TV. Which makes you wonder why the EU even bothered to regulate all this. So, nothing prevents manufacturers from having an ultra-dark mode on 8K TVs that conforms to the EU's limits.

The reason 8K TVs don't sell is a much simpler one: Nobody has a good reason to own one, because there is no tangible benefit over 4K. This in turn means there is no demand for 8K content, much less enough demand to justify the rather steep demands of 8K when it comes to bitrate (even if VVC is used, or even ECM).

Personally, I want to see what ECM will achieve when it comes to 4K content, which is still rare in broadcasting (especially FTA). Could we see a 4K HLG10 channel finally fit in the space of a typical FHD H.264 channel?

(one way we might see 8K panels become useful is to provide "glasses-free" stereoscopic 4K content, since the black dots of the "glasses-free" technology will be invisible with such a panel, but that's a slim chance I'll admit, and that still won't make 8K content useful, just the panels)

[benwaggoner]

Originally Posted by kurkosdr

Personally, I want to see what ECM will achieve when it comes to 4K content, which is still rare in broadcasting (especially FTA). Could we see a 4K HLG10 channel finally fit in the space of a typical FHD H.264 channel?

We can generally fit a 4K HDR-10 HEVC 10-bit into the same space as a 1080p H.264 SDR channel with statmux. I don't think we need to wait for ECM!

(one way we might see 8K panels become useful is to provide "glasses-free" stereoscopic 4K content, since the black dots of the "glasses-free" technology will be invisible with such a panel, but that's a slim chance I'll admit, and that still won't make 8K content useful, just the panels)

Glasses-free for more than one person requires lenticular screens, which lose horizontal detail proportional to the number of viewing angles available. 16 is about the lowest that can kinda work with careful sitting, and 32 when it becomes natural feeling. So a 64K panel would give us full 1080p detail with 32 viewing angles.

[kurkosdr]

Originally Posted by benwaggoner

We can generally fit a 4K HDR-10 HEVC 10-bit into the same space as a 1080p H.264 SDR channel with statmux. I don't think we need to wait for ECM!

No, you can't, the efficiency gains aren't there, especially when the HDR tax (20%) is taken into account. For each 1Mbits of 1080p H.264 SDR, for 4K HDR HEVC you need:
1*4*1.2*0.6=2.88, so 288% of a 1080p H.264 SDR broadcast
(assuming a reduction to 60% of bitrate for HEVC compared to H.264 for UHD content)

Now, in a parallel universe where broadcasters give actually good 1080p (15Mbps), you could just barely do it (replace FHD with UHD on a 1:1 basis), but this is not the universe we live in, most broadcasters don't even give 6Mbps of statistical bitrate to FHD (and even premium ones don't give above 7Mbps of statistical bitrate to FHD).

The only way UHD works in broadcast is by squeezing FHD channels even further to make space for a couple of UHD channels per 30 FHD channels or so.

[FranceBB]

Well, another additional consideration is that channels are actually 1080i not 1080p, which does actually allow lower bitrates compared to encoding the whole thing progressively. Only with UHD there's been the move from 25i TFF to 50p and that of course requires a lot more bandwidth.

So yeah, you can't really put a UHD channel in using the same bandwidth of a FULL HD one. By the way, above we were talking about HDR10, but I don't really think that there's gonna be a move to HDR10 anytime soon for linear. When UHD arrived in 2013-2015, early adopters settled on BT2020 SDR, which is why when HDR arrived in 2017 everyone shifted towards HLG to not screw people with early TVs over and we've been stuck in a loop ever since.

Currently, a broadcaster has to keep the very same channel with the very same content playing in:

- 720x576 SD (anamorphic flagged 16x9) 4:2:0 25i TFF BT601 SDR 8bit MPEG-2 2.5 Mbits
- 1920x1080 FULL HD 16x9 4:2:0 25i TFF BT709 SDR 8bit H.264 12 Mbits
- 3840x2160 UHD 16x9 4:2:0 50p BT2020 HLG HDR 10bit H.265 18 Mbits

Keep in mind that the ideal UHD 50p bitrate is actually 25 Mbits and it used to be that, but bandwidth on hotbird is so constrained that we were forced to lower it down to 18 Mbits. Also keep in mind that the efficiency of live hardware encoding isn't comparable to the one of offline software encoding. On top of that, in Italy we had to shut down the Sky Cinema UHD channel as it was far too expensive to run. Right now, the only UHD channel still going is the Sky Sports one. This didn't really get a very good reception and we were scolded at the UHD meeting in Milan earlier on this year 'cause in their view if we don't innovate and bring UHD over to customers, who will? Well, obviously streaming companies will, but that's not linear. Streaming companies also don't have the issue of keeping compatibility with older UHD SDR TVs as they can easily put out a PQ stream and it would be just an additional file on the CDN that a player can choose. For a broadcaster, either you just put it there instead of the HLG one and eventually end up screwing someone over or you put out another UHD channel but keep the HLG one, which is never gonna happen. This is why I said that I was excited about H.266 VVC and 8K. I mean, 8K won't really be something important in terms of resolution, so much so that lots of people won't even notice the resolution shift from 4K, but it will finally give broadcasters the excuse to move to PQ once and for all (and potentially go to 12bit instead of 10bit although early drafts still showed 10bit as the standard).

[kurkosdr]

Originally Posted by FranceBB

By the way, above we were talking about HDR10, but I don't really think that there's gonna be a move to HDR10 anytime soon for linear. When UHD arrived in 2013-2015, early adopters settled on BT2020 SDR, which is why when HDR arrived in 2017 everyone shifted towards HLG to not screw people with early TVs over and we've been stuck in a loop ever since.

Sure, but the 20% HDR tax applies to HLG10 too.

Originally Posted by FranceBB

Well, another additional consideration is that channels are actually 1080i not 1080p, which does actually allow lower bitrates compared to encoding the whole thing progressively. Only with UHD there's been the move from 25i TFF to 50p and that of course requires a lot more bandwidth.

Wait a minute... is there another bitrate tax from moving from interlaced (and 25p encoded as MBAFF in interlaced) to 50p? I assumed that, since there is no interlaced in HEVC (it's technically in the standard but not implemented or used anywhere), the "60% of the bitrate for UHD (compared to H.264)" figure included the 50p tax but not the HDR tax. But I may be wrong. If so, how much is the 50p tax?

Originally Posted by FranceBB

Currently, a broadcaster has to keep the very same channel with the very same content playing in:

- 720x576 SD (anamorphic flagged 16x9) 4:2:0 25i TFF BT601 SDR 8bit MPEG-2 2.5 Mbits
- 1920x1080 FULL HD 16x9 4:2:0 25i TFF BT709 SDR 8bit H.264 12 Mbits
- 3840x2160 UHD 16x9 4:2:0 50p BT2020 HLG HDR 10bit H.265 18 Mbits

If premium broadcasters were smart, they'd give subscribers new HEVC-capable decoder boxes when they renewed their contract all the way back in 2019, so in 2-3 years' time everyone would have HEVC-capable decoder boxes. This would allow them to shut down everything else and keep only HEVC, and then use the bitrate savings of HEVC from FHD SDR content to give subscribers more UHD channels.

I mean, they've already screwed us over by marrying their cards to their decoder boxes (and making Common Interface worthless in the process), the ability to periodically "refresh" hardware is the silver lining of that. But nope: penny-pinching, accountants, focus on the next quarter etc etc, they'd rather bleed subscribers to OTT and Streaming every year than look beyond the next quarter. The only premium broadcasters who have a reason to broadcast H.264 today are those moving UHD customers to OTT. But the ones trying to make UHD happen on broadcast, if they were forward-looking, they'd already be giving subscribers free UHD VVC boxes when renewing, but I know, I'm dreaming, but still, free UHD HEVC decoder boxes is realistic.

About FTA, 2 UHD channels per 30 FHD channels will remain the name of the game until ECM comes in the next "transition" (at least a decade from now), because again, good luck telling people they need to get FTA receiver boxes for their still-new UHD TVs.

[FranceBB]

Originally Posted by kurkosdr

Wait a minute... is there another bitrate tax from moving from interlaced (and 25p encoded as MBAFF in interlaced) to 50p? If so, how much is the 50p tax?

There sure is. 25i can be safely encoded in around 7-12 Mbits (and the bandwidth is shifted around according to how "important" an event is by a department called "technology"). This is because a field is literally just 1920x540 and you have two fields (even and odd) one with the very same image and one with the image immediately after. For things like sports and news, those are truly two different fields and they're bobbed to 50p by the decoder or by the TV by filling the "void" (i.e the gap between one and the other) with some kind of interpolation. It used to be a simple cubic interpolation in the early hardware LCD TV days but decoders got smarter and the image doesn't "jump up and down" any longer when bobbed. For movies and TV series (generally 23,976 with 4% speed up + pitch adjustment applied to get 25p) those two fields are exactly the same as they're encoded as progressive and flagged as interlaced. Unlike MPEG-2 where those would be encoded very inefficiently *coff coff damn SD channels coff coff* H.264 with MBAFF recognises that they're progressive and encodes them very efficiently. TVs nowadays also recognise that they're progressive and don't bob them, thus preventing useless processing and the unnecessary introduction of aliasing, however bad things could happen if you have like a ticker interlaced displaying a message. Anyway that's beyond the point. The point is that to have a truly 50p stream you would need 1920x1080 with 50 frames per second which is very different from 1920x540 fields, so you need much more bandwidth. I'd say 15 Mbits in a live hardware encoder, hence why I said that for a UHD HDR HLG 50p 10bit channel in H.265 you would ideally need 25 Mbits and you can't possibly get any lower than 18 Mbits without really pushing it (i.e before even customers start complaining).


p.s about replacing decoders (boxes), if it was for me, I would have gifted a Sky Q box to everyone, got rid of the dark gray SD decoder, the white FULL HD one and the black My Sky one to move people over to H.265. Then I would have closed all the SD channels, implemented UHD downscaling on the decoder side and only really kept the necessary bunch of FULL HD with the UHD ones showing different content, instead of having the same content on three different versions of Sky Cinema playing like we have now for three different channels for four different decoders. Unfortunately, my saying is worth nothing, I'm just an encoder who "gets things done" but I have no say on how those things go. I think someone did this exercise once, saw that the cost was too high and just forfeit.

[rwill]

Originally Posted by FranceBB

For movies and TV series (generally 23,976 with 4% speed up + pitch adjustment applied to get 25p) those two fields are exactly the same as they're encoded as progressive and flagged as interlaced. Unlike MPEG-2 where those would be encoded very inefficiently *coff coff damn SD channels coff coff* H.264 with MBAFF recognises that they're progressive and encodes them very efficiently.

Mpeg-2 supports Frame/Field coding on a Macroblock level as well if pictures are coded in Frame Structure. Not Mpeg-2's problem if some cheap hardware encoder does not support these modes when set to interlaced.

[kurkosdr]

Originally Posted by FranceBB

There sure is. 25i can be safely encoded in around 7-12 Mbits (and the bandwidth is shifted around according to how "important" an event is by a department called "technology").

Yes, I know how statistical multiplexing works and what interlaced is, and the terms "HDR tax" and "50p tax" were obviously humorous (the government isn't coming to take some of your bitrate).

Let me rephrase the question:
- What bitrate range do you consider acceptable for HEVC SDR FHD 25i?
- What bitrate range do you consider acceptable for HEVC SDR FHD 50p?
(with the source material having been shot in native 50p)

[FranceBB]

Originally Posted by rwill

Mpeg-2 supports Frame/Field coding on a Macroblock level as well if pictures are coded in Frame Structure. Not Mpeg-2's problem if some cheap hardware encoder does not support these modes when set to interlaced.

Wait, it does? That's a very nice surprise!
On the other hand, given that pretty much no one is developing new SDI MPEG-2 hardware encoders and that no business would justify the expense of replacing the already functioning ones, I guess we're left with what we have. Anyway, this is a very nice surprise.

Originally Posted by kurkosdr

the terms "HDR tax" and "50p tax" were obviously humorous (the government isn't coming to take some of your bitrate).

Dunno, with Kier Starmer around anything is possible XD

Originally Posted by kurkosdr

Let me rephrase the question:
- What bitrate range do you consider acceptable for HEVC SDR FHD 25i?
- What bitrate range do you consider acceptable for HEVC SDR FHD 50p?
(with the source material having been shot in native 50p)

For H.264:
FULL HD 25i 12 Mbits
FULL HD 50p 15 Mbits

For H.265:
FULL HD 25i 8 Mbits
FULL HD 50p 9 Mbits

H.265 can generally encode contents 35% more efficiently than H.264 for progressive outputs. Unfortunately, the same isn't true for interlaced contents as most encoders never really focused on interlaced stuff and as result its efficiency is slightly impaired. Keep in mind that H.266 doesn't even support interlaced encoding at all, so...

p.s all those refer to live hardware encoding, obviously software offline encoding would achieve far better results at lower bitrates.

[Z2697]

I'd say if you compare the equivalent resolutions (i.e. half the height for 50p) the 50p will be very close if not better.
Also worth noting there's no MBAFF support in HEVC. The interlaced support is just separated fields encoded like a normal sequence each with SEI denoting the field order.