Enhanced Compression Model (ECM) / H.267: VVC successor

[birdie]

Source 1 (https://www.mpeg.org/meetings/mpeg-147/).

Source 2 (https://www.rethinkresearch.biz/articles/h-267-vvcs-heir-officially-proposed/).

Fraunhofer's reference implementation: https://vcgit.hhi.fraunhofer.de/ecm/ECM

[modus-ms325c]

VVC hasn't even picked up momentum and they're already making a "sequel"?

[birdie]

VVC hasn't even picked up momentum and they're already making a "sequel"?

Hasn't been good enough :o

[FranceBB]

VVC hasn't even picked up momentum

It will, give it time.
It will finally give TV manufacturers an excuse to bring 8K to customers.

[hajj_3]

It will, give it time.
It will finally give TV manufacturers an excuse to bring 8K to customers.

Nope. EU regulations have an energy efficiency limit for tvs. It isn't possible for 8k tvs to be energy efficient which is why 8k tvs have disappeared. Manufacturers are less likely to make them if some of the wealthiest areas of the world can't buy them.

VVC hasn't even picked up momentum and they're already making a "sequel"?

the mpeg roadmap shows it being ratified at the end of 2025/beginning of 2026, by that time vvc will be 5.5yrs old.

[ksec]

Why create another thread when we already have one?

[benwaggoner]

VVC hasn't even picked up momentum and they're already making a "sequel"?
It's standard for the new codec R&D to start relatively soon after the prior standard has been completed. There's always a bunch of leftover tools that didn't make the cut to take another run at. Generally each new generation allows ~10x more encode compute and ~2x more decode compute, so techniques that didn't make sense for one generation can make sense for the following one.

[benwaggoner]

Nope. EU regulations have an energy efficiency limit for tvs. It isn't possible for 8k tvs to be energy efficient which is why 8k tvs have disappeared. Manufacturers are less likely to make them if some of the wealthiest areas of the world can't buy them.

Moreover, the industry hasn't been able to demonstrate that 8K moving image content offers a better visual experience than a 4K downscale of the same content. An 8K panel gives 12-16 RGB sub pixels per YUV content pixel, which offers some theoretical benefit, but not a practical one at reasonable viewing distances for natural image content. 8K does look better for computer monitors running productivity apps and such.

The minimum pixel size of 2nd gen quantum dot OLED and microLED are also still too big to make an 8K TV out of it at an acceptable living room size. So the displays the offer the best image quality for movie content on a calibrated display in a darker viewing environment aren't available in 8K either. Of course, this is a temporary situation. I expect EU-legal 8K TVs to become feasible in the next few years.

That said, my primary TV/gaming monitor is an 85" Samsung Q900C, and it is a great display and experience. Nice to be in the USA where our power restrictions aren't as limited (and I use it in a darker room, so ambient light adaptation keeps power draw lower).

[kurkosdr]

Nope. EU regulations have an energy efficiency limit for tvs. It isn't possible for 8k tvs to be energy efficient which is why 8k tvs have disappeared. Manufacturers are less likely to make them if some of the wealthiest areas of the world can't buy them.

It's worth mentioning that the EU doesn't impose a hard limit on TV power consumption in general, it only imposes a hard limit on the default mode (typically called "natural" or similar by the manufacturer), and nothing prevents you from switching your TV to "dynamic" or "standard" mode which has no power limits (that is, after you select "yes" on the silly warning about power consumption being higher in those modes, which of course it is). Most TVs urge you to pick a mode during installation nowadays, so you realistically can't be accidentally stuck in the default mode on a modern TV. Which makes you wonder why the EU even bothered to regulate all this. So, nothing prevents manufacturers from having an ultra-dark mode on 8K TVs that conforms to the EU's limits.

The reason 8K TVs don't sell is a much simpler one: Nobody has a good reason to own one, because there is no tangible benefit over 4K. This in turn means there is no demand for 8K content, much less enough demand to justify the rather steep demands of 8K when it comes to bitrate (even if VVC is used, or even ECM).

Personally, I want to see what ECM will achieve when it comes to 4K content, which is still rare in broadcasting (especially FTA). Could we see a 4K HLG10 channel finally fit in the space of a typical FHD H.264 channel?

(one way we might see 8K panels become useful is to provide "glasses-free" stereoscopic 4K content, since the black dots of the "glasses-free" technology will be invisible with such a panel, but that's a slim chance I'll admit, and that still won't make 8K content useful, just the panels)

[benwaggoner]

Personally, I want to see what ECM will achieve when it comes to 4K content, which is still rare in broadcasting (especially FTA). Could we see a 4K HLG10 channel finally fit in the space of a typical FHD H.264 channel?
We can generally fit a 4K HDR-10 HEVC 10-bit into the same space as a 1080p H.264 SDR channel with statmux. I don't think we need to wait for ECM!

(one way we might see 8K panels become useful is to provide "glasses-free" stereoscopic 4K content, since the black dots of the "glasses-free" technology will be invisible with such a panel, but that's a slim chance I'll admit, and that still won't make 8K content useful, just the panels)
Glasses-free for more than one person requires lenticular screens, which lose horizontal detail proportional to the number of viewing angles available. 16 is about the lowest that can kinda work with careful sitting, and 32 when it becomes natural feeling. So a 64K panel would give us full 1080p detail with 32 viewing angles.

[kurkosdr]

We can generally fit a 4K HDR-10 HEVC 10-bit into the same space as a 1080p H.264 SDR channel with statmux. I don't think we need to wait for ECM!
No, you can't, the efficiency gains aren't there, especially when the HDR tax (20%) is taken into account. For each 1Mbits of 1080p H.264 SDR, for 4K HDR HEVC you need:
1*4*1.2*0.6=2.88, so 288% of a 1080p H.264 SDR broadcast
(assuming a reduction to 60% of bitrate for HEVC compared to H.264 for UHD content)

Now, in a parallel universe where broadcasters give actually good 1080p (15Mbps), you could just barely do it (replace FHD with UHD on a 1:1 basis), but this is not the universe we live in, most broadcasters don't even give 6Mbps of statistical bitrate to FHD (and even premium ones don't give above 7Mbps of statistical bitrate to FHD).

The only way UHD works in broadcast is by squeezing FHD channels even further to make space for a couple of UHD channels per 30 FHD channels or so.

[FranceBB]

Well, another additional consideration is that channels are actually 1080i not 1080p, which does actually allow lower bitrates compared to encoding the whole thing progressively. Only with UHD there's been the move from 25i TFF to 50p and that of course requires a lot more bandwidth.

So yeah, you can't really put a UHD channel in using the same bandwidth of a FULL HD one. By the way, above we were talking about HDR10, but I don't really think that there's gonna be a move to HDR10 anytime soon for linear. When UHD arrived in 2013-2015, early adopters settled on BT2020 SDR, which is why when HDR arrived in 2017 everyone shifted towards HLG to not screw people with early TVs over and we've been stuck in a loop ever since.

Currently, a broadcaster has to keep the very same channel with the very same content playing in:

- 720x576 SD (anamorphic flagged 16x9) 4:2:0 25i TFF BT601 SDR 8bit MPEG-2 2.5 Mbits
- 1920x1080 FULL HD 16x9 4:2:0 25i TFF BT709 SDR 8bit H.264 12 Mbits
- 3840x2160 UHD 16x9 4:2:0 50p BT2020 HLG HDR 10bit H.265 18 Mbits

Keep in mind that the ideal UHD 50p bitrate is actually 25 Mbits and it used to be that, but bandwidth on hotbird is so constrained that we were forced to lower it down to 18 Mbits. Also keep in mind that the efficiency of live hardware encoding isn't comparable to the one of offline software encoding. On top of that, in Italy we had to shut down the Sky Cinema UHD channel as it was far too expensive to run. Right now, the only UHD channel still going is the Sky Sports one. This didn't really get a very good reception and we were scolded at the UHD meeting in Milan earlier on this year 'cause in their view if we don't innovate and bring UHD over to customers, who will? Well, obviously streaming companies will, but that's not linear. Streaming companies also don't have the issue of keeping compatibility with older UHD SDR TVs as they can easily put out a PQ stream and it would be just an additional file on the CDN that a player can choose. For a broadcaster, either you just put it there instead of the HLG one and eventually end up screwing someone over or you put out another UHD channel but keep the HLG one, which is never gonna happen. This is why I said that I was excited about H.266 VVC and 8K. I mean, 8K won't really be something important in terms of resolution, so much so that lots of people won't even notice the resolution shift from 4K, but it will finally give broadcasters the excuse to move to PQ once and for all (and potentially go to 12bit instead of 10bit although early drafts still showed 10bit as the standard).

[kurkosdr]

By the way, above we were talking about HDR10, but I don't really think that there's gonna be a move to HDR10 anytime soon for linear. When UHD arrived in 2013-2015, early adopters settled on BT2020 SDR, which is why when HDR arrived in 2017 everyone shifted towards HLG to not screw people with early TVs over and we've been stuck in a loop ever since.
Sure, but the 20% HDR tax applies to HLG10 too.

Well, another additional consideration is that channels are actually 1080i not 1080p, which does actually allow lower bitrates compared to encoding the whole thing progressively. Only with UHD there's been the move from 25i TFF to 50p and that of course requires a lot more bandwidth.
Wait a minute... is there another bitrate tax from moving from interlaced (and 25p encoded as MBAFF in interlaced) to 50p? I assumed that, since there is no interlaced in HEVC (it's technically in the standard but not implemented or used anywhere), the "60% of the bitrate for UHD (compared to H.264)" figure included the 50p tax but not the HDR tax. But I may be wrong. If so, how much is the 50p tax?

Currently, a broadcaster has to keep the very same channel with the very same content playing in:

- 720x576 SD (anamorphic flagged 16x9) 4:2:0 25i TFF BT601 SDR 8bit MPEG-2 2.5 Mbits
- 1920x1080 FULL HD 16x9 4:2:0 25i TFF BT709 SDR 8bit H.264 12 Mbits
- 3840x2160 UHD 16x9 4:2:0 50p BT2020 HLG HDR 10bit H.265 18 Mbits
If premium broadcasters were smart, they'd give subscribers new HEVC-capable decoder boxes when they renewed their contract all the way back in 2019, so in 2-3 years' time everyone would have HEVC-capable decoder boxes. This would allow them to shut down everything else and keep only HEVC, and then use the bitrate savings of HEVC from FHD SDR content to give subscribers more UHD channels.

I mean, they've already screwed us over by marrying their cards to their decoder boxes (and making Common Interface worthless in the process), the ability to periodically "refresh" hardware is the silver lining of that. But nope: penny-pinching, accountants, focus on the next quarter etc etc, they'd rather bleed subscribers to OTT and Streaming every year than look beyond the next quarter. The only premium broadcasters who have a reason to broadcast H.264 today are those moving UHD customers to OTT. But the ones trying to make UHD happen on broadcast, if they were forward-looking, they'd already be giving subscribers free UHD VVC boxes when renewing, but I know, I'm dreaming, but still, free UHD HEVC decoder boxes is realistic.

About FTA, 2 UHD channels per 30 FHD channels will remain the name of the game until ECM comes in the next "transition" (at least a decade from now), because again, good luck telling people they need to get FTA receiver boxes for their still-new UHD TVs.

[FranceBB]

Wait a minute... is there another bitrate tax from moving from interlaced (and 25p encoded as MBAFF in interlaced) to 50p? If so, how much is the 50p tax?

There sure is. 25i can be safely encoded in around 7-12 Mbits (and the bandwidth is shifted around according to how "important" an event is by a department called "technology"). This is because a field is literally just 1920x540 and you have two fields (even and odd) one with the very same image and one with the image immediately after. For things like sports and news, those are truly two different fields and they're bobbed to 50p by the decoder or by the TV by filling the "void" (i.e the gap between one and the other) with some kind of interpolation. It used to be a simple cubic interpolation in the early hardware LCD TV days but decoders got smarter and the image doesn't "jump up and down" any longer when bobbed. For movies and TV series (generally 23,976 with 4% speed up + pitch adjustment applied to get 25p) those two fields are exactly the same as they're encoded as progressive and flagged as interlaced. Unlike MPEG-2 where those would be encoded very inefficiently *coff coff damn SD channels coff coff* H.264 with MBAFF recognises that they're progressive and encodes them very efficiently. TVs nowadays also recognise that they're progressive and don't bob them, thus preventing useless processing and the unnecessary introduction of aliasing, however bad things could happen if you have like a ticker interlaced displaying a message. Anyway that's beyond the point. The point is that to have a truly 50p stream you would need 1920x1080 with 50 frames per second which is very different from 1920x540 fields, so you need much more bandwidth. I'd say 15 Mbits in a live hardware encoder, hence why I said that for a UHD HDR HLG 50p 10bit channel in H.265 you would ideally need 25 Mbits and you can't possibly get any lower than 18 Mbits without really pushing it (i.e before even customers start complaining).


p.s about replacing decoders (boxes), if it was for me, I would have gifted a Sky Q box to everyone, got rid of the dark gray SD decoder, the white FULL HD one and the black My Sky one to move people over to H.265. Then I would have closed all the SD channels, implemented UHD downscaling on the decoder side and only really kept the necessary bunch of FULL HD with the UHD ones showing different content, instead of having the same content on three different versions of Sky Cinema playing like we have now for three different channels for four different decoders. Unfortunately, my saying is worth nothing, I'm just an encoder who "gets things done" but I have no say on how those things go. I think someone did this exercise once, saw that the cost was too high and just forfeit.

[rwill]

For movies and TV series (generally 23,976 with 4% speed up + pitch adjustment applied to get 25p) those two fields are exactly the same as they're encoded as progressive and flagged as interlaced. Unlike MPEG-2 where those would be encoded very inefficiently *coff coff damn SD channels coff coff* H.264 with MBAFF recognises that they're progressive and encodes them very efficiently.

Mpeg-2 supports Frame/Field coding on a Macroblock level as well if pictures are coded in Frame Structure. Not Mpeg-2's problem if some cheap hardware encoder does not support these modes when set to interlaced.

[kurkosdr]

There sure is. 25i can be safely encoded in around 7-12 Mbits (and the bandwidth is shifted around according to how "important" an event is by a department called "technology").
Yes, I know how statistical multiplexing works and what interlaced is, and the terms "HDR tax" and "50p tax" were obviously humorous (the government isn't coming to take some of your bitrate).

Let me rephrase the question:
- What bitrate range do you consider acceptable for HEVC SDR FHD 25i?
- What bitrate range do you consider acceptable for HEVC SDR FHD 50p?
(with the source material having been shot in native 50p)

[FranceBB]

Mpeg-2 supports Frame/Field coding on a Macroblock level as well if pictures are coded in Frame Structure. Not Mpeg-2's problem if some cheap hardware encoder does not support these modes when set to interlaced.

Wait, it does? That's a very nice surprise!
On the other hand, given that pretty much no one is developing new SDI MPEG-2 hardware encoders and that no business would justify the expense of replacing the already functioning ones, I guess we're left with what we have. Anyway, this is a very nice surprise.

the terms "HDR tax" and "50p tax" were obviously humorous (the government isn't coming to take some of your bitrate).


Dunno, with Kier Starmer around anything is possible XD


Let me rephrase the question:
- What bitrate range do you consider acceptable for HEVC SDR FHD 25i?
- What bitrate range do you consider acceptable for HEVC SDR FHD 50p?
(with the source material having been shot in native 50p)


For H.264:
FULL HD 25i 12 Mbits
FULL HD 50p 15 Mbits

For H.265:
FULL HD 25i 8 Mbits
FULL HD 50p 9 Mbits

H.265 can generally encode contents 35% more efficiently than H.264 for progressive outputs. Unfortunately, the same isn't true for interlaced contents as most encoders never really focused on interlaced stuff and as result its efficiency is slightly impaired. Keep in mind that H.266 doesn't even support interlaced encoding at all, so...

p.s all those refer to live hardware encoding, obviously software offline encoding would achieve far better results at lower bitrates.

[Z2697]

I'd say if you compare the equivalent resolutions (i.e. half the height for 50p) the 50p will be very close if not better.
Also worth noting there's no MBAFF support in HEVC. The interlaced support is just separated fields encoded like a normal sequence each with SEI denoting the field order.

[Jamaika]

ffmpeg.exe -y -i "input.mp4" -f yuv4mpegpipe -vf scale=1920:1080:in_range=full:_out_range=full,format=yuv420p10le -frames:v 100 -strict -1 111.y4m
EncoderApp_ECM.exe --SummaryVerboseness -c "encoder_randomaccess_ecm.cfg" --InputFile=111.y4m --BitstreamFile=117.vvc --SourceWidth=1920 --SourceHeight=1080 --FrameRate=30000/1001 --InputBitDepth=10 --InternalBitDepth=0 --OutputBitDepth=10 --MSBExtendedBitDepth=10 --InputChromaFormat=420 --ChromaFormatIDC=420 --ConformanceWindowMode=0 --FramesToBeEncoded=100 --MatrixCoefficients=1 --InputColorPrimaries=-1 --LMCSSignalType=0 --Level=5.1 --BDPCM=1 --Tier=main --HashME=1 --IBC=1 --MaxCUWidth=16 --MaxCUHeight=16 --CTUSize=32 --MaxBTLumaISlice=32 --MaxBTChromaISlice=32 --MaxBTNonISlice=32 --MaxTTLumaISlice=32 --MaxTTChromaISlice=32 --MaxTTNonISlice=32 --Log2MaxTbSize=5 --ColorTransform=0 --VideoFullRange=1 --InputSampleRange=1 --AspectRatioInfoPresent=1 --ChromaLocInfoPresent=1 --RCCpbSize=2000 --RateControl=1 --TargetBitrate=6000000 --IntraPeriod=256 --RCCpbSaturation=0
VVCSoftware: ECM Encoder Version 14.1-c917037 (VTM-10.0) [Windows][GCC 11.5.0][64 bit] [SIMD=AVX2]
https://www.sendspace.com/file/jtfshj

[ksec]

ECM 15 is out. Beyond the initial 25% reduction it does seems anything more is moving fairly slowly.

[birdie]

Exploring the future of video codecs at ITU's 2025 MPEG Workshop, where industry leaders discuss the next decade of video compression technology. (https://blog.mainconcept.com/defining-the-next-generation-of-video-codecs-itus-2025-mpeg-workshop)

Some murky news on H.266/H.267 and the future of MPEG video codecs.

The video of the conference is here (https://itu.zoom.us/rec/play/6YwibaIBANc-Mb6nailcR4Dfa1buIMDm2SEoSFT85Z3Z49WXQOUq29PBLaHnA6kfModQS_VGqc8jj6gJ.ZubOce9UhdgWijXQ?canPlayFromShare=true&from=share_recording_detail&continueMode=true&componentName=rec-play&originRequestUrl=https%3A%2F%2Fitu.zoom.us%2Frec%2Fshare%2FBRiq4IaOB0UHf35RKdoUYltftrJRofQ4IhSQgwOyB1cPPGSmhiBUq1T0qxYH-5kQ.A8IXYooxXDbmBZNN).

MPEG seems not to understand why H.266 adoption has been so bad/slow/non-existent. Looks like no one even raised the question of a very bad licensing situation in regard to H.266.

[Z2697]

It has now gotten to the point like "whatever man, whatever".
AOM will take over.

[benwaggoner]

MPEG seems not to understand why H.266 adoption has been so bad/slow/non-existent. Looks like no one even raised the question of a very bad licensing situation in regard to H.266.
It's not that MPEG members don't know. I've talked to plenty of people who were involved in that session, and they know. IP issues were called out as a substantial problem that needs to be addressed in a number of the presentations.

It's more that the MPEG and ITU processes don't allow discussion about IP issues, just technical ones. They literally aren't allowed to consider patents and licensing in designing the technical specifications, in the presumption that "industry will figure it out later, because all stakeholders are invested in a new format's success."

Which obviously isn't true at all these days, but would require some amendments to the group charters to allow.

And from a purely technical perspective, a purely technical approach has good dividends. VVC as a codec outperforms AV1 in compression efficiency with substantially less complex decoder requirements. If VVC had AOM licensing, it would be broadly used today. But it doesn't, and so VVC use is only planned in particular markets (Brazil) that believe they've handed the IP issues for themselves.

[kurkosdr]

It's not that MPEG members don't know. I've talked to plenty of people who were involved in that session, and they know. IP issues were called out as a substantial problem that needs to be addressed in a number of the presentations.

It's more that the MPEG and ITU processes don't allow discussion about IP issues, just technical ones. They literally aren't allowed to consider patents and licensing in designing the technical specifications, in the presumption that "industry will figure it out later, because all stakeholders are invested in a new format's success."
These processes were established under the assumption that patent holders want the new standard to achieve mass adoption and want repeat customers. This assumption stopped being true when the patent holders realised they had lost the future of web video to WebM (which would later become AOM) and HEVC would never become universal on the web like AVC is. So, the patent holders focused on milking their remaining customers, more specifically broadcasters and premium video providers like Netflix (who charge subscribers extra for UHD quality and hence want guaranteed compatibility with UHD Smart TVs). The fossil record is a bit unclear here, but it happened around the time Chrome refused to license and bundle an HEVC decoder in Chrome and around the time VP9 was announced (becoming the first "better than H.264" format that wasn't an ISO/ITU format). The walkout from the MPEG LA pool and the founding of the HEVC Advance pool happened shortly after. Sure, the patent holders could have offered better terms for HEVC (for example, no royalties for software decoders and no content fees on free web video) and make HEVC as universal as AVC, but that's not what they wanted to do.

VVC is the same milking attempt but worse, and ECM will be even worse. HEVC will be the last major ISO/ITU format.

[kurkosdr]

And from a purely technical perspective, a purely technical approach has good dividends. VVC as a codec outperforms AV1 in compression efficiency with substantially less complex decoder requirements. If VVC had AOM licensing, it would be broadly used today. But it doesn't, and so VVC use is only planned in particular markets (Brazil) that believe they've handed the IP issues for themselves.
Ok, you've saved a couple of cents in silicon costs and you've saved a couple of cents in bitrate costs. Now how much do you have to pay to the 20 different patent licensing entities in VVC? Didn't this undo your savings? Also, silicon costs will go down in time, and so will bitrate costs, but the VVC patent holders will probably try to milk you more in the future. Do you realise why broadcasters like Netflix and Disney+ won't go near VVC?

Terrestrial and satellite Broadcasters could be a market for VVC, since they have to deal with fixed bitrate allocations decided at the government level and want all the bitrate savings they can get, but they are stuck with HEVC for HDR and 4K HDR content for compatibility reasons and that won't change any time soon.

[FranceBB]

Terrestrial and satellite Broadcasters could be a market for VVC, since they have to deal with fixed bitrate allocations decided at the government level and want all the bitrate savings they can get, but they are stuck with HEVC for HDR and 4K HDR content for compatibility reasons and that won't change any time soon.

The pandemic and the economic crisis that followed afterwards didn't help either, so much so that the 8K productions are extremely rare and aside from a bunch of Japanese documentaries and a few Japanese movies, there's nothing actively produced in 8K. Some productions are shot in 6K and downscaled to 4K and in some rare cases like Das Boot (produced by our colleagues in Sky Deutschland) several shots were performed in 8K and then downscaled to 4K. Not only 8K TVs are nowhere near mass adoption (heck, they're basically nowhere to be seen), but aside from cinema productions and a few tv series, most countries are even finding the cost of making 4K live productions too high and are resorting to FULL HD 50p/60p log 10bit which is then converted to PQ (yes, PQ, not HLG) and upscaled to 4K to "trick" the customers (I'm looking at you FOX Sports). Anyway this is to say that we're nowhere near 8K, so the justification to move from H.265 HEVC to H.266 VVC, even for broadcasters, is really not there just yet considering that the gains at lower resolutions are fewer and that most TVs can't even decode it...

[kurkosdr]

The pandemic and the economic crisis that followed afterwards didn't help either, so much so that the 8K productions are extremely rare and aside from a bunch of Japanese documentaries and a few Japanese movies, there's nothing actively produced in 8K. Some productions are shot in 6K and downscaled to 4K and in some rare cases like Das Boot (produced by our colleagues in Sky Deutschland) several shots were performed in 8K and then downscaled to 4K. Not only 8K TVs are nowhere near mass adoption (heck, they're basically nowhere to be seen), but aside from cinema productions and a few tv series, most countries are even finding the cost of making 4K live productions too high and are resorting to FULL HD 50p/60p log 10bit which is then converted to PQ (yes, PQ, not HLG) and upscaled to 4K to "trick" the customers (I'm looking at you FOX Sports). Anyway this is to say that we're nowhere near 8K, so the justification to move from H.265 HEVC to H.266 VVC, even for broadcasters, is really not there just yet considering that the gains at lower resolutions are fewer and that most TVs can't even decode it...

I know I will sound like a broken record, but 8K is a solution looking for a problem to solve. The thought process behind 8K is "Hey, you know we can skip cutting this motherglass into four pieces but sell it as one big expensive panel with quadruple the resolution of 4K instead, right? Now, is there a problem we can apply this solution to?" I mean, just how large is your TV and how close are your viewing distances that you need more than 8.3 megapixels of resolution for video?

VVC would have been useful to make 4K HDR bitrates more tolerable, but this is not happening because broadcasters are locked into HEVC for 4K HDR (in the same way broadcasters ignored Mpeg-4 Part 2 because they were locked into Mpeg-2 for SD) and streaming is going to AV1.

8K is useful in video production, since you can slice and dice a video (for example centering shots) and still end up with an effective resolution of at least 4K. So, 8K cameras do solve a real-world problem. But I doubt they capture VVC, cameras typically use their own formats like Redcode.

[benwaggoner]

Ok, you've saved a couple of cents in silicon costs and you've saved a couple of cents in bitrate costs. Now how much do you have to pay to the 20 different patent licensing entities in VVC? Didn't this undo your savings? Also, silicon costs will go down in time, and so will bitrate costs, but the VVC patent holders will probably try to milk you more in the future. Do you realise why broadcasters like Netflix and Disney+ won't go near VVC?
Yeah, exactly. Which is why we're seeing AV1 HW heading towards ubiquity and VVC largely missing in action, despite designs being completed a while ago.

Terrestrial and satellite Broadcasters could be a market for VVC, since they have to deal with fixed bitrate allocations decided at the government level and want all the bitrate savings they can get, but they are stuck with HEVC for HDR and 4K HDR content for compatibility reasons and that won't change any time soon.
You mean for backwards compatibility reason? Set top boxes create so much friction for codec updates, since they have to be replaced wholesale to deprecate an old codec type. We saw MPEG-2 lasting a lot longer in those than any other consumer tech.

[benwaggoner]

I know I will sound like a broken record, but 8K is a solution looking for a problem to solve. The thought process behind 8K is "Hey, you know we can skip cutting this motherglass into four pieces but sell it as one big expensive panel with quadruple the resolution of 4K instead, right? Now, is there a problem we can apply this solution to?" I mean, just how large is your TV and how close are your viewing distances that you need more than 8.3 megapixels of resolution for video?
To visually resolve >4K signal you need to sit so close to the screen that the viewing angle to the sides of the screen are terrible. Even with expert viewers with unusually acute vision. The average person couldn't tell the difference in an uncompressed A/B test, let alone just watching compressed video (with the same bitrate for 4K and 8K). For real-time encoding at reasonable bitrates, 4K would typically look better due to fewer compression artifacts.

VVC would have been useful to make 4K HDR bitrates more tolerable, but this is not happening because broadcasters are locked into HEVC for 4K HDR (in the same way broadcasters ignored Mpeg-4 Part 2 because they were locked into Mpeg-2 for SD) and streaming is going to AV1.
MPEG-4 part 2 also suffered from codec licensing ambiguity (which inspired AVC to get its act together, a lesson since forgotten) and was around a 20-30% bitrate reduction, not enough to go through the pain of switching codecs for. When AVC came out with sane, clear licensing and 50% reduction, that was worth it.

8K is useful in video production, since you can slice and dice a video (for example centering shots) and still end up with an effective resolution of at least 4K. So, 8K cameras do solve a real-world problem. But I doubt they capture VVC, cameras typically use their own formats like Redcode.
Yeah, oversampling on production can always find a use. But those are going to be in professional formats, in the high end some sort of Beyer pattern recording ala RAW.

Distributing >4K for moving image content that people will watch the entire frame of at once just doesn't make sense.

Digital signage and other applications can find uses.

[hajj_3]

mpeg association looks like they are taking in consideration market conditions not just technical merit for their next gen video codec. Maybe they want 1 patent pool, maybe lower cost, maybe a guarantee of no streaming costs.

Introduction
To help ensure that actions are in accordance with industry needs, MPEG is conducting a ‘market needs’ study for a next-generation video codec. Both MPEG and non-MPEG members are invited to submit comments using this template.

These ‘market needs’ studies have been performed by MPEG in the past for a diverse range of technical areas. The template below is written in a generic way and has not been specifically modified for the specific next-generation video work. Market needs are considered to be complementary information to technical requirements and use cases. This template is requested for work prior to an exploratory stage, and for new amendments, versions or profiles of an existing standard.


source: https://www.mpeg.org/wp-content/uploads/mpeg_meetings/149_Geneva/w24910.zip

[Z2697]

mpeg association looks like they are taking in consideration market conditions not just technical merit for their next gen video codec. Maybe they want 1 patent pool, maybe lower cost, maybe a guarantee of no streaming costs.



source: https://www.mpeg.org/wp-content/uploads/mpeg_meetings/149_Geneva/w24910.zip

I think it's just "buzzword".
We thought they should've learned from the disaster of HEVC.
They gave us "EVC baseline" garbage and go "yay, free and better than AVC".

The reality is no one is eagerly asking for a next gen MPEG codec anymore.
I don't think AOM is the one and only 100% pure morally superior good-hearted godlike savior, but at least they are doing better than MPEG.

[ksec]

The reality is no one is eagerly asking for a next gen MPEG codec anymore.

It is not just MPEG video codec. It is any video codec. From internet companies POV.

The thing is, as I have been saying for many years and may be soon a decade. As bandwidth cost continue to drop, the incentive to invest into a new codec also continue to faint away. Because storage cost isn't dropping anywhere near as fast as bandwidth. This is specifically on the Internet and not broadcast. With the new explosion of AI, what the industry predicted bandwidth and port cost price falling would slow down turns out continue to accelerate.

The investment of encoding all videos file into a new Codec, and storing it, which cost CPU cycle and Storage does not make much of ROI to the saving of using less bandwidth. The wait and see approach means adoption continue to be slow. And this limit / slow down hardware maker from implementing it.

I think the bar for a next replacement codec needs to be 80% BD-Rate of x264 while being decoding and encoding efficient. That is a tall order. It may be somewhat counter- intuitive, what used to be internet / technology companies dictating video codec market acceptance in the past 10-15 years, will now leads to TV broadcasting leading or spear heading for changes. Because they are the ones with limited bandwidth, and they will need to pick and choose something that can be served in board casting and internet streaming. Something like Brazil TV 3.0. Hopefully VVC + LCEVC.

I really hope there is an JPEXG-XL moment that shows how something could be so much better.

[Z2697]

It is not just MPEG video codec. It is any video codec. From internet companies POV.

The thing is, as I have been saying for many years and may be soon a decade. As bandwidth cost continue to drop, the incentive to invest into a new codec also continue to faint away. Because storage cost isn't dropping anywhere near as fast as bandwidth. This is specifically on the Internet and not broadcast. With the new explosion of AI, what the industry predicted bandwidth and port cost price falling would slow down turns out continue to accelerate.

The investment of encoding all videos file into a new Codec, and storing it, which cost CPU cycle and Storage does not make much of ROI to the saving of using less bandwidth. The wait and see approach means adoption continue to be slow. And this limit / slow down hardware maker from implementing it.

I think the bar for a next replacement codec needs to be 80% BD-Rate of x264 while being decoding and encoding efficient. That is a tall order. It may be somewhat counter- intuitive, what used to be internet / technology companies dictating video codec market acceptance in the past 10-15 years, will now leads to TV broadcasting leading or spear heading for changes. Because they are the ones with limited bandwidth, and they will need to pick and choose something that can be served in board casting and internet streaming. Something like Brazil TV 3.0. Hopefully VVC + LCEVC.

I really hope there is an JPEXG-XL moment that shows how something could be so much better.

JPEG-XL is so much better only because we've been stuck with JPEG-1 for decades, plus other "modern" choices we have today are basically some frankenstein weird stuff.

[nhw_pulsar]

JPEG-XL is so much better only because we've been stuck with JPEG-1 for decades, plus other "new" choices we have today are basically some frankenstein weird stuff.

Hi,

When you speak about these "new" choices that resemble frankenstein weird stuff, are you talking about the other new choices on the Internet that are WebP, AVIF, HEIC? If so, you think they're weird because they're based on a video codec (JPEG XL would be the evolution of JPEG image codec, and WebP is VP8 intra and AVIF and HEIC are VP10/HEVC intra)?

Or with these "new" choices, you make reference to other (weird) stuff?

Cheers,
Raphael

[Z2697]

Hi,

When you speak about these "new" choices that resemble frankenstein weird stuff, are you talking about the other new choices on the Internet that are WebP, AVIF, HEIC? If so, you think they're weird because they're based on a video codec (JPEG XL would be the evolution of JPEG image codec, and WebP is VP8 intra and AVIF and HEIC are VP10/HEVC intra)?

Or with these "new" choices, you make reference to other (weird) stuff?

Cheers,
Raphael

Yeah, I mean the video codec based format. It might be a bit exaggerated to call them frankenstein. But they are inferior to JPEG-XL in almost every aspect.

But the "distance" between JPEG-1 and JPEG-XL is the key. It's like comparing MPEG2 to HEVC.

[nhw_pulsar]

Yeah, I mean the video codec based format. It might be a bit exaggerated to call them frankenstein. But they are inferior to JPEG-XL in almost every aspect.

But the "distance" between JPEG-1 and JPEG-XL is the key. It's like comparing MPEG2 to HEVC.

Ok, thank you for your reply.


Cheers,
Raphael

[birdie]

JXL has not had its moment yet.

Zero major web browsers support it out of the box.

Firefox sort of supports it except it needs to be enabled via about:config that is hidden for mobile users and nearly impossible to discover for desktop users.

[GeoffreyA]

It's baffling that we've got a premier image format collecting dust, whilst the "Frankenformats" are strutting their feathers on the runway of the web.

[kurkosdr]

It's baffling that we've got a premier image format collecting dust, whilst the "Frankenformats" are strutting their feathers on the runway of the web.
Because the "Frankenformats" were the first viable royalty-free successor to JPEG, so they captured the market. Also, the "Frankenformats" are widely used by Google, so if some NPE comes out of the woodwork and starts demanding content fees, they will realistically go after Google or some other big player first. Instead, if Google is not using JPEG XL and your website does, any NPEs will go after you.

It's what killed EVC too: VP9 and AV1 captured the market EVC was going for, and using EVC means painting a big red target on your back for NPEs looking to sue for content fees, since no big player uses EVC.

[GeoffreyA]

Because the "Frankenformats" were the first viable royalty-free successor to JPEG, so they captured the market. Also, the "Frankenformats" are widely used by Google, so if some NPE comes out of the woodwork and starts demanding content fees, they will realistically go after Google or some other big player first. Instead, if Google is not using JPEG XL and your website does, any NPEs will go after you.

It's what killed EVC too: VP9 and AV1 captured the market EVC was going for, and using EVC means painting a big red target on your back for NPEs looking to sue for content fees, since no big player uses EVC.

Good points. And well said.

[benwaggoner]

I think the bar for a next replacement codec needs to be 80% BD-Rate of x264 while being decoding and encoding efficient. That is a tall order. It may be somewhat counter- intuitive, what used to be internet / technology companies dictating video codec market acceptance in the past 10-15 years, will now leads to TV broadcasting leading or spear heading for changes. Because they are the ones with limited bandwidth, and they will need to pick and choose something that can be served in board casting and internet streaming. Something like Brazil TV 3.0. Hopefully VVC + LCEVC.
I presume you mean 80% BD-RATE savings over x264. so same quality at 20% the bitrate.

That's already achievable with modern codecs for 4K HDR, as both were well out of scope when H.264 was originally designed focused on 8-bit SD SDR. Even 1080p needed High Profile's 8x8 block option to do that efficiently.

[benwaggoner]

But the "distance" between JPEG-1 and JPEG-XL is the key. It's like comparing MPEG2 to HEVC.
Sheesh, more like H.261 to HEVC!

[Tommy Carrot]

ECM encoder windows build (MSVC):
https://www.mediafire.com/file/xgy8ozes3nof3wp/ecm_250410.7z/file

[hajj_3]

https://i.imgur.com/Xqi07BK.png

It looks like MPEG might be ratifying ECM in mid-2026, just 1 year away. It is the "enhanced compression beyond vvc capabilities".

[ksec]

It looks like MPEG might be ratifying ECM in mid-2026, just 1 year away. It is the "enhanced compression beyond vvc capabilities".

I hope they have testing kit out soon this time around. Time to market has been a much debated topic for codec in recent years.

I also wonder how well it would do with LCEVC.

[benwaggoner]

I hope they have testing kit out soon this time around. Time to market has been a much debated topic for codec in recent years.

I also wonder how well it would do with LCEVC.
LC-EVC is out of loop for the decoder, so it should work as well with this as anything else.

[hajj_3]

ECM has been delayed from mid-2026 to the end of 2026, it is the "enhanced compression beyond VVC capabilities":

https://i.ibb.co/67cMf86m/ecm.png

[nhw_pulsar]

Yes, it seems that "enhanced compression beyond VVC capabilities" will be ECM. JVET released a Call for Evidence/codecs that should perform against ECM anchors.

This time, something new is that JVET said for the evaluation it could be a mix of PSNR scores and subjective visual reviews by expert viewers, but it seems that in the end, it is PSNR that will make the decision, and yes it seems normally ECM the best PSNR performer, even end-to-end learned video codecs don't seem ready to challenge ECM on PSNR at now...

We'll soon see what has been selected in their next press release of the last meeting.

[birdie]

An update (https://www.streamingmediaglobal.com/Articles/News/Featured-News/AI-helps-InterDigital-reach-beyond-VVC-in-race-to-develop-next-gen-codec-171972.aspx):

InterDigital had begun work on ECM in 2021, a year after VVC was finalised. Designed purely for research and without taking account of encoder complexity, by the end of 2024 ECM had reached version 18 and was demonstrating a coding gain of 28% over VVC.

In purely visual tests the company claims it can achieve 50% gains for some sequences.

“Overall more than two thirds of the sequences were gaining 30%,” says Le Léannec.

Competing participants in this next stage, including InterDigital, will now work until January 2027 before presenting results back to ISO/ITU for assessment. Finalisation of the new standard is expected by end of 2029.

Key research aspects include optimising the trade-off between bitrate and visual fidelity, developing fast encoding methods suitable for constrained devices, and advancing performance in emerging use cases like HDR, 8K, gaming, and user-generated content.

The standardisation phase will start after January 2027 and will be a collaborative effort led by JVET.



No word on addressing the licensing mess.

[hajj_3]

An update (https://www.streamingmediaglobal.com/Articles/News/Featured-News/AI-helps-InterDigital-reach-beyond-VVC-in-race-to-develop-next-gen-codec-171972.aspx):



No word on addressing the licensing mess.

2029 is crazy! VVC was ratified in 2020, that is such a huge amount of time between new releases. I guess they know that VVC isn't going to take off so they want to spend more time making such ECM has a large compression improvement in order to incentivise people to adopt it. AV2 is going to kick VVC's ass and there will be no competition from ECM for years.