Versatile Video Coding (VVC) / H.266: HEVC successor

[kurkosdr]

Originally Posted by birdie

Some interesting thoughts about the codec from Jan Ozer.

Yeah, none of this matters. AVC was pretty much hardware-decode only in its early years, I remember it bringing my laptop's Core Duo T2500 to its knees on software that didn't support the "assisted H.264 decoding" offered by the laptop's ATI Mobility Radeon X1600 (and had to do the decoding entirely in software as a result). And that was on a brand-new high-end laptop (surpassed only by Alienwares and such). HEVC was also notorious for bringing Ultrabooks with U-class Intel CPUs to their knees (if they didn't have HEVC hardware decode).

None of this prevented adoption. Hardware manufacturers quickly added support because they had to. The "no-content-fees" policy of AVC made it the default format for HD (pushing VC-1 and then VP8 to irrelevance), and HEVC became the de facto standard for HDR and 4K for premium services and broadcast (the premium services thing caused by the WebM camp dragging their feet on HDR support), so again, manufacturers added support because they had to add support.

VVC's problem is that nobody uses VVC. You have to be insane to deal with the 20 different patent pools that claim patents on it and pay whatever content fees they ask to in order gain marginal bitrate savings over AV1, the economics are not there. Yes, I know, Brazil. The TVs sold there will support VVC decoding in hardware. See? Where there is demand by the content, there is hardware decoding.

Originally Posted by ksec

Its not like x266 is even here anyway

Why all the emojis? What demand is there to justify speedy x266 development? The MulticoreWare people are more focused on MV-HEVC support for x265 (which has demand because of the Apple Vision Pro) than x266. And yes, the fact Apple chose MV-HEVC and not whatever the VVC equivalent is for Apple Vision Pro, despite Apple Vision Pro thing launching 4 years after VVC was formalized and despite being a clean-sheet design with no backwards compatibility constraints (Apple Vision Pro doesn't even use standard Dolby Vision) says a lot about VVC.

[modus-ms325c]

maybe AOM can develop their own take on MV-HEVC (either in the form of shoving it to AV1 or designing AV2 with it in mind) or something.

[kurkosdr]

Originally Posted by modus-ms325c

maybe AOM can develop their own take on MV-HEVC (either in the form of shoving it to AV1 or designing AV2 with it in mind) or something.

The technology behind MVC and MV-HEVC is patented to the teeth. MPEG LA once had a separate patent pool solely for MVC before they rolled the patents into the AVC pool:

https://web.archive.org/web/20220306.../programs/mvc/

https://web.archive.org/web/20240204...s/MVC-att1.pdf

And that's just for MVC (not MV-HEVC). And since the technology in MVC is entirely novel (and not an iteration of ideas pioneered many years ago like 2D video compression is), some of those patents are probably blocking.

So, in order for AOM to offer this technology, there needs to be enough demand for full-resolution stereoscopic content for VR headsets other than the Apple Vision Pro and the patents have to expire. In the meantime, H-SBS should do, after all that's what all stereoscopic video on YouTube is (not that their official player supports it natively anymore, but a few people still upload H-SBS content).

[Z2697]

Originally Posted by kurkosdr

the patents have to expire

It's not like that they had to wait for patents to expire to create something that can compete with HEVC. (and even outcompete)
They already did fight through a ton of patents.
But yeah, there should be enough incentive to "push" them through that.

[kurkosdr]

Originally Posted by Z2697

It's not like that they had to wait for patents to expire to create something that can compete with HEVC. (and even outcompete)

Oh, they did, part of the reason VP9 was possible is because lots of key patents from MPEG-2 had expired by then. So, it was a matter of using brute force (for example larger block sizes and longer prediction lengths for inter-frame) plus some innovative workarounds such as AltRef frames.

But if the core technology is patented, unless you can find a genius workaround, you are blocked.

[birdie]

Originally Posted by kurkosdr

says a lot about VVC.

I concur, sadly Jan Ozer is not here.

[Z2697]

Originally Posted by kurkosdr

Oh, they did, part of the reason VP9 was possible is because lots of key patents from MPEG-2 had expired by then. So, it was a matter of using brute force (for example larger block sizes and longer prediction lengths for inter-frame) plus some innovative workarounds such as AltRef frames.

But if the core technology is patented, unless you can find a genius workaround, you are blocked.

On2 codecs before VP9 do exist.
Maybe not as big of a deal, but still there.
Who knows how the patents system work.

[benwaggoner]

Originally Posted by modus-ms325c

maybe AOM can develop their own take on MV-HEVC (either in the form of shoving it to AV1 or designing AV2 with it in mind) or something.

I'm not aware of any implementations of it, but there is some basic stuff in specs for how it could be done. Not enough for interoperable implementations, but it's a pretty straightforward extrapolation if the market is there.

Alas, the stereoscopic market wasn't even big enough to include it in UHD Blu-ray. VR headsets using MV-HEVC are really the only new place stereoscopic has been used in over a decade, and the viewer eyeball hours per month are still going to be very low.

We'd need to see some combination of a much bigger market or much bigger bitrate savings (storage is limited on headsets) to justify creation of a new format, decoder IP, professional grade encoders, etcetera.

Possibly with AV2, as it offers a lot better savings. AV1 wasn't really enough more efficient than HEVC intrinsically to have a sustainable, practical bitrate advantage until encoders matured enough over the last 1-2 years.

New codecs always present as competing against the current state of competing codecs, with numbers that are mostly reference encoder versus reference encoder. But in the 3-5 years it takes to get good, high performance, psychovisually optimized production encoders, the last generation encoders are improving as well, so the goalposts keep moving. With sustained efforts eventually more efficient core bitstream features can pull ahead.

[Z2697]

Maybe there couble be a more "universal" solution, like an extension that's compatible with multiple formats, similar to how LC-EVC supports multiple codecs.

[benwaggoner]

Originally Posted by Z2697

Maybe there couble be a more "universal" solution, like an extension that's compatible with multiple formats, similar to how LC-EVC supports multiple codecs.

Yeah. At a high level good stereoscopic encoding is making the alternate eye an enhancement layer that can reference its own past frames and frames in the primary eye at the same time. Existing temporal enhancement layer implementations can be largely repurposed with some extra signaling, with the alternate eye functioning as a frame behind the main eye at the same timestamp.

There are implementation tweaks to keep the visual quality in both eyes consistant without excess bitrate overhead, but getting the basic thing working is much more bitstream syntax than low level DSP stuff.

[ShortKatz]

Originally Posted by kurkosdr

Why all the emojis? What demand is there to justify speedy x266 development? The MulticoreWare people are more focused on MV-HEVC support for x265 (which has demand because of the Apple Vision Pro) than x266.

Seems to me they are more focused on improving ARM support at the moment, didn't see a MV-HEVC commit since November 2024.

[benwaggoner]

Originally Posted by kurkosdr

Why all the emojis? What demand is there to justify speedy x266 development? The MulticoreWare people are more focused on MV-HEVC support for x265 (which has demand because of the Apple Vision Pro) than x266. And yes, the fact Apple chose MV-HEVC and not whatever the VVC equivalent is for Apple Vision Pro, despite Apple Vision Pro thing launching 4 years after VVC was formalized and despite being a clean-sheet design with no backwards compatibility constraints (Apple Vision Pro doesn't even use standard Dolby Vision) says a lot about VVC.

It's also a lot easier to extend a HEVC decoder to support multi view than it is to add a whole new decoder. It's a pretty trivial amount of extra mm^2 of wafer to add it that way, and Ateme already had a commercial MV-HEVC encoder available.

Plus Apple had added HW accelerated encoding for MV-HEVC in the M1 processor, so they already had an ecosystem. Getting decent encoders available was a harder lift than the decoders, I strongly suspect.

[benwaggoner]

Originally Posted by ShortKatz

Seems to me they are more focused on improving ARM support at the moment, didn't see a MV-HEVC commit since November 2024.

MV-HEVC is a pretty straightforward feature; once it's working, I don't know how much additional refinement is needed. You'll need some rate control to make sure the quality of the two views is equivalent, but that should have been pretty extensible based on the existing B-frame rate control.

And ARM is a huge deal right now given the rapid growth of ARM in data centers, and secondarily on M-series Macs. We've seen speed almost triple with current builds on recent ARM processors compare to what was there 2-3 years ago. Which already had some ARM optimizations, but didn't dive nearly as deep as x86-64, and ARM got some pretty great SIMD extensions to take advantage of.

The throughput/$ of current x265 builds on our latest Graviton processor AWS instances is pretty amazing.

[benwaggoner]

Originally Posted by Z2697

On2 codecs before VP9 do exist.
Maybe not as big of a deal, but still there.
Who knows how the patents system work.

Yeah:

• VP6 was the leading codec in Flash for years
• VP3 was the basis for the Theora codec, and a decent CD-ROM codec for its era. Poor rate control, though.
• VP7 was used in Move Networks' pioneering adaptive streaming implementation. Great tech, but they failed due to adopting a business model that left them competing with their partners at every stage of the encoding & distribution process.
• TrueMotion and TrueMotion-S (retconned to VP1 & 2, Back when On2 was still The Duck Corporation) were decently popular CD-ROM codecs, used in a bunch of Star Trek titles and other stuff. I wrote an article about those for DV Magazine 25+ years ago.

I had a meeting with On2 before Google bought them, sheesh almost 20 years ago, where we realized I'd been working with the company on various projects longer than any current employee had worked there.

Wow, I'm such an industry graybeard now, and I have the beard to show it! I was actually doing video encoding early enough that I was excited about the potential of these new-fangled CD-ROMs coming to market.

[LigH]

Ben will ever since be known as "Video Kenobi".

[foxyshadis]

Originally Posted by benwaggoner

Yeah:

• VP6 was the leading codec in Flash for years

I'd even say VP6 propped up flash long past its sell-by date, since there was no other good way (at the time) to get video delivered to every browser. As soon as there was, it disappeared into the night.

For all their faults, particularly their constant braggadocio, On2 definitely did their part to push the state of the art and competitiveness in video encoding, even after they were bought by Google. Who definitely had no idea how to make a video codec, so even if VP8 was a hot mess of an "open" codec, it helped build the foundation for AV1.

[LigH]

New uploads: [Windows][GCC 15.1.0][64 bit]

Fraunhofer VVC Encoder ver. 1.13.1 d091d89

Fraunhofer VVC Decoder ver. 3.0.0 cf443af

[benwaggoner]

Originally Posted by foxyshadis

I'd even say VP6 propped up flash long past its sell-by date, since there was no other good way (at the time) to get video delivered to every browser. As soon as there was, it disappeared into the night.

Yeah. VP6's real strength was in advanced post processing including grain synthesis, deblocking, deringing, etcetera. Turning those features off revealed quite mediocre baseband video, way overtuned for PSNR with lots of the sorts of artifacts that causes.

Once Flash supported H.264 with much better compression efficiency, better tuned and faster encoders, and much better pricing, that was that for the most part.

VP7 did get used in Move Networks' pioneering ABR solution, but Move's attempt to monetize and thus compete with every stage of the video encoding, delivery, and playback pipeline, and thus competing with most potential partners, caused them to fade quite quickly.

For all their faults, particularly their constant braggadocio, On2 definitely did their part to push the state of the art and competitiveness in video encoding, even after they were bought by Google. Who definitely had no idea how to make a video codec, so even if VP8 was a hot mess of an "open" codec, it helped build the foundation for AV1.

They did a good amount of valuable work back to the early CD ROM days with the TrueMotion codecs (VP1 & VP2), back when they were called The Duck Corporation. IDR-only codecs originally made for performant playback and random access for discs where compression efficiency wasn't paramount.

[Yups]

Originally Posted by birdie

For VVC, almost 100% of TVs don't support HW VVC decoding and their SoCs are too weak to support it in software. For mobile users, again no HW support, so VVC videos will kill your battery.

Maybe 50 thousand or so Lunar Lake owners? Nope, that won't work for PC/laptop users either.

Looks like VVC will not be adopted in anything consumer any time soon or maybe ever.

Oh, what about the [warez] scene? Nothing. Completely ignored because the encoder is crazy slow.

Panther Lake mobile is going to support VVC decoding but that's a 2026 generation. As for AMD and Nvidia it's not looking good even for next year. Maybe with their next gen GPUs which is more likely a 2027 generation. This is a very slow adoption, much slower than AV1.