AOmedia next (AV2) codec

[ksec]

Originally Posted by soresu

A bit odd to start work on v14 so early then if they are finished with the research work on AV2.

IMHO someone(s) got an itchy trigger finger and made promises the AOM couldn't realistically keep given the state the project was in at the time.

Their press releases basically stated that they were releasing the actual codec by the end of 2025 - when in reality the reference encoder is not even into beta yet and the spec is nowhere to be seen.

For all the mess MPEG has allowed to happen, AOM could still learn some lessons from their regular meetings that leave little doubt as to what is going on.

I am not surprised, it was actually the same when they released AV1.0.

I was planning to say it when they released AV2, but I may as well say it now, AV2 was supposed to be released in 2020 on their original roadmap and AV3 in 2022.

We are now 2026.....

[hajj_3]

Originally Posted by ksec

I am not surprised, it was actually the same when they released AV1.0.

I was planning to say it when they released AV2, but I may as well say it now, AV2 was supposed to be released in 2020 on their original roadmap and AV3 in 2022.

We are now 2026.....

I think that is because OEMS said that they didn't want that as they would have to use lots of die space to keep supporting lots of new codecs. I'm glad they didn't rush out a successor. Most phones don't even have an AV1 decoder so convincing them to have lots of new decoders would have been impossible.

[soresu]

Originally Posted by hajj_3

I think that is because OEMS said that they didn't want that as they would have to use lots of die space to keep supporting lots of new codecs. I'm glad they didn't rush out a successor. Most phones don't even have an AV1 decoder so convincing them to have lots of new decoders would have been impossible.

No surprises there - I'm honestly surprised that Qualcomm and Apple ever caved given how invested they were in trying to push an MPEG standard instead to no avail, but aside from them basically every other OEM had AV1 decoders in their SoCs years earlier (including all the PC GPU ODMs from 2020), and contrary to popular thought QC and Apple are far from the only players in phone town, especially in the east, and with lower end phones in the west.

[Z2697]

Well you know, Apple is one of the funding members of AOMedia actually.

[benwaggoner]

Originally Posted by hajj_3

I doubt it. They probably know of some patents that a violated but want to stay quiet instead of announce them now so that the AOM can't remove those tools from the final version so they can extract money out of companies.

Both are likely scenarios in my personal opinion.

There's also a huge difference between "such a good essential patent that it will hold up under appeal" and "a bunch of patents that it is easier to pay off the holder for than any one organization to spend years and millions taking to court." Patent trolls rely on the latter a lot.

[soresu]

Originally Posted by Z2697

Well you know, Apple is one of the funding members of AOMedia actually.

Which doesn't mean that much in the grand scheme of things.

What really matters is development + patent contributions and adoption rate.

They joined 2 months before AV1 released (ie no dev contribution), and as we know their hw asic adoption rate was pretty terrible.

Best case scenario their patents helped AV2 development and bolstered the defense of AV1 vs Sisvel - but I have no idea about that on either count.

If anyone has any information about Apple employees contributing to AVM work directly then by all means chime in, I'm open to being convinced.

We'll see how fast their adoption rate is for AV2 given they have no excuse for being late to the party this time around.

[ksec]

Originally Posted by Z2697

Well you know, Apple is one of the funding members of AOMedia actually.

Apple's name wasn't or its logo even added until very late in the cycle. IRRC about a year and a half later.

And their logo wasn't even official.

[benwaggoner]

Originally Posted by hajj_3

I think that is because OEMS said that they didn't want that as they would have to use lots of die space to keep supporting lots of new codecs. I'm glad they didn't rush out a successor. Most phones don't even have an AV1 decoder so convincing them to have lots of new decoders would have been impossible.

Yeah. The rule of thumb is that a new codec has to offer the potential of reducing bitrate in around half for the same quality to be adopted, or unlock important new content classes. Releasing them more frequently than those sorts of gains can be delivered just splits the installed base. And die space is becoming a bigger concern now that we can't expect Moore's Law to drop the price of a given decoder block in half every couple of years. New codecs have to be carefully designed to offer the maximum compression efficiency gains in the lowest mm^2 delta possible. AV1 didn't do well on that metric, as it inherited lots of x86 software decode design. A VVC decoder is actually a fair bit smaller than a AV1 decoder while offering superior compression efficiency. Hardware decoder complexity seems to have been a much bigger priority in AV2 development.

MPEG-2 was maybe 20% more efficient than MPEG-1, but added support for interlaced. MPEG-4 part 2 was potentially maybe 20-30% more efficient than MPEG-2, which wasn't enough to overcome its own patent licensing challenges. Thus H.264, which brought really innovative new technology to hit (and eventually exceed) 50% reduction compared to MPEG-2, and a chastened set of IP holders who signed on to the MPEG-LA terms. HEVC brought 4K and HDR and about 50% bitrate reduction, enabling it to overcome its licensing ambiguity (and leveraging good will and hopeful expectations from H.264). VVC does over 50% bitrate reductions, but no new content types of mass market interest (and sustains the bad experiences and dashed expectations from HEVC).

So far AV1 hasn't been enough better or close enough to universal than H.264 to replace it, although it has seen additive use, particularly for SD content, from user-generated content (which doesn't need hardware DRM or hardware decoder), and more recently from premium streaming companies (as the critical mass of devices with HW DRM and decoding has crossed ROI thresholds).

My personal guess on the day when a streaming provider could viably stop having a H.264 fallback available is likely to hit sometime in the early 2030's. Younger readers may not remember, but some MPEG-4 part 2 and VC-1 fallbacks weren't finally deprecated until around ten years ago, after the last pre-H.264 devices dropped out of significant use.

[benwaggoner]

Originally Posted by soresu

Best case scenario their patents helped AV2 development and bolstered the defense of AV1 vs Sisvel - but I have no idea about that on either count.

Having essential patents for one aspect of a codec isn't much defense against a non-practicing entity having an essential patent on another. When two companies working in good faith have a bunch of patents, they can cross-license. But a NPE is only it in for the money, and so will act in parasitic revenue-maximize way with little regard for the health of the overall market.

We'll see how fast their adoption rate is for AV2 given they have no excuse for being late to the party this time around.

I don't know if "excuse" is fair. Putting a decode block on a SoC still causes incremental heat and power draw even when it is unused. Not a huge deal in a desktop GPU, but a very real concern in a modern phone trying to optimize for every percent of battery life and peak compute performance. And of course, more area directly increases cost of a SoC, as it reduces per-wafer yield. Apple likely had a target process density and market penetration where AV1 decode had a sufficiently non-negative ROI, and introduced it then. If AV2 decode adds significant incremental complexity on top of AV1, we can expect different companies to time introduction based on similar logic.

All going well, we could start seeing AV2 HW decoders in mass-market products in the latter part of 2028, and maybe a few lower volume things earlier than that. There are a fair number of long poles that don't start moving until the bitstream spec is fully locked down. AV2 is close enough that I am sure initial design work has begun. But real commitments for the 2026 simulations for 2027 test SoCs for 2028 products won't happen before the spec is final.

[soresu]

Originally Posted by benwaggoner

I don't know if "excuse" is fair. Putting a decode block on a SoC still causes incremental heat and power draw even when it is unused. Not a huge deal in a desktop GPU, but a very real concern in a modern phone trying to optimize for every percent of battery life and peak compute performance. And of course, more area directly increases cost of a SoC, as it reduces per-wafer yield. Apple likely had a target process density and market penetration where AV1 decode had a sufficiently non-negative ROI, and introduced it then. If AV2 decode adds significant incremental complexity on top of AV1, we can expect different companies to time introduction based on similar logic.

My wording of excuse was based on the other smartphone and streaming SoC players besides QC and Apple which had AV1 decoders in their SoCs years earlier.

That's early 2020 for Amlogic, April 2020 for Mediatek, and January 2021 for Samsung.

Even Rockchip's long delayed RK3588 came out in late 2021/early 2022 - at least 6-9 months before the first QC SD 8 Gen2 in November 2022 and a good year and a half before Apple's first SoC in September 2023.

TV SoCs are known for lagging more than a little behind the state of the art, but they still had AV1 support significantly before these flagship mobile SoC/device brands.

I know all of the many excuses both QC and Apple have used when this subject comes up, but frankly based on the amount of money they throw about on state of the art process nodes and silicon design I simply don't buy any of it for a second.

It would be far easier to take such explanations as honest truth rather than corporate dissembling if they didn't have potential skin in the MPEG/proprietary codec game, the details of which have been addressed on this very forum multiple times.

When you add in the time to ASIC decoder implementation after the HEVC 02/12 standard for both QC and Apple it does display a pretty obvious proprietary engineering bias by comparison:

1.5 yrs for SD 805, and 3.5 yrs for Apple A9.

vs AV1 timeframe:

4.5 yrs for SD 8 Gen2 (SoC only - most devices usally lag months behind), and 5.5 yrs for Apple A17.

[ksec]

I know all of the many excuses both QC and Apple have used when this subject comes up, but frankly based on the amount of money they throw about on state of the art process nodes and silicon design I simply don't buy any of it for a second.

It is not just about process. All TV SoC including many other electronics devices and even Intel iGPU can do hardware AV1 decode before Mobile SoC because they have a much higher power budget to work with and are on lower end process where iteration is far cheaper. These hardware IP are far less energy usage sensitive. Anywhere from 2-4W. Mobile SoC not only have to do it in Sub 1W. they do it in sub 0.5W with later iteration to 0.3W. Which is both AVC and HEVC.

And an extreme power efficient design for new codec just doesn't come in day 1. Let alone on new node. If you have an ultra efficient AV2 hardware decoder IP ready today it will still be 2027 iPhone at the earliest. But given IP design and integration takes time even an 2028 release is too optimistic. And that is assuming the spec is done and you have testing kit ready along with other conformance requirement sets. Which we have zero today. In reality unless they are well prepared, which in typical AOM fashion they are not, 2030 is when you should expect hardware decoder on Mobile Phone for AV2.

That is why recent MPEG meetings has ( finally ) talked about Codec Time to Market. May be stop releasing spec without all the realistic requirements around it.

And all of that is ignoring delays in management decisions, patients issues, whether we should put resources into designing the IP in the first place, video decoder block updating schedule. A new Soc doesn't;t always mean a new decoder block. All the other nuance which people overlook and take it for granted. Shipping Hardware isn't magic, most people, especially software developers got so used to Internet doing the delivery and logistics and shipping software correcting with a press of a button on SaaS, or ship a new version to App Store to approve in days and forget the whole Hardware shipping takes months and cant easily be updated. There are a lot of testing involved.

And lastly, 2026 marks the year I have been saying this for 10 years. Every single day passed is another day favouring good old H.264 AVC High Profile. Storage Cost isn't falling as it used to, in fact in recent few months it is increasing, and the trend will be continue the increase till 2028 / 2029. Storing video in another format has cost depending on their business models. i.e Youtube, Facebook and Netflix as well as Broadcasting are all very different. But thanks to AI Networking cost is projected to keep falling. It is actually easier to broadcast 20%+higher bitrate AVC on the internet then switching to another codec. Since AVC is a baseline requirement and not looking to go away any time soon. That makes LCEVC looks like an increasingly attractive option. And as much as people wants to believe HEVC will be taking over soon, as I would have also liked, 40% of Android dont support HEVC Hardware decode. Google Pixel 5 in 2020 dont support it or have difficulty playing HEVC files.

I was really hoping AOM, having learned its lesson with AV1, would have done a much better job with AV2 and potentially take over H.264. But as of 10 of January. This doesn't look like the case.

[Spyros]

VideoLAN - @videolan

First demo of upcoming new open codec AV2, with playback inside VLC4 at #CES2026 on a normal laptop!

AV2 is coming soon and it's developed by the @a4omedia
to have the best royalty-free video codec!

If you want to see the demo during #CES26, contact us!

Source: Tweet @ XCancel