Alliance for Open Media codecs

[Ayumu111]

Originally Posted by benwaggoner

Premium quality ABR live streaming still requires a whole lot of fast cores. I doubt we'll ever see a fixed-function hardware encoder be quality-competitive again, given the ever-rising complexity of modern codecs.

I concur with Ben that high-quality adaptive bitrate (ABR) live streaming requires powerful hardware to attain high-quality compression. As codec standards and standards become increasingly complex, hardware encoders may find it difficult to keep up. Software-based streaming solutions that leverage the power of fast core processors will continue to be the preferred method for high-quality live streaming.

[Spyros]

The AOMedia Research Workshop Europe 2023 will take place in a few days (June 19th), in Ghent, Belgium.

The conference program is in the image below, more details about each presentation can be found at the official website.

The focus seems to be on AV2 and a retrospective of AV1 development and the technologies that were used in it.

[nisha_gomes]

Originally Posted by benwaggoner

It is weird they are comparing to H.264 Baseline VeryFast, which is a lot faster than realtime these days. I presume the relative savings would be a lot lower compared to real world encodes.
Matching x265 slow with high density hardware encoding with AV1 is really impressive. High quality live AV1 has been a challenge so far.

The progress in comparing AV1 with highly efficient encoding methods like H.264 Baseline VeryFast and x265 slow is impressive, especially considering the historical challenges in achieving high-quality live AV1 encoding.

[LigH]

New uploads: (MSYS2; MinGW32 / MinGW64: GCC 13.1.0)

AOM v3.6.1-843-g93ca91b96

rav1e 0.6.1 p20230711-1-g2447fcc

dav1d 1.2.1-43-g9278a14

SVT-AV1 v1.6.0-2-gba18204e

avif 0.11.1-41ed914
dav1d [dec]:1.2.1-43-g9278a14, aom [enc/dec]:3.6.1-842-gff3c4c8a6, rav1e [enc]:0.6.1 (p20230711-1-g2447fcc)

[mzso]

Hi!
Is Youtube still the only notable user of AV1 nowadays?

[butterw2]

Just like VP9 ?

[mzso]

Originally Posted by butterw2

Just like VP9 ?

Not sure how that's relevant.

[benwaggoner]

Originally Posted by mzso

Not sure how that's relevant.

It is historically relevant, as YouTube has been the primary user and driver of VP8, 9, and 10/AV1. VP8 and VP9 were really limited by the reference/sample encoder being highly tuned for YouTube scenarios and not well suited for for other use cases, like highly parallelized encoding of premium content. That caused some chicken/egg problems that prevented VP9 from getting encoders nearly as refined as those for H.264 and HEVC. I don't think VP9 as a bitstream could have outperformed HEVC in real-world applications, but VP9 encoders weren't able to capture as much of the potential of the bitstream as H.264 and HEVC encoders have, as the market for those commercial encoders is orders of magnitude higher.

For example, VP8/9 had very serialized decoding and limited on encoding. Which was fine for YouTube, which encodes in little slices of times on lots of different edge compute nodes as available. But premium content where time-to-publish is essential and encoding 3x more slowly to reduce bitrate by 10% is excellent ROI, it's a big limitation. And when secure DRM is a hard requirement and dropped frames cause cancelled subscriptions or purchase/rental refunds, having reliably fast decoders, preferably HW, are much more important.

While user generated content gets a lot of eyeball hours, the economic value of improved quality and speed/efficiency tradeoffs are very different, and the $/hour invested are a tiny fraction of what premium content gets.

AV1 has a much healthier ecosystem than any prior VPx version (going back to VP3 even), with more and better encoders available. So it's a compare and contrast.

[CodecWar]

Hello! Today, I bring some exciting news about the AV2 codec developed by the AVM team. As many of you might be aware, AV2 is the successor to AV1 and aims to provide even better video compression and quality. In this post, I'll be discussing the recent advancements in AV2 and how it compares to its predecessor, AV1.

Active Development of AV2 Codec:
The AVM team has been hard at work, continuously refining the AV2 codec to enhance its performance. Over the last couple of years, they have released several research tags to showcase their progress. In August 2021, the research2 tag was unveiled, followed by research3 in May 2022, and research4 in April 2023.

Comparative Analysis on AOM AV2 CTC Streaming Classes:
Recently, a comprehensive report was published that compared the performance of these three AV2 implementations on AOM AV2 CTC Streaming classes, namely A5, A4, A3, and A2, focusing on resolutions up to FullHD. The objective was to evaluate the quality metrics of PSNR (Peak Signal-to-Noise Ratio), VMAF (Video Multi-Method Assessment Fusion), and SSIM (Structural Similarity Index) for the different research releases.

Steady Quality Improvements:
The results of the study were nothing short of impressive. Across all quality metrics, there was a consistent improvement as the research progressed from version 2 to version 4. For example, the PSNR: Y (luma) with BD-Rate data aggregation showed a remarkable 8.76% improvement between research3 and research4. This demonstrates the commitment of the AVM team to refining their codec and pushing the boundaries of video compression technology.

Spread and Variance:
It's important to note that while the best and worst results also improved with each iteration, the spread and variance between different video streams increased. This suggests that the AV2 codec is becoming more specialized in handling certain types of video content optimally.

Conclusions from the Study:
Based on the findings of this study, some significant conclusions were drawn:

On average, the AOM_post-AV1 (research 3 released in 2022 ) codec showed an 8.76% improvement over the AOM_post-AV1(research 2 released in 2021) reference codec across all video streams.
Comparing the average performance on all video streams, the AOM_post-AV1 (research 4 - Apr23) codec outperformed the AOM_post-AV1 (research 3 - 2022) reference codec by an impressive 15.15%.
In summary, the AV2 codec is showing immense promise and dedication from the AVM team. As the development progresses, it is evident that AV2 is significantly surpassing its predecessor, AV1, in terms of video quality and compression efficiency. However, with the increasing specialization, the codec might perform differently depending on the content being encoded.

It's an exciting time for video compression enthusiasts, and we can't wait to see how AV2 evolves further. Feel free to share your thoughts and insights in the comments below! Let's discuss the future of video codecs together.

Source: https://codecwar.com/compare/av2-research-evolution

[benwaggoner]

If all we are seeing on net is a 15.15% efficiency improvement, that is very overwhelming. A new codec needs a plausible path to around a 100% efficiency improvement (same quality, half the bitrate) to displace a prior one, or needs to add a substantial and compelling new feature (like broadcast-grade interlaced SD and HD video for MPEG-2, HDR for HEVC). 15% over AV1 is still a lot worse compared to VVC, which will have HW decoders shipping in products by end of 2023. It's also lower than the delta from VVC to current post-VVC test implementations from MPEG.

[hajj_3]

Originally Posted by benwaggoner

If all we are seeing on net is a 15.15% efficiency improvement, that is very overwhelming.

*underwhelming. AV2 isn't being ratified right now, so i wouldn't pay much attention that it is only 15% better. In a year i'm sure it will be more than that.

[Beelzebubu]

Originally Posted by CodecWar

On average, the AOM_post-AV1 codec showed an 8.76% improvement over the AOM_post-AV1 reference codec across all video streams.
Comparing the average performance on all video streams, the AOM_post-AV1 codec outperformed the AOM_post-AV1 reference codec by an impressive 15.15%.

So X outperformed X by 8.76% and 15.15%? I assume some "AOM_post-AV1" labels above are meant to be something else?

[CodecWar]

Originally Posted by Beelzebubu

So X outperformed X by 8.76% and 15.15%? I assume some "AOM_post-AV1" labels above are meant to be something else?

I've updated the post to make it clear that it compares the same codec in 3 releases :
research2 Aug21,
research3 - May22,
research 4 - Apr22

[benwaggoner]

Originally Posted by hajj_3

*underwhelming. AV2 isn't being ratified right now, so i wouldn't pay much attention that it is only 15% better. In a year i'm sure it will be more than that.

Yeah, no one would release a new codec with that small a delta. Us graybeards remembered what happened when MPEG-4 pt 2 came out with only maybe 20-30% more efficiency than MPEG-2. Just not worth the hassle.

Hence H.264, which was a pretty ground-up rethink of how to encode video that could deliver on 100% efficiency improvements.

[benwaggoner]

Originally Posted by CodecWar

I've updated the post to make it clear that it compares the same codec in 3 releases :
research2 Aug21,
research3 - May22,
research 4 - Apr22

Thank you for the clarification.

I'm still surprised that the rate of improvement isn't higher, though. Lots of H.264 patents have expired, and thus don't have to be worked around, plus years more of academic research and work on the specific codec.

I am guessing that the ML stuff isn't refined enough to deliver material benefits yet. That's probably the make or break for AV2. I find myself atypically unsure of how big a real-world difference that could make. We could well wind up still not knowing even at release, as there are SO many new ways an encoder can be tuned and can optimize a bitstream with ML as an option. Incorporating ML model overhead into rate control alone is a chewy interesting challenge.

[hajj_3]

avif 1.0.0 has been released: https://github.com/AOMediaCodec/liba...ses/tag/v1.0.0

[ksec]

Originally Posted by CodecWar

Conclusions from the Study:
Based on the findings of this study, some significant conclusions were drawn:

On average, the AOM_post-AV1 (research 3 released in 2022 ) codec showed an 8.76% improvement over the AOM_post-AV1(research 2 released in 2021) reference codec across all video streams.
Comparing the average performance on all video streams, the AOM_post-AV1 (research 4 - Apr23) codec outperformed the AOM_post-AV1 (research 3 - 2022) reference codec by an impressive 15.15%.
In summary, the AV2 codec is showing immense promise and dedication from the AVM team. As the development progresses, it is evident that AV2 is significantly surpassing its predecessor, AV1, in terms of video quality and compression efficiency. However, with the increasing specialization, the codec might perform differently depending on the content being encoded.

Source: https://codecwar.com/compare/av2-research-evolution

This doesn't make any sense.

the AOM_post-AV1 (research 4 - Apr23) codec outperformed the AOM_post-AV1 (research 3 - 2022) reference codec by an impressive 15.15%.

So R4 outperform R3 by 15.15%. Except your link below shown

AOM_post-AV1 codec is better than AOM_post-AV1 reference codec by 15.15%, in average on all video streams.

The Reference is R2. i.e R4 is 15.15% better than R*2*.

Improvement in percentage also doesn't make sense. It is BD-Rate ( Which the report actually suggest ). i.e Reduction. ( Which is why I assume benwaggoner was confused and mentioned 100% improvement as in efficiency as 50% BD-Rate )

Now if I remember correctly, R2 is something like 10% BD-Rate on AV1. So that is 15.5% on top of 10%, multiply to less than 25%.

Which if I have to guess AV2 at R4 isn't even at VVC level. On the other hand, latest H.267 development is already at 30%+ on top of VVC.

Did I mention AV2 was suppose to be out in ( *cough* ) 2020.

So yes, I do think it is quite underwhelming. Previously Google was so fixated that next gen video codec being ML based and doesn't seems to offer any alternative. I wonder if R1- R4 is still ML based.

[Gravitator]

Originally Posted by ksec

On the other hand, latest H.267 development is already at 30%+ on top of VVC.

Привет!
Will they go above blocks 128?

[benwaggoner]

Originally Posted by ksec

Which is why I assume benwaggoner was confused and mentioned 100% improvement as in efficiency as 50% BD-Rate )

Yeah, this is a place where we could be crisper in the industry on the terminology. A reduction in bitrate is the reciprocal of an improvement in efficiency. So, a 100% improvement in compression efficiency IS a 50% reduction in bitrate. A 300% improvement in compression efficiency is a 75% reduction in bitrate. We could save some real confusion if we clearly specified which we meant when we are talking % better.

So yes, I do think it is quite underwhelming. Previously Google was so fixated that next gen video codec being ML based and doesn't seems to offer any alternative. I wonder if R1- R4 is still ML based.

I gather that there was some demonstration of value in some ML tool in the latest tests, but shy of 2%. AV2 doesn't seem to have any big organizing principle for how to make a big jump in compression efficiency, or any novel tools that would really have a big impact.

I'd hoped that the expiration of so many H.264 and other patents since AV1 would allow the use of tools AOM had wanted to use in AV1 but couldn't. It doesn't seem to have been the case. Perhaps the VPx core structure to avoid patents was so far off from the MPEG codec development process that tools weren't easily reusable?

[hajj_3]

libaom 3.7 is out: https://aomedia.googlesource.com/aom/+/refs/tags/v3.7.0