AOmedia next (AV2) codec

[Tommy Carrot]

I've tested the latest build, and it's finally not crashing on higher resoulutions, so i could get a broader impression. Definitely a noticable improvement over AV1, detail retention and motion stability are both significantly better. The overall quality is pretty close to VVC, albeit much slower, but that is understandable for a work-in progress reference encoder. Interesting behaviour of the encoder is that if i dont specify keyframe intervals, the first keyframe will be encoded to a significantly higher quality, but it takes so much space that the overall quality for a given bitrate is always better if i set keyframe intervals (in my case 150).

[CodecWar]

Its just conducted a study comparing AV1 and AV2 video codecs : https://codecwar.com/compare/av1-vs-av2

[CodecWar]

Originally Posted by CodecWar

Its just conducted a study comparing AV1 and AV2 video codecs : https://codecwar.com/compare/av1-vs-av2

PS

Need to log in to see the all graphs

[Jamaika]

How to use this for ffmpeg?

[libavm-av2 @ 0000028e412307a0] v5.0.0-76120a1
Output #0, webm, to 'output_avm.webm':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.15.100
  Stream #0:0(und): Video: av1, yuv420p10le(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 3000 kb/s, 29.97 fps, 1k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.30.102 libavm-av2
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1(und): Audio: opus, 48000 Hz, stereo, flt, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.30.102 libopus
[vost#0:0/libavm-av2 @ 0000028e40f16e30] Error submitting a packet to the muxer: Invalid data found when processing input
[out#0/webm @ 0000028e40dd67d0] Error muxing a packet

[hajj_3]

Here is a list of the remaining european VC-1 patents and their expiry dates:

EP 1,528,812 - 2024-08-11
EP 2,290,991 - 2024-08-11
EP 1,661,387 - 2024-09-02
EP 1,656,793 - 2024-09-03
EP 1,656,794 - 2024-09-03
EP 1,658,726 - 2024-09-03
EP 1,665,761 - 2024-09-03
EP 2,285,113 - 2024-09-03
EP 2,285,114 - 2024-09-03
EP 2,285,115 - 2024-09-03
EP 2,323,398 - 2024-09-03
EP 2,323,399 - 2024-09-03
EP 2,323,406 - 2024-09-03
EP 2,451,161 - 2024-09-03
EP 2,466,894 - 2024-09-03
EP 2,466,895 - 2024-09-03
EP 1,549,064 - 2024-11-08

AV2 can use these in under 12 months if needed. Most european h264 patents expire by March 2024 too for the original spec of h264 that is. All Xvid patents have expired too.

[benwaggoner]

Originally Posted by hajj_3

Here is a list of the remaining european VC-1 patents and their expiry dates:

EP 1,528,812 - 2024-08-11
EP 2,290,991 - 2024-08-11
EP 1,661,387 - 2024-09-02
EP 1,656,793 - 2024-09-03
EP 1,656,794 - 2024-09-03
EP 1,658,726 - 2024-09-03
EP 1,665,761 - 2024-09-03
EP 2,285,113 - 2024-09-03
EP 2,285,114 - 2024-09-03
EP 2,285,115 - 2024-09-03
EP 2,323,398 - 2024-09-03
EP 2,323,399 - 2024-09-03
EP 2,323,406 - 2024-09-03
EP 2,451,161 - 2024-09-03
EP 2,466,894 - 2024-09-03
EP 2,466,895 - 2024-09-03
EP 1,549,064 - 2024-11-08

AV2 can use these in under 12 months if needed. Most european h264 patents expire by March 2024 too for the original spec of h264 that is. All Xvid patents have expired too.

Not that VC-1 is of practical use these days, but the VC-1 reference encoder was pretty much the commercial WMV implementation, so included a lot of interesting techniques that would now be available. Its adaptive deadzone implementation had some really good implementation (and thank goodness, as its delta QP signaling had a high overhead).
It also did a lot of early HEVC-like variable interprediction block size and shape stuff. And a decent overlap transform.

I don't know how applicable any of that would be to AV2, however. It's been a long time since I lived and breathed VC-1 optimization.

[hajj_3]

AV2 talk on youtube by google engineers from August 2024: https://www.youtube.com/watch?v=bCwjTVJ3F3w

They said that AV2 compression is currently 24-25% better than AV1 for normal content and video games and 30%+ better for screen-content e.g powerpoint presentations, screen sharing etc. They said that they are focusing more on 1080p rather than 4k as most content on social media is 1080p. They said they are focusing on making the amount of die space low for hardware decoders.

Google's goal is a 40% compression improvement over AV1. Apple would be happy with 25-30%. They are aiming for a 2x increase in complexity. They are aiming to finalise AV2 in 1-2yrs. They are considering adding alpha channel support and considering adding up to 16 bit colour depth support. Presumably alpha channel and 16bit are for a future AVIF image format.

[Jamaika]

Okay it's amazing. Unfortunately av1 creates video quite slowly compared to hevc and av2 even slower. Although everyone writes that the processor load is the same. However visually the avm project looks better and runs faster than ECM i.e. h267.
I don't know why the C++ additions in av2 are under <pthread.h> for gcc???
Not all the fixes work under gcc. The question is whether gcc doesn't work or the fixes have been there for half a year???
Yes av2 can be added to libavif, libwebp2 but no one has foreseen viewer.

Detele error: CONFIG_ENABLE_IBC_NAT, CONFIG_IBP_WEIGHT

[ksec]

Originally Posted by hajj_3

AV2 talk on youtube by google engineers from August 2024: https://www.youtube.com/watch?v=bCwjTVJ3F3w

They said that AV2 compression is currently 24-25% better than AV1 for normal content and video games and 30%+ better for screen-content e.g powerpoint presentations, screen sharing etc. They said that they are focusing more on 1080p rather than 4k as most content on social media is 1080p. They said they are focusing on making the amount of die space low for hardware decoders.

Google's goal is a 40% compression improvement over AV1. Apple would be happy with 25-30%. They are aiming for a 2x increase in complexity. They are aiming to finalise AV2 in 1-2yrs. They are considering adding alpha channel support and considering adding up to 16 bit colour depth support. Presumably alpha channel and 16bit are for a future AVIF image format.

Looks like they have ( finally ) learned their lesson with AV1. Looking forward to AV2.

[oibaf]

Originally Posted by Jamaika

Okay it's amazing. Unfortunately av1 creates video quite slowly compared to hevc and av2 even slower. Although everyone writes that the processor load is the same. However visually the avm project looks better and runs faster than ECM i.e. h267.
I don't know why the C++ additions in av2 are under <pthread.h> for gcc???
Not all the fixes work under gcc. The question is whether gcc doesn't work or the fixes have been there for half a year???
Yes av2 can be added to libavif, libwebp2 but no one has foreseen viewer.

Detele error: CONFIG_ENABLE_IBC_NAT, CONFIG_IBP_WEIGHT

You should report it here: https://gitlab.com/AOMediaCodec/avm/-/issues

[Jamaika]

I know but nobody is in a hurry here. I don't even know if there is interest in the subject. I treat it only as a curiosity. Someone can take a picture of AV2 with the provided link to libwebp2. Compare it with WEBP, WEBP2, AV1, JXL, JXS, JP2000HT and that's it. The websites don't provide information today when the h267 or av2 codecs will be implemented?

[hajj_3]

Originally Posted by Jamaika

I know but nobody is in a hurry here. I don't even know if there is interest in the subject. I treat it only as a curiosity. Someone can take a picture of AV2 with the provided link to libwebp2. Compare it with WEBP, WEBP2, AV1, JXL, JXS, JP2000HT and that's it. The websites don't provide information today when the h267 or av2 codecs will be implemented?

The latest H.267 roadmap (page4) shows that it will be ratified in late 2026/early 2027. As for AV2 The 4 month old video i posted said they are planning to finalise it in 1-2yrs.

[Jamaika]

Originally Posted by hajj_3

The latest H.267 roadmap (page4) shows that it will be ratified in late 2026/early 2027. As for AV2 The 4 month old video i posted said they are planning to finalise it in 1-2yrs.

However something did come up.
MPEG Immersive Video v.2 2023-2035 - I understand that it will not be renewed.
https://github.com/MPEGGroup/isobmff/tags
File Format (ISOBMFF) v.9 - ... but where to download git to mpeg-5 container.
... Audio Coding for Machines 2025-2027 sounds mysterious too

[ksec]

Originally Posted by hajj_3

AV2 talk on youtube by google engineers from August 2024: https://www.youtube.com/watch?v=bCwjTVJ3F3w

They said that AV2 compression is currently 24-25% better than AV1 for normal content and video games and 30%+ better for screen-content e.g powerpoint presentations, screen sharing etc. They said that they are focusing more on 1080p rather than 4k as most content on social media is 1080p. They said they are focusing on making the amount of die space low for hardware decoders.

Google's goal is a 40% compression improvement over AV1. Apple would be happy with 25-30%. They are aiming for a 2x increase in complexity. They are aiming to finalise AV2 in 1-2yrs. They are considering adding alpha channel support and considering adding up to 16 bit colour depth support. Presumably alpha channel and 16bit are for a future AVIF image format.

I just rewatched the video, I have high doubt on how they will achieve an extra 15% compression with next 12 months and ratified the standard in another 6.

I cant help but thank given the history of AOM constantly over promise and under deliver I am as skeptical as usual. I hope they have a free AV2 hardware decoder ready as well that is energy efficient as they originally promised with AV1.

[olduser217]

Originally Posted by ksec

I just rewatched the video, I have high doubt on how they will achieve an extra 15% compression with next 12 months and ratified the standard in another 6.

I cant help but thank given the history of AOM constantly over promise and under deliver I am as skeptical as usual. I hope they have a free AV2 hardware decoder ready as well that is energy efficient as they originally promised with AV1.

Referring to the history of AV1, I won't be surprised if AV2 standard is completed beyond the initial promised date (unless AOM compromises the compression target).

The free AV2 hardware design may not be much useful for major SoC companies, as it may not suit the existing architecture of their design that includes other standards.

[kurkosdr]

Originally Posted by hajj_3

Here is a list of the remaining european VC-1 patents and their expiry dates:

EP 1,528,812 - 2024-08-11
EP 2,290,991 - 2024-08-11
EP 1,661,387 - 2024-09-02
EP 1,656,793 - 2024-09-03
EP 1,656,794 - 2024-09-03
EP 1,658,726 - 2024-09-03
EP 1,665,761 - 2024-09-03
EP 2,285,113 - 2024-09-03
EP 2,285,114 - 2024-09-03
EP 2,285,115 - 2024-09-03
EP 2,323,398 - 2024-09-03
EP 2,323,399 - 2024-09-03
EP 2,323,406 - 2024-09-03
EP 2,451,161 - 2024-09-03
EP 2,466,894 - 2024-09-03
EP 2,466,895 - 2024-09-03
EP 1,549,064 - 2024-11-08

AV2 can use these in under 12 months if needed. Most european h264 patents expire by March 2024 too for the original spec of h264 that is. All Xvid patents have expired too.

Why would a company like Google, which distributes video from US territory (YouTube) and encoder and decoder software from US territory, care about the fact that the European patents have expired? There is the theoretical possibility of Google moving its operations completely outside the US and changing headquarters to Europe, but realistically, it's not happening.

Same for H.264, until the US patents expire too, the technologies they describe are out of reach for WebM. In fact, since WebM is meant to be royalty-free on a (reasonably) global basis, the patents for every major market have to expire, so that possibly includes Brazil.

The expiration of VC-1 patents for every country except the US and Brazil sometime this year will benefit companies that make Blu-ray players (since they'll only have to pay patent royalties for products sold in the US and Brazil for VC-1), but that's it.

[kurkosdr]

Originally Posted by CodecWar

Its just conducted a study comparing AV1 and AV2 video codecs : https://codecwar.com/compare/av1-vs-av2

If you are still reading this thread, the website doesn't work.

[benwaggoner]

Originally Posted by olduser217

Referring to the history of AV1, I won't be surprised if AV2 standard is completed beyond the initial promised date (unless AOM compromises the compression target).

My sense from various stakeholders is that the standard will be done this year. I don't think they'd hold it another year to get a few more percent savings.

Among other things, the market needs seem clearer than ever, with VVC licensing not having coalesced into something straightforward yet.

[benwaggoner]

Originally Posted by kurkosdr

Why would a company like Google, which distributes video from US territory (YouTube) and encoder and decoder software from US territory, care about the fact that the European patents have expired? There is the theoretical possibility of Google moving its operations completely outside the US and changing headquarters to Europe, but realistically, it's not happening.

Google distributes video from data centers all around the world, not just the USA. Any company of that size with that pervasive a reach has a legal locus pretty much everywhere.

Same for H.264, until the US patents expire too, the technologies they describe are out of reach for WebM. In fact, since WebM is meant to be royalty-free on a (reasonably) global basis, the patents for every major market have to expire, so that possibly includes Brazil.

Except for patents held by AOM member companies, who license them for AOM use. Which is a pretty decent slice of codec IP.

[kurkosdr]

Originally Posted by benwaggoner

Google distributes video from data centers all around the world, not just the USA. Any company of that size with that pervasive a reach has a legal locus pretty much everywhere.

Yes, but they also distribute video from the US, so unless those US patents expire, they cannot use the technologies those US patents describe in AV2, because then AV2 goes out of reach for their US datacenters.

Originally Posted by benwaggoner

Except for patents held by AOM member companies, who license them for AOM use. Which is a pretty decent slice of codec IP.

It's patent-encumbered but royalty-free, in fact, AOM doesn't have a royalty structure, if you trigger the "defensive termination" clauses you can't pay to license the AOM patents even if you wanted to.

(when I said "the patents for every major market have to expire" above, I obviously meant the ones not held by AOM)