AOmedia next (AV2) codec

[birdie]

Looks like it's being quite actively worked on:

https://gitlab.com/AOMediaCodec/avm

[Tommy Carrot]

I wonder how much efficiency improvement were they able to achieve so far over AV1. I know the bitstream is not finalized, so the current version is not very usable in the long run, but i'd still like to try it out, so i'd appreciate if someone could share a windows build of this codec. I'm also curious about ECM (https://vcgit.hhi.fraunhofer.de/ecm/ECM).

[paul97]

https://gitlab.com/AOMediaCodec/avm/-/pipelines/839933609 I suggest this, though at the moment the deblocking is a bit destroyed.
aomenc.exe C:\Users\Use\Videos\cs.yuv --cpu-used=9 --threads=4 --min-qp=57 --width=1366 --height=768 --max-qp=124 -o C:\Users\Use\Videos\single.ivf -
aomdec C:\Users\Use\Videos\single.ivf -o C:\Users\Use\Videos\csao.y4m
ffmpeg.exe -i C:\Users\Use\Videos\csao.y4m C:\Users\Use\Videos\aua.png
on i3 330m I can't decode more than one frame with yuv and aomenc

Job Build (x86_64-mingw-gcc): [inspection-accounting]
just search inspect and will appear

[Tommy Carrot]

Thank you! I've taken (x86_64-mingw-gcc): [nasm] from the latest main branch build, and after some DLL hunting, i managed to get it work.

The encoder crashes on most of my test clips, so i could only test it on some SD samples. So far it seems to be a few % more efficient than AV1, so from a purely technical standpoint, it's still quite far behind VVC, at least in SD stuff.

[hajj_3]

experimental AV2 support added to avif: https://github.com/AOMediaCodec/libavif/pull/1361/commits/4ab0eade81837e6c5709fff8829316c5e273de82

[birdie]

experimental AV2 support added to avif: https://github.com/AOMediaCodec/libavif/pull/1361/commits/4ab0eade81837e6c5709fff8829316c5e273de82

That's weird. AVIF was finalized or so it seemed.

Now it's not? Now you can have AVIF images based on AV2 features and no existing application will be able to read them?

[nevcairiel]

Its no different from having a MP4 file with a new video codec. Why redefine the container if it works?

[Jamaika]

AV1 SIMD does not work with GCC 13.0.0
Does AV2 SIMD work with GCC 13.0.0?

[Tommy Carrot]

I've tested the latest build, and it's finally not crashing on higher resoulutions, so i could get a broader impression. Definitely a noticable improvement over AV1, detail retention and motion stability are both significantly better. The overall quality is pretty close to VVC, albeit much slower, but that is understandable for a work-in progress reference encoder. Interesting behaviour of the encoder is that if i dont specify keyframe intervals, the first keyframe will be encoded to a significantly higher quality, but it takes so much space that the overall quality for a given bitrate is always better if i set keyframe intervals (in my case 150).

[CodecWar]

Its just conducted a study comparing AV1 and AV2 video codecs : https://codecwar.com/compare/av1-vs-av2

[CodecWar]

Its just conducted a study comparing AV1 and AV2 video codecs : https://codecwar.com/compare/av1-vs-av2

PS

Need to log in to see the all graphs

[Jamaika]

How to use this for ffmpeg?
[libavm-av2 @ 0000028e412307a0] v5.0.0-76120a1
Output #0, webm, to 'output_avm.webm':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf60.15.100
Stream #0:0(und): Video: av1, yuv420p10le(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 3000 kb/s, 29.97 fps, 1k tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
encoder : Lavc60.30.102 libavm-av2
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Stream #0:1(und): Audio: opus, 48000 Hz, stereo, flt, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
encoder : Lavc60.30.102 libopus
[vost#0:0/libavm-av2 @ 0000028e40f16e30] Error submitting a packet to the muxer: Invalid data found when processing input
[out#0/webm @ 0000028e40dd67d0] Error muxing a packet

[hajj_3]

Here is a list of the remaining european VC-1 patents and their expiry dates:

EP 1,528,812 - 2024-08-11
EP 2,290,991 - 2024-08-11
EP 1,661,387 - 2024-09-02
EP 1,656,793 - 2024-09-03
EP 1,656,794 - 2024-09-03
EP 1,658,726 - 2024-09-03
EP 1,665,761 - 2024-09-03
EP 2,285,113 - 2024-09-03
EP 2,285,114 - 2024-09-03
EP 2,285,115 - 2024-09-03
EP 2,323,398 - 2024-09-03
EP 2,323,399 - 2024-09-03
EP 2,323,406 - 2024-09-03
EP 2,451,161 - 2024-09-03
EP 2,466,894 - 2024-09-03
EP 2,466,895 - 2024-09-03
EP 1,549,064 - 2024-11-08

AV2 can use these in under 12 months if needed. Most european h264 patents expire by March 2024 too for the original spec of h264 that is. All Xvid patents have expired too.

[benwaggoner]

Here is a list of the remaining european VC-1 patents and their expiry dates:

EP 1,528,812 - 2024-08-11
EP 2,290,991 - 2024-08-11
EP 1,661,387 - 2024-09-02
EP 1,656,793 - 2024-09-03
EP 1,656,794 - 2024-09-03
EP 1,658,726 - 2024-09-03
EP 1,665,761 - 2024-09-03
EP 2,285,113 - 2024-09-03
EP 2,285,114 - 2024-09-03
EP 2,285,115 - 2024-09-03
EP 2,323,398 - 2024-09-03
EP 2,323,399 - 2024-09-03
EP 2,323,406 - 2024-09-03
EP 2,451,161 - 2024-09-03
EP 2,466,894 - 2024-09-03
EP 2,466,895 - 2024-09-03
EP 1,549,064 - 2024-11-08

AV2 can use these in under 12 months if needed. Most european h264 patents expire by March 2024 too for the original spec of h264 that is. All Xvid patents have expired too.
Not that VC-1 is of practical use these days, but the VC-1 reference encoder was pretty much the commercial WMV implementation, so included a lot of interesting techniques that would now be available. Its adaptive deadzone implementation had some really good implementation (and thank goodness, as its delta QP signaling had a high overhead).
It also did a lot of early HEVC-like variable interprediction block size and shape stuff. And a decent overlap transform.

I don't know how applicable any of that would be to AV2, however. It's been a long time since I lived and breathed VC-1 optimization.

[hajj_3]

AV2 talk on youtube by google engineers from August 2024: https://www.youtube.com/watch?v=bCwjTVJ3F3w

They said that AV2 compression is currently 24-25% better than AV1 for normal content and video games and 30%+ better for screen-content e.g powerpoint presentations, screen sharing etc. They said that they are focusing more on 1080p rather than 4k as most content on social media is 1080p. They said they are focusing on making the amount of die space low for hardware decoders.

Google's goal is a 40% compression improvement over AV1. Apple would be happy with 25-30%. They are aiming for a 2x increase in complexity. They are aiming to finalise AV2 in 1-2yrs. They are considering adding alpha channel support and considering adding up to 16 bit colour depth support. Presumably alpha channel and 16bit are for a future AVIF image format.

[Jamaika]

Okay it's amazing. Unfortunately av1 creates video quite slowly compared to hevc and av2 even slower. Although everyone writes that the processor load is the same. However visually the avm project looks better and runs faster than ECM i.e. h267.
I don't know why the C++ additions in av2 are under <pthread.h> for gcc???
Not all the fixes work under gcc. The question is whether gcc doesn't work or the fixes have been there for half a year???
Yes av2 can be added to libavif, libwebp2 but no one has foreseen viewer.

Detele error: CONFIG_ENABLE_IBC_NAT, CONFIG_IBP_WEIGHT

[ksec]

AV2 talk on youtube by google engineers from August 2024: https://www.youtube.com/watch?v=bCwjTVJ3F3w

They said that AV2 compression is currently 24-25% better than AV1 for normal content and video games and 30%+ better for screen-content e.g powerpoint presentations, screen sharing etc. They said that they are focusing more on 1080p rather than 4k as most content on social media is 1080p. They said they are focusing on making the amount of die space low for hardware decoders.

Google's goal is a 40% compression improvement over AV1. Apple would be happy with 25-30%. They are aiming for a 2x increase in complexity. They are aiming to finalise AV2 in 1-2yrs. They are considering adding alpha channel support and considering adding up to 16 bit colour depth support. Presumably alpha channel and 16bit are for a future AVIF image format.

Looks like they have ( finally ) learned their lesson with AV1. Looking forward to AV2.

[oibaf]

Okay it's amazing. Unfortunately av1 creates video quite slowly compared to hevc and av2 even slower. Although everyone writes that the processor load is the same. However visually the avm project looks better and runs faster than ECM i.e. h267.
I don't know why the C++ additions in av2 are under <pthread.h> for gcc???
Not all the fixes work under gcc. The question is whether gcc doesn't work or the fixes have been there for half a year???
Yes av2 can be added to libavif, libwebp2 but no one has foreseen viewer.

Detele error: CONFIG_ENABLE_IBC_NAT, CONFIG_IBP_WEIGHT

You should report it here: https://gitlab.com/AOMediaCodec/avm/-/issues

[Jamaika]

I know but nobody is in a hurry here. I don't even know if there is interest in the subject. I treat it only as a curiosity. Someone can take a picture of AV2 with the provided link to libwebp2. Compare it with WEBP, WEBP2, AV1, JXL, JXS, JP2000HT and that's it. The websites don't provide information today when the h267 or av2 codecs will be implemented?

[hajj_3]

I know but nobody is in a hurry here. I don't even know if there is interest in the subject. I treat it only as a curiosity. Someone can take a picture of AV2 with the provided link to libwebp2. Compare it with WEBP, WEBP2, AV1, JXL, JXS, JP2000HT and that's it. The websites don't provide information today when the h267 or av2 codecs will be implemented?

The latest H.267 roadmap (https://www.mpeg.org/wp-content/uploads/2024/12/Roadmap_after_MPEG_148.pdf) (page4) shows that it will be ratified in late 2026/early 2027. As for AV2 The 4 month old video i posted said they are planning to finalise it in 1-2yrs.

[Jamaika]

The latest H.267 roadmap (https://www.mpeg.org/wp-content/uploads/2024/12/Roadmap_after_MPEG_148.pdf) (page4) shows that it will be ratified in late 2026/early 2027. As for AV2 The 4 month old video i posted said they are planning to finalise it in 1-2yrs.
However something did come up.
MPEG Immersive Video v.2 2023-2035 - I understand that it will not be renewed.
https://github.com/MPEGGroup/isobmff/tags
File Format (ISOBMFF) v.9 - ... but where to download git to mpeg-5 container.
... Audio Coding for Machines 2025-2027 sounds mysterious too

[ksec]

AV2 talk on youtube by google engineers from August 2024: https://www.youtube.com/watch?v=bCwjTVJ3F3w

They said that AV2 compression is currently 24-25% better than AV1 for normal content and video games and 30%+ better for screen-content e.g powerpoint presentations, screen sharing etc. They said that they are focusing more on 1080p rather than 4k as most content on social media is 1080p. They said they are focusing on making the amount of die space low for hardware decoders.

Google's goal is a 40% compression improvement over AV1. Apple would be happy with 25-30%. They are aiming for a 2x increase in complexity. They are aiming to finalise AV2 in 1-2yrs. They are considering adding alpha channel support and considering adding up to 16 bit colour depth support. Presumably alpha channel and 16bit are for a future AVIF image format.

I just rewatched the video, I have high doubt on how they will achieve an extra 15% compression with next 12 months and ratified the standard in another 6.

I cant help but thank given the history of AOM constantly over promise and under deliver I am as skeptical as usual. I hope they have a free AV2 hardware decoder ready as well that is energy efficient as they originally promised with AV1.

[olduser217]

I just rewatched the video, I have high doubt on how they will achieve an extra 15% compression with next 12 months and ratified the standard in another 6.

I cant help but thank given the history of AOM constantly over promise and under deliver I am as skeptical as usual. I hope they have a free AV2 hardware decoder ready as well that is energy efficient as they originally promised with AV1.

Referring to the history of AV1, I won't be surprised if AV2 standard is completed beyond the initial promised date (unless AOM compromises the compression target).

The free AV2 hardware design may not be much useful for major SoC companies, as it may not suit the existing architecture of their design that includes other standards.

[kurkosdr]

Here is a list of the remaining european VC-1 patents and their expiry dates:

EP 1,528,812 - 2024-08-11
EP 2,290,991 - 2024-08-11
EP 1,661,387 - 2024-09-02
EP 1,656,793 - 2024-09-03
EP 1,656,794 - 2024-09-03
EP 1,658,726 - 2024-09-03
EP 1,665,761 - 2024-09-03
EP 2,285,113 - 2024-09-03
EP 2,285,114 - 2024-09-03
EP 2,285,115 - 2024-09-03
EP 2,323,398 - 2024-09-03
EP 2,323,399 - 2024-09-03
EP 2,323,406 - 2024-09-03
EP 2,451,161 - 2024-09-03
EP 2,466,894 - 2024-09-03
EP 2,466,895 - 2024-09-03
EP 1,549,064 - 2024-11-08

AV2 can use these in under 12 months if needed. Most european h264 patents expire by March 2024 too for the original spec of h264 that is. All Xvid patents have expired too.
Why would a company like Google, which distributes video from US territory (YouTube) and encoder and decoder software from US territory, care about the fact that the European patents have expired? There is the theoretical possibility of Google moving its operations completely outside the US and changing headquarters to Europe, but realistically, it's not happening.

Same for H.264, until the US patents expire too, the technologies they describe are out of reach for WebM. In fact, since WebM is meant to be royalty-free on a (reasonably) global basis, the patents for every major market have to expire, so that possibly includes Brazil.

The expiration of VC-1 patents for every country except the US and Brazil sometime this year will benefit companies that make Blu-ray players (since they'll only have to pay patent royalties for products sold in the US and Brazil for VC-1), but that's it.

[kurkosdr]

Its just conducted a study comparing AV1 and AV2 video codecs : https://codecwar.com/compare/av1-vs-av2
If you are still reading this thread, the website doesn't work.

[benwaggoner]

Referring to the history of AV1, I won't be surprised if AV2 standard is completed beyond the initial promised date (unless AOM compromises the compression target).
My sense from various stakeholders is that the standard will be done this year. I don't think they'd hold it another year to get a few more percent savings.

Among other things, the market needs seem clearer than ever, with VVC licensing not having coalesced into something straightforward yet.

[benwaggoner]

Why would a company like Google, which distributes video from US territory (YouTube) and encoder and decoder software from US territory, care about the fact that the European patents have expired? There is the theoretical possibility of Google moving its operations completely outside the US and changing headquarters to Europe, but realistically, it's not happening.
Google distributes video from data centers all around the world, not just the USA. Any company of that size with that pervasive a reach has a legal locus pretty much everywhere.

Same for H.264, until the US patents expire too, the technologies they describe are out of reach for WebM. In fact, since WebM is meant to be royalty-free on a (reasonably) global basis, the patents for every major market have to expire, so that possibly includes Brazil.
Except for patents held by AOM member companies, who license them for AOM use. Which is a pretty decent slice of codec IP.

[kurkosdr]

Google distributes video from data centers all around the world, not just the USA. Any company of that size with that pervasive a reach has a legal locus pretty much everywhere.
Yes, but they also distribute video from the US, so unless those US patents expire, they cannot use the technologies those US patents describe in AV2, because then AV2 goes out of reach for their US datacenters.


Except for patents held by AOM member companies, who license them for AOM use. Which is a pretty decent slice of codec IP.
It's patent-encumbered but royalty-free, in fact, AOM doesn't have a royalty structure, if you trigger the "defensive termination" clauses you can't pay to license the AOM patents even if you wanted to.

(when I said "the patents for every major market have to expire" above, I obviously meant the ones not held by AOM)

[benwaggoner]

Yes, but they also distribute video from the US, so unless those US patents expire, they cannot use the technologies those US patents describe in AV2, because then AV2 goes out of reach for their US datacenters.
At Google's scale, they absolutely could encode in different countries for distribution in specific countries in order to get around patent stuff. At least for Premium content; YouTube has quite different cost models.

It's patent-encumbered but royalty-free, in fact, AOM doesn't have a royalty structure, if you trigger the "defensive termination" clauses you can't pay to license the AOM patents even if you wanted to.
Correct. AOM provides both a list of "here's patents you can use royalty-free if you're using an AOM codec" and "here's companies that can't sue you for using an AOM codec."

Patent farms and submarine patents are the biggest risks, in my opinion, as they're not ecosystem stakeholders but simply are trying to extract as much revenue as possible irrespective any ecosystem damage.

Thank goodness for Unified Patents! They do great work invalidating low-quality patents and making patent farms have to work harder and face more risk in ecosystem disrupting activites.

[paul97]

please anyone can build avm merge request cdef enhancement or master for windows ten sse4? thanks in advance

[Jamaika]

Not everyone will be happy. For those willing, AVM codec AVX.
https://www.sendspace.com/file/xc74zd

[benwaggoner]

please anyone can build avm merge request cdef enhancement or master for windows ten sse4? thanks in advance
What are you trying to do on such ancient hardware? Given the complexity of the in-development AV2 encoder today, I doubt you'd be able to get much encoded before the next update ;).

[ksec]

My sense from various stakeholders is that the standard will be done this year. I don't think they'd hold it another year to get a few more percent savings.

Among other things, the market needs seem clearer than ever, with VVC licensing not having coalesced into something straightforward yet.

Then it is bad. And AOM as usual. I guess I am still too naive and too optimistic even at my age.

If they want it done this year they should at least release a beta version or 0.8 draft version. I think many have said this but I guess it is worth repeating. AOM is anything but Open.

[Z2697]

AOM is anything but Open.

What makes you think like this?

[GeoffreyA]

What makes you think like this?

I'm not sure, but doesn't Google have mostly tight fingers over contributions to be codebase?

[modus-ms325c]

I'm not sure, but doesn't Google have mostly tight fingers over contributions to be codebase?
ironically enough (and technically keeping with the "tight fingers" subject), this is kinda how cuavas handles outsiders' contributions to the MAME codebase as of late.
just ask the "bannister forums" folks.

[Tommy Carrot]

Working AV2 windows build (MSVC):
https://www.mediafire.com/file/3ytz9ql4zwlfr4m/av2_250425.7z/file

If someone wants to try encoding with it, i'd recommend to set --enable-uneven-4way-partitions to 0. I had crashes with it on certain resolutions, unless i disable it.

[GeoffreyA]

Working AV2 windows build (MSVC):
https://www.mediafire.com/file/3ytz9ql4zwlfr4m/av2_250425.7z/file

If someone wants to try encoding with it, i'd recommend to set --enable-uneven-4way-partitions to 0. I had crashes with it on certain resolutions, unless i disable it.

It took me one hour and 15 minutes to encode three seconds, so I gave up hope :)

[Z2697]

I think they (AOM and its predecessor) have changed the direction of the official library: at least from libvpx to libaom, they are both reference implementation and "production ready", at least they tried to. Now the AVM is not production ready at all, just like reference implementations in the MPEG side.

[GeoffreyA]

I think they (AOM and its predecessor) have changed the direction of the official library: at least from libvpx to libaom, they are both reference implementation and "production ready", at least they tried to. Now the AVM is not production ready at all, just like reference implementations in the MPEG side.

Yes. The encoder seems to be single threaded, but I didn't check through the whole process.

[BlueSwordM]

The main trick with CWG, aka proposals, is that they have to be rock solid to find themselves into production.

Their definition of rock-solid depends on a lot of members: hardware teams, software teams and worst of all, PSNR/SSIM scores.

Some of us have had to do a lot of work convincing peeps that a feature that increases perceptual quality and SSIMU2/butteraugli-jxl/XPSNR scores at the cost of non perceptual blur metrics, PSNR/SSIM.

2 features were added with the behaviors cited above and yet, some people complained :P

They did manage to go through, but it was hard.

[Jamaika]

The main trick with CWG, aka proposals, is that they have to be rock solid to find themselves into production.
Their definition of rock-solid depends on a lot of members: hardware teams, software teams and worst of all, PSNR/SSIM scores.
Some of us have had to do a lot of work convincing peeps that a feature that increases perceptual quality and SSIMU2/butteraugli-jxl/XPSNR scores at the cost of non perceptual blur metrics, PSNR/SSIM.
2 features were added with the behaviors cited above and yet, some people complained :P
They did manage to go through, but it was hard.

I created something but not everything works for me under gcc.
I deleted CONFIG_ENABLE_IBC_NAT, CONFIG_IBP_WEIGHT.
Frames-I for AV2 work for me under libwebp2. I'll add plus for AV2 for large size 5212x3468 support.
av2enc_avx.exe -q 100 -444 -size 280x420 -effort 7 -threads 4 -pass 1 -tune ssim image_jpeg.wp2 -d output.av2
tune butteraugli workes with latest jpegxl. Delete default BT709:
/*if (img->mc != 0 && img->mc != AOM_CICP_MC_BT_709 &&
img->mc != AOM_CICP_MC_BT_601 && img->mc != AOM_CICP_MC_BT_470_B_G) {
ERROR(
"Only BT.709 and BT.601 matrix coefficients supported in "
"tune=butteraugli mode. Identity matrix is treated as BT.601.");
}*/

av1enc_avx.exe -q 100 -444 -size 280x420 -effort 7 -threads 4 -tune butteraugli image_jpeg.wp2 -d output.av1

I wasn't able to create a movie.
https://gitlab.com/AOMediaCodec/avm/-/commit/0ee2f9e68420aef0d69aa91cd975decc71064bcd
ffmpeg_avx2.exe -i "video.mp4" -f rawvideo -vf scale=1920:1080: in_range=limited: out_range=limited -frames:v 500 -pix_fmt yuv420p - | aomenc.exe --verbose --limit=500 --input-bit-depth=8 --i420 --width=1920 --height=1080 --ivf --good --passes=1 --pass=1 --bit-depth=8 --fps=30000/1001 --threads=20 --cpu-used=9 --end-usage=cq --qp=0 -o avm_yuv420p10le.ivf -

Assertion failed: is_inter_compound_mode(mode), file c:\gcc1150\x86_64-w64-mingw32\include\av1\common\blockd.h, line 178

https://www.sendspace.com/file/gh11if

[benwaggoner]

The main trick with CWG, aka proposals, is that they have to be rock solid to find themselves into production.

Their definition of rock-solid depends on a lot of members: hardware teams, software teams and worst of all, PSNR/SSIM scores.

Some of us have had to do a lot of work convincing peeps that a feature that increases perceptual quality and SSIMU2/butteraugli-jxl/XPSNR scores at the cost of non perceptual blur metrics, PSNR/SSIM.

2 features were added with the behaviors cited above and yet, some people complained :P

They did manage to go through, but it was hard.
Sir, you did the work of Heroes!


People treat PSNR like some sort of fundamental definition of distortion rather than the decades-old "eh, easy to calculate and better than the other obvious alternatives" metric it truly is. And the square root of squares gets you a free absolute value conversion.

Lots of folks with a computer science background but without a neuropsychology or vision science background can assume that PSNR is what's before psychovisual optimization. But it's ALL psychovisual optimizations. It's just some of them are so old we don't really think about how they came to be.

sRGB/Rec. 709 is a perceptual optimization that makes 8-bit encoding work well within the range of what good CRTs used to be capable of reproducing.

Gamma is a psychovisaul optimization based on a first-order approximation of the human visual and CRT EOTF, easily implemented in simple analog circuitry. PQ is the second order approximation (but assumes a specific fixed ambient light). The non-psychovisual implementation would be linear light values.

Quant/Lambda tables are psychovisual optimizations based on our more accurate perception of horizontal and vertical details versus diagonal. And historically the PAL and NTSC versions embedded the different default sample aspect ratios (720x480 versus 720x576). The non-psychovisual implementation would be a flat quantizer.

Frequency transforms like DCT and iDCT themselves are perceptual optimizations about how our eyes and brains detect edges more than absolute brightness and color spatially. The non-psychovisual implementation would probably be pixel-based. And square pixels themselves are more of a computational friendly approach. Why not triangles or hexagons?

[paul97]

Have you got an AVM build for sse4 Windows 10 65 bit as they fixed msvc build yesterday? Thank You. Do it needs to compile wsl or bash.

[Jamaika]

https://www.sendspace.com/file/azxpwg