AOmedia next (AV2) codec

[hajj_3]

The Alliance for Open Media just announced AV2: https://aomedia.org/press%20releases/Alliance-for-Open-Media-Releases-AV2-Codec/

Surprisingly they didn't include any benchmarks in their blog post.

[rwill]

OK... so it's not like they had no other choice, they essentially did decoding design twice just to save some ASIC chips on ironically, high-end GPUs...
Weird choice.

Perhaps they need the die area to put in extra GPU cores?

Everyone is guessing here, and so am I, but what I could gather is that the Radeon HD 2900 has a R600 chip in 80nm process, the Radeon HD 2600 has a RV630 chip in 65nm process and the Radeon HD 2400 has a RV610 chip in 65nm process, of which the first available silicon revision had their UVD in a broken state (also called HD 2350).

So, different chips in different states of development and verification launched at around the same time I guess, as was tradition.

[rwill]

FPGAs...

I don't know much about FPGAs, except that FPGA stands for Field Programmable Gate Array, where "Field" refers to the environment they operate in. As in, they can be (re-)programmed in the field.

[kurkosdr]

I don't know much about FPGAs, except that FPGA stands for Field Programmable Gate Array, where "Field" refers to the environment they operate in. As in, they can be (re-)programmed in the field.
Being field programmable and being field re-programmable are not the same. In the same way that a CD-R is recordable but not re-recordable.

[Z2697]

Who would have thought, that the reference encoder should be deterministic? :rolleyes:


e9c4ad5cc80867d5c54dcd5367787975d1e70148 /home/bruh/ml.ivf
0784e463b95eb9d8c060e2f389142c343d5755ea /home/bruh/ml,.ivf

[birdie]

Who would have thought, that the reference encoder should be deterministic? :rolleyes:

This mustn't be happening. Are they using AI or what?

[Z2697]

Probably not a AI thing.
Maybe it has two states, sometimes the "baseline_gf_interval" starts as 10, sometimes it's 8.
Is there more? IDK, I haven't run it a lot of times.

[Jamaika]

A few progress summaries of AVM AVX2 decoding of photos and videos.
AV2 photos can be created in my ffmpeg and libwebp2 8bit. Here are the results.
Surprisingly, the latest dav2d decoder decoded 8bit AV2. However, I don't know how to decode a 12bit photo, even with a 16bit decoder.
4K photos take me a good 5 minutes to create.
For those curious, I've added my files in gcc. I know VS2026 is more popular.
I don't know why cpu-used has to be zero or one for the avm encoder to work.

https://github.com/AOMediaCodec/avm/commit/457cd58681a747465661baccb1f32095bc5b7774
https://www.sendspace.com/file/gln7fc

https://code.videolan.org/videolan/dav2d/-/commit/9f497073e1cce56830e475ebfc110be2874d7e72
https://www.sendspace.com/file/joqcx9

https://chromium.googlesource.com/codecs/libwebp2/+/d5920d8fe6abb5b3ba9b9bef3924f18433dd6d37
https://www.sendspace.com/file/5opq0e

ffmpeg_vulkan_avx2.exe -loglevel warning -y -i "image_21447_24bit.png" -y -c:v libavm-av2 -an -qp 52 -vf "format=yuv420p" -frames:v 1 -cpu-used 8 -still-picture 1 -tune ssim output_avm_08bit.av2
ffmpeg_vulkan_avx2.exe -loglevel warning -y -i "image_21447_24bit.png" -y -c:v libavm-av2 -an -qp 42 -vf "format=yuv444p12le" -frames:v 1 -cpu-used 8 -still-picture 1 -tune ssim output_avm_12bit.av2
cwp2_avx2.exe "image_21447_24bit.png" -info -q 100 -8bits -mt -effort 7 -uv_mode 2 -csp 0 -o image_21447_24bit.wp2
av2enc_avx2.exe -q 100 -444 -effort 7 -threads 4 -pass 1 -tune ssim image_21447_24bit.wp2 -d image_21447_24bit.av2
dav2d_xxbit_avx2.exe -i input_xxbit.av2 -o output_xxbit.y4m
What can be said about videos? It's already a bit slow. Very slow encoder. Pervasive blur. The lower the framerate, the more blur. IVF files don't open in DAV2D. I tested it for HD at 100 frames at 60 fps. I think we'll have to wait another six months.

AVM IVF sample video:
avmenc_avx2.exe --verbose --threads=16 --cpu-used=1 --end-usage=cbr --target-bitrate=3000 -o __AV2.ivf __REF.y4m
https://www.sendspace.com/file/u2br2t

[GeoffreyA]

Best to wait for SVT-AV2 and ignore the reference encoder.

[rwill]

Best to wait for SVT-AV2 and ignore the reference encoder.


Any development news?

And where is x266? xD

[GeoffreyA]

Any development news?

And where is x266? xD

Not yet. I've searched but nothing. We'll have to e-mail one of the SVT-AV1 maintainers to get insight.

As for x266, if you look up at the stars on a clear night, you might just spot it, twinkling fitfully as its SAO filter adapts to the changing scene.

[Z2697]

Any development news?

And where is x266? xD

x266 is a bigger joke.
Need y266 :)

[hajj_3]

And where is x266? xD

good joke :P

[kurkosdr]

x266 is a bigger joke.
Need y266 :)
Or, you know, now that the world has alternatives (see thread subject), Fraunhofer and friends can stop expecting free lunches from open-source devs (commercial or hobbyist) and further improve VVenC.

[rwill]

Or, you know, now that the world has alternatives (see thread subject), Fraunhofer and friends can stop expecting free lunches from open-source devs (commercial or hobbyist) and further improve VVenC.

Interesting take, given that SVT-AV1's recent psychovisual improvements are just ports of x264's psy-RD from 2008. And the people that did the port had to be quite 'active' to get the changes upstream. What got it relabeled to, --ac-bias ?

I personally would have given credit where credit is due, but then again I am just some old guy and not one of these modern developers with a blog, discord, patreon, etc.

[kurkosdr]

Interesting take, given that SVT-AV1's recent psychovisual improvements are just ports of x264's psy-RD from 2008. And the people that did the port had to be quite 'active' to get the changes upstream. What got it relabeled to, --ac-bias
Well, here is your answer: The devs behind x264 would rather contribute to SVT-AV1 (even if it means having to be quite 'active' to get their changes upstream) than contribute to VVenC just to help Fraunhofer and friends make more money.

When it comes to x264 and x265, hobbyist devs contributed (and contribute) because they want to get better video on their H.264 or H.265 hardware players and TVs (since H.264 is the lowest common denominator for HD and H.265 for HDR and 4K), but since every device that supports VVC also supports AV1 (with the inverse not always being true), why not contribute to the format that is not only more popular but also more closely aligns with the values of open-source?

Which leaves commercial open-source developers such as Multicoreware: Fraunhofer and friends are welcome to bet their fortunes on commercial open-source developers such as Multicoreware delivering a capable encoder for a semi-dead standard like VVC. Or they can further improve VVenC and stop expecting free lunches. Their choice.

PS: At the expense of being on-topic, it seems like AV2 has scooped ECM/H.267, so again, it looks like AOM standard will be the standard that will be both the more popular one of its generation and royalty-free. I am eager to be proven wrong in industries such as broadcast and see ECM be so good that broadcast will break compatibility with HEVC receivers (since broadcast needs all the bitrate savings it can get and doesn't particularly care about royalty-free-ness), but I highly doubt it.

[birdie]

The first two pieces of hardware IP (https://www.allegrodvt.com/news/pulsar-decoder-ip-support-av2-video-codec/) (1) for AV2 (https://chipsnmedia.com/en/company/news.php?boardid=news_en&mode=view&idx=50&sk=&sw=&offset=&category=) (2) decoding have been released.

What is decoding hardware Intellectual Property? It refers to a pre-designed, dedicated physical circuit (like a microchip block) used to process and decompress digital data. In computing, it typically means using specialized hardware to decode compressed video or network streams instead of forcing the main CPU to do it.

Essentially, it's a blueprint in silicon that other companies can integrate into their GPUs, APUs, SoCs, and so on, to enable you to enjoy energy-efficient hardware decoding. What does this mean in practical terms? In around 24–36 months, the first devices featuring HW AV2 decoding will be available. Could be less if the stars align ;-)

The products info pages:
* Pulsar D400 (https://www.allegrodvt.com/products-ip/pulsar-video-decoder-ip-d400-series/)
* WAVE63F1 (https://chipsnmedia.com/en/products/video-4.php)
* VC9800D (https://www.verisilicon.com/en/IPPortfolio/HantroVC9800D) (actual decoder that now supports AV2 but it's notable as well)

What's interesting is that:
World-class next-gen AV2 HW IP confirmed for future flagship devices of a major North American client
Future products are already in the pipeline. What might they be? No idea, but it could be either NVIDIA, AMD or Intel. Apple is much less likely. They normally drag their feet for years before embracing new codecs.

[Z2697]

So do the three giants design their own codec chip or use those SIP core(s)?

[Zebulon84]

… it could be either NVIDIA, AMD or Intel. Apple is much less likely … Shouldn't Qualcomm be on that list ? Their chip are in just a few computer, but on many smartphones, where "energy-efficient hardware decoding" is essential.

[Z2697]

Shouldn't Qualcomm be on that list ? Their chip are in just a few computer, but on many smartphones, where "energy-efficient hardware decoding" is essential.

Mediatek was earlier in the AV1 game, earliest in the smartphone beezness probably.

[birdie]

Shouldn't Qualcomm be on that list ? Their chip are in just a few computer, but on many smartphones, where "energy-efficient hardware decoding" is essential.

Qualcomm was reluctant to embrace AV1, that's why I "forgot" about them.

NVIDIA, Intel, and AMD all shipped consumer PC hardware AV1 decode in 2020, while Qualcomm did not add it to flagship Snapdragon phones until Snapdragon 8 Gen 2, announced November 15, 2022.

But this time around it very well could be Qualcomm. I'm 95% sure the next major Mediatek SoC architecture will support HW AV2 decoding and I don't think Qualcomm would want to be treated like a second class citizen in the game. The stakes are far too high nowadays.

And, it could be Google's Tensor as well since their RTL division is just too weak and they continue to churn subpar SoCs.

[rwill]

Well, here is your answer: The devs behind x264 would rather contribute to SVT-AV1 (even if it means having to be quite 'active' to get their changes upstream) than contribute to VVenC just to help Fraunhofer and friends make more money.

I don't think x264 devs contributed to SVT-AV1 ever. I think that certain SVT-AV1 splinter group devs took what they could use from x264 source code, obfuscated it in some of their forks, and submitted it upstream to SVT-AV1 mainline. This is all well legally and within x264's GPL license but the way I see AV1 advocates advertise the 'new' features - the ancestry of the inventions seem to have been forgotten along the way. I find this rather sad.