AOmedia next (AV2) codec

[GeoffreyA]

Windows binary:

https://www.mediafire.com/file/mgzzhxbtdmr3sjm/av2_260414.7z/file

Don't expect anything usable tho, the bitstream is not frozen yet, and this encoder is VERY slow right now, encoding a couple of frames takes hours.

Much appreciated!

I got the encoding and decoding working, but it is indeed very slow at present.

[Z2697]

Hopefully with them taken SVT-AV1 "in", we can see a practical encoder for AV2 in a shorter time.

[GeoffreyA]

Yes, one hopes it won't be too hard to extend the SVT implementation to AV2. Going back to the start of how it was with AV1 would be intolerable.

[excellentswordfight]

The Linux rescue vessel arrived to save me, but I declined, deciding to remain aboard my sinking ship, Valiant Windows X, till the oceans overrun her. I've taken refuge on the ESU deck :-)
OT, but as someone who is pretty much forced stick to windows on my dailydriver due to Microsoft 365 and Adobe suite, I highly recommend using WSL if your not already, i do pretty much all cli/terminal stuff (scripting/python styff, encoding, compiling etc) through that.

[GeoffreyA]

OT, but as someone who is pretty much forced stick to windows on my dailydriver due to Microsoft 365 and Adobe suite, I highly recommend using WSL if your not already, i do pretty much all cli/terminal stuff (scripting/python styff, encoding, compiling etc) through that.

I haven't tried WSL, but thanks for the tip. I doubt whether I'd ever migrate to Linux. Which is a problem, considering the recent decline of Windows.

[birdie]

Has anyone tested AV2 already? I understand the reference encoder is dead slow but it doesn't take more than a couple of seconds to see what it produces in the end.

I'm extremely interested in visually lossless encoding for 1080p and 4K sources, that's when the bitrate is not extremely high but not low either and the fine details are preserved as much as possible.

[GeoffreyA]

birdie,

Here's a test I did last night after getting the encoding-decoding pipeline working. Using a --cq-level of 20, a bitrate of 6,244 kbps resulted. In terms of speed, it encodes about a frame per minute on my 6-core 4600G. Originally, it was slower, but raising the --threads setting reduced time by a factor of 2 to 3. Still, utilisation is far from optimal.

https://workupload.com/file/zF54U3cWCdL

You can change the -frames:v setting to control how many frames to tackle; the reference clip has 96. Be careful with FFmpeg's -y setting, which overwrites files without asking. The binaries are from Tommy Carrot above. Thanks.


:: REF.mp4 -> REF.yuv -> AV2.ivf -> AV2.yuv -> AV2.mkv (FFV1 lossless)

ffmpeg -loglevel warning -y -i __REF.mp4 -frames:v 1 -c:v rawvideo __REF.yuv

avmenc --verbose --threads=16 --width=1920 --height=1080 --fps=24000/1001 --cpu-used=8 --end-usage=q --cq-level=20 -o __AV2.ivf __REF.yuv

avmdec --verbose --threads=16 --rawvideo --output=__AV2.yuv __AV2.ivf

ffmpeg -loglevel warning -y -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -r 23.976 -i __AV2.yuv -c:v ffv1 __AV2.mkv

[Kurt.noise]

dav2d is here

https://code.videolan.org/videolan/dav2d

[hajj_3]

dav2d is here

https://code.videolan.org/videolan/dav2d

Annoyingly they wrote it in C not Rust or even Zig. Those other languages would have reduced the chance of security holes. Rust was in its infancy when Dav1d was created so that was understandable but it is definitely mature now. I hope they have access to Anthropic Mythos AI to check for security holes.

[Z2697]

Annoyingly they wrote it in C not Rust or even Zig. Those other languages would have reduced the chance of security holes. Rust was in its infancy when Dav1d was created so that was understandable but it is definitely mature now. I hope they have access to Anthropic Mythos AI to check for security holes.

Well well, look how rust brought down cloudflare last year.
And any SIMD optimization will just go straight into the unsafe territory.

[GeoffreyA]

Rust vs. C/C++ reminds of Java vs. C++ in an earlier era, and how Java was specifically designed to target what James Gosling perceived as C++'s weaknesses. Well, I suppose C/C++ will always have a challenger in each age.

[Z2697]

Ahh the magic of LLVM, written in C++, initially written for C/C++. :)

[GeoffreyA]

A chicken-and-egg situation :)

[kurkosdr]

Rust vs. C/C++ reminds of Java vs. C++ in an earlier era, and how Java was specifically designed to target what James Gosling perceived as C++'s weaknesses. Well, I suppose C/C++ will always have a challenger in each age.
What killed Java was being tied to Sun's JVM, which meant a performance hit and having to install JavaSE (with the Windows installer trying to upsell you some crappy toolbar that Sun got bribed to include). Oh, and the UI not using the native OS theme, another "perk" of the JVM (back then, people cared about apps feeling native, now we've gotten so used to the GUI equivalent of brusselization (https://en.wikipedia.org/wiki/Brusselization) that we don't care).

Yes, I know you can technically compile Java into a native binary, but that's not what Sun encouraged you to do.

Another issue was Java outsourcing memory management entirely to the garbage collector, which sounded good back then, but it wasn't.

Rust has none of those issues.

That said, Java is big on business software, for Linux shops at least.

[Z2697]

What killed Java was being tied to Sun's JVM, which meant a performance hit and having to install JavaSE (with the Windows installer trying to upsell you some crappy toolbar that Sun got bribed to include). Oh, and the UI not using the native OS theme, another "perk" of the JVM (back then, people cared about apps feeling native, now we've gotten so used to the GUI equivalent of brusselization (https://en.wikipedia.org/wiki/Brusselization) that we don't care).

Yes, I know you can technically compile Java into a native binary, but that's not what Sun encouraged you to do.

Another issue was Java outsourcing memory management entirely to the garbage collector, which sounded good back then, but it wasn't.

Rust has none of those issues.

That said, Java is big on business software, for Linux shops at least.

And Android ;)

[GeoffreyA]

What killed Java was being tied to Sun's JVM, which meant a performance hit and having to install JavaSE (with the Windows installer trying to upsell you some crappy toolbar that Sun got bribed to include). Oh, and the UI not using the native OS theme, another "perk" of the JVM (back then, people cared about apps feeling native, now we've gotten so used to the GUI equivalent of brusselization (https://en.wikipedia.org/wiki/Brusselization) that we don't care).

Yes, I know you can technically compile Java into a native binary, but that's not what Sun encouraged you to do.

Another issue was Java outsourcing memory management entirely to the garbage collector, which sounded good back then, but it wasn't.

Rust has none of those issues.

That said, Java is big on business software, for Linux shops at least.

Agreed. Java ran terribly slow because of the JVM, and the GUIs were ugly. Even uglier, in my opinion, was the language's verbose syntax and perverse attempt to avoid C++'s "weaknesses."

Rust is different, running as fast as C/C++ while enforcing memory safety. But I see a similarity in the attempt to replace the old languages and a sentiment of "Rust is better."

[hajj_3]

AVM 15.0.0 released: https://gitlab.com/AOMediaCodec/avm/-/tags/research-v15.0.0

[Z2697]

Has the specification changed in the recent versions?
Or is it just bugfix for reference implementation?

[hajj_3]

Has the specification changed in the recent versions?
Or is it just bugfix for reference implementation?

the draft spec still uses v13. Once AV2 has been ratified they will update the spec which will presumably be v15, v15.1 or v16.

[rwill]

A month ago:

AVM v14.0 is now complete: https://gitlab.com/AOMediaCodec/avm/-/tags/research-v14.0.0

This will likely be the official version of AV2 v1.0. We could see an official announcement that AV2 has been ratified within the next few days.

Now:

the draft spec still uses v13. Once AV2 has been ratified they will update the spec which will presumably be v15, v15.1 or v16.

I want the VCEG back...

[kurkosdr]

I want the VCEG back...

Yeah, because ISO standards never have "corrigenda" that fix mistakes (and even remove whole-ass profiles):
https://en.wikipedia.org/wiki/Advanced_Video_Coding#Versions

If anything, ISO standards are absolutely horrible in this regard, they'll string everyone along by adding stuf such as stereoscopic 3D and 360-degree video as "versions" to the standard (instead of making those a new -potentially backwards-compatible- standard), which conveniently adds more patents to the patent pool to "justify" keeping the royalties as high as when the standard was new.

[ksec]

Yeah, because ISO standards never have "corrigenda" that fix mistakes (and even remove whole-ass profiles):
https://en.wikipedia.org/wiki/Advanced_Video_Coding#Versions

If anything, ISO standards are absolutely horrible in this regard, they'll string everyone along by adding stuf such as stereoscopic 3D and 360-degree video as "versions" to the standard (instead of making those a new -potentially backwards-compatible- standard), which conveniently adds more patents to the patent pool to "justify" keeping the royalties as high as when the standard was new.

You are completely misrepresenting what rwill is saying. Not only are the profile and version two different thing, he is saying the rollout between the two are different as well as the marketing. But you manage to draw in the patent argument into it again.

Needless to say the promise of AV2 was end of last year, we are fast coming into 2H of this year already.

Again I don't want them to rush it and much rather take their time, but they shouldn't have made that promise.

[kurkosdr]

You are completely misrepresenting what rwill is saying. Not only are the profile and version two different thing, he is saying the rollout between the two are different as well as the marketing. But you manage to draw in the patent argument into it again.

Needless to say the promise of AV2 was end of last year, we are fast coming into 2H of this year already.

Again I don't want them to rush it and much rather take their time, but they shouldn't have made that promise.

Huh? The VCEG doesn't have a predictable release pattern for new standards either, and the fact that their spec needs to be amended with several "corrigenda" to fix various mistakes, clarify information, and add and remove profiles, means that the "Version 1" of their spec is a draft really, the fact that they slap "Version 1" on the standard for formal reasons is irrelevant.

[Z2697]

Huh? The VCEG doesn't have a predictable release pattern for new standards either, and the fact that their spec needs to be amended with several "corrigenda" to fix various mistakes, clarify information, and add and remove profiles, means that the "Version 1" of their spec is a draft really, the fact that they slap "Version 1" on the standard for formal reasons is irrelevant.

OK, so we shouldn't have a release H.264 specification until at least August 2024.

[kurkosdr]

OK, so we shouldn't have a release H.264 specification until at least August 2024.
H.264 stablized with Version 3 (and even that needed two corrigenda as Version 4 and 5 that are essential for a correct implementation), so this is your clue that VCEG wasn't ready when they released "Version 1", Version 1 and 2 were actually drafts.

But anyway, my point is that anyone who praises the VCEG for releasing a complete "Version 1" spec that isn't an officialized draft is wearing rose-tinted glasses. Same for assuming that VCEG ever stuck to any sort of release timeline.

[Z2697]

H.264 stablized with Version 3 (and even that needed two corrigenda as Version 4 and 5 that are essential for a correct implementation), so this is your clue that VCEG wasn't ready when they released "Version 1", Version 1 and 2 were actually drafts.

But anyway, my point is that anyone who praises the VCEG for releasing a complete "Version 1" spec that isn't an officialized draft is wearing rose-tinted glasses. Same for assuming that VCEG ever stuck to any sort of release timeline.

Oh Version 7 would be better, because it added back the slightly tweaked version of High 4:4:4 (Predictive) Profile that got removed in Version 5!
Who's to say MVC isn't important? Version 11 then!
Version 14 with it's higher level is critical for today's need!
Maybe Version 20 with BT.2020 support is even better?

[rwill]

I always thought "patent troll" referred to something else but here we are...

[kurkosdr]

Oh Version 7 would be better, because it added back the slightly tweaked version of High 4:4:4 (Predictive) Profile that got removed in Version 5!
Who's to say MVC isn't important? Version 11 then!
Version 14 with it's higher level is critical for today's need!
Maybe Version 20 with BT.2020 support is even better?
Everyone has settled on Version 3 + relevant corrigenda (with the notable exception of Blu-ray 3D), nobody is broadcasting BT.2020 or HDR10 in H.264. But all those corrigenda and the addition and removal of profiles, even for the High Profile(s) that everyone is using, is what you need to know to understand that Version 1 was an officialized draft.

[Z2697]

Everyone has settled on Version 3 + relevant corrigenda (with the notable exception of Blu-ray 3D), nobody is broadcasting BT.2020 or HDR10 in H.264. But all those corrigenda and the addition and removal of profiles, even for the High Profile(s) that everyone is using, is what you need to know to understand that Version 1 was an officialized draft.

I've seen Baseline and Main profile in the wild (and not that rare, really). Produced to this day.
Which is pretty much Version 1.

And like I said, the removed profile is High 4:4:4 Profile and it's later added back with a little tweaking as High 4:4:4 Predictive Profile, basically with only one more possible bitdepth.
It's important, it enables lossless mode! Version 7 FTW!

Where did you see a major change in the High profile, the most popular profile?

This is too off topic so will be my last reply regarding it.

[kurkosdr]

I've seen Baseline and Main profile in the wild (and not that rare, really). Produced to this day.
Which is pretty much Version 1.

And like I said, the removed profile is High 4:4:4 Profile and it's later added back with a little tweaking as High 4:4:4 Predictive Profile, basically with only one more possible bitdepth.
It's important, it enables lossless mode! Version 7 FTW!

Where did you see a major change in the High profile, the most popular profile?

This is too off topic so will be my last reply regarding it.
I am not talking about backwards-compatibility changes necessarily (though this did happen with the removal of the High 4:4:4 Profile), but the fact the spec needs to be clarified and mistakes fixed. So, let's stop pretending that only AOM releases specs early and with mistakes and omissions, VCEG does too.

[Z2697]

I am not talking about backwards-compatibility changes necessarily (though this did happen with the removal of the High 4:4:4 Profile), but the fact the spec needs to be clarified and mistakes fixed. So, let's stop pretending that only AOM releases specs early and with mistakes and omissions, VCEG does too.

No, quite the opposite.
How many versions of AV1 specification we have seen?
2.
https://github.com/AOMediaCodec/av1-spec/tags

The problem is there's no release, not too many/early of them.

(talking about AOM thing again so I count this as back to topic :rolleyes:)

[rwill]

I don't know how much AV2 changed from AV1 but if its just a couple more tools they could have named it AV1 High Profile instead of naming it AV2...

[kurkosdr]

No, quite the opposite.
How many versions of AV1 specification we have seen?
2.
https://github.com/AOMediaCodec/av1-spec/tags

The problem is there's no release, not too many/early of them.

(talking about AOM thing again so I count this as back to topic :rolleyes:)

Again, the VCEG doesn't commit to a release timeline either, and their releases don't follow a predictable pattern either. You are judging AOM based on a non-existent standard.

[kurkosdr]

I don't know how much AV2 changed from AV1 but if its just a couple more tools they could have named it AV1 High Profile instead of naming it AV2...
Why? So they can confuse the heck out of normal people like the ISO people did with their 2 different standards named MPEG-4 (Visual and AVC) and their multiple non-backwards-compatible profiles for MPEG-4 Visual (SP and ASP) and MPEG-4 AVC (Baseline, Main, High Profile)?

If you've tried getting MPEG-4 AVC videos to work on portable devices on the 2000s (and eventually quit and use good ol' lowest-common-denominator MPEG-4 Visual SP in the form of the Divx Home Theater format, which at least is a consistent standard as long as you don't use anamorphic), you know what I mean. And I don't even blame the manufacturers. Good luck documenting this smorgasbord of confusingly-named standards and profiles in a manual that normal people can read.

When the WebM/AOM guys introduce a new (non-backwards-compatible) profile, at least they have the courtesy to bump the format number, if by a fractional amount (for example: VP9 -> VP9.2), they don't invent confusing standard names and profile names with the words Advanced, High, Simple, Baseline, Main etc randomly thrown in. Just try explaining to a normal person the difference between MPEG-4 Visual Advanced Simple Profile and MPEG-4 Advanced Video Coding Baseline Profile

[kurkosdr]

Anyway, back on topic (again), give AOM time to get AV2 in a release-worthy shape, it's not like VCEG has any predictable release pattern or commit to any release dates either.

[Z2697]

VP9.2 is VP9 Profile 2...

What are we doing?

I guess we have to accept that they (webm/aom) don't usually have a release.
https://www.webmproject.org/vp9/
VP9 is still draft v0.7 :rolleyes:
Let's call it VP8.7

And I don't think we have a shared understanding of backwards compatibility.
I'll share mine, it means what used to work, continue to work, not new things just magically work like old ones. ("new things" as there's actual new feature)
(of course it's possible to design something that partially work like old, and that can be called backwords compatibility as well)

[GeoffreyA]

And I don't think we have a shared understanding of backwards compatibility.
I'll share mine, it means what used to work, continue to work, not new things just magically work like old ones. ("new things" as there's actual new feature)
(of course it's possible to design something that partially work like old, and that can be called backwords compatibility as well)

My view of compatibility is that there are different types. On one hand, we have what used to work and still does. For example, decades-old Windows binaries continuing to work on NT. Here, it is the OS that has backwards compatibility. (Or perhaps Windows binaries have forward compatibility :-)

On the other hand, we have things like codecs, and backwards compatibility could be viewed as working on old decoders and devices. If full-spec AV2 played on AV1 decoders, it would be backwards compatible. Formats decomposing to a legacy core would be partly backwards compatible, with graceful loss of the new features.

[kurkosdr]

On the other hand, we have things like codecs, and backwards compatibility could be viewed as working on old decoders and devices. If full-spec AV2 played on AV1 decoders, it would be backwards compatible. Formats decomposing to a legacy core would be partly backwards compatible, with graceful loss of the new features.
Yes, that's the kind of backwards compatibility I meant. Blu-ray 3D is an example. But when it comes to those different MPEG-4 AVC profiles, the new profiles aren't backwards compatible with the old profiles, but good luck figuring that out from names like "Baseline", "Main", and "High" unless you memorize the profile names. I mean, does "Main" mean "complete" and hence is a superset of High Profile or is it a subset of "High"?

At least WebM and AOM follow a rational naming scheme VP9 Profile 1, VP9 Profile 2.

And I don't think we have a shared understanding of backwards compatibility.
I'll share mine, it means what used to work, continue to work, not new things just magically work like old ones. ("new things" as there's actual new feature)
That's one definition of backwards compatibility, and "MPEG-4" fails even at that: For example, there is no guarantee a MPEG-4 AVC player will play MPEG-4 Visual streams (Blu-ray players can play MPEG-4 AVC but aren't required to play MPEG-4 Visual, and I am not sure Blu-ray players are required to play MPEG-4 AVC Baseline and Main either, since High Profile is not an exact superset of Baseline and Main).

[Z2697]

It looks like for WebM/AOM things work in backwards, you have numbered profiles in VP8.7, but then in AV1 you have Main, High, and Professional.

MPEG-4 is a big thing, they are different parts. There's no such thing as "backward compatibility" between them.
If you want "inter-compability" maybe you should also design a decoder that decodes the AAC (MPEG-4 Audio) "layer" (imagine there is such layer) of MPEG-4 Visual?
Important note: It's not the same as being able to decode both formats, it's decoding AAC "out of" MPEG-4 Visual.
And even AAC is just a part of MPEG-4 Part 3.

What are we doing really?

The name doesn't matter, VP9 is a draft forever and people seem to be happy about it.
AV2 is a draft but probably there will be a finalized version, still doesn't matter.
What matters is the joke about "end of 2025".

[Kurt.noise]

Finally, the first release of AV2 is officially planned next week (https://github.com/AOMediaCodec/avm/commit/87aa8dc44e8a40f992604111a5ce83cf413ec9b7)...

btw, a dav2d x64 Windows build for those whom interested : https://www.mediafire.com/file/m0hgq5tdabsggf8/dav2d-0.0.1-142_x64.7z/file

[Jamaika]

Does anyone know where to download free av2 samples to test dav2d in ffmpeg?

[GeoffreyA]

Does anyone know where to download free av2 samples to test dav2d in ffmpeg?

Don't know, but you can use this script to encode short clips:

https://forum.doom9.org/showthread.php?p=2030467#post2030467

[Jamaika]

Thanks for the codecs and samples

[ksec]

H.264 stablized with Version 3 (and even that needed two corrigenda as Version 4 and 5 that are essential for a correct implementation), so this is your clue that VCEG wasn't ready when they released "Version 1", Version 1 and 2 were actually drafts.

But anyway, my point is that anyone who praises the VCEG for releasing a complete "Version 1" spec that isn't an officialized draft is wearing rose-tinted glasses. Same for assuming that VCEG ever stuck to any sort of release timeline.

The point is VCEG never promised any time line in the first place. It is ready when it is ready. But AOM has repeatedly stated unrealistic timeline and roadmap in nearly all instances.

Do we need to go back we were suppose to have AV2 in 2020 and AV3 in 2022?

Overpromise and underdeliver is what they have done not just AOM but since the days on On2. Plenty of people wished they have learned their lesson with AV1, but all signs points to not so much so.

Anyway back to the topic. Av2 releasing soon and most importantly, dav2d is in really good shape from the get go!. Looking forward to testing results and hopefully some AV2 image codec comparison against JPEG XL.

[hajj_3]

But AOM has repeatedly stated unrealistic timeline and roadmap in nearly all instances.

Do we need to go back we were suppose to have AV2 in 2020 and AV3 in 2022?

Overpromise and underdeliver is what they have done not just AOM but since the days on On2.

Not really. They originally said that but after speaking to manufacturers they realised that manufacturers don't want to have to use more silicon every 3yrs for a new codec. They other reason is that a know codec every 3yrs would see little adoption. Having a 8yr gap between AV1 and AV2 was a good idea as AV1 hardware decoders are now in all computers and all high end smartphones. Low end phones are supposed to get AV1 support this year or next year.

[GeoffreyA]

Low end phones are supposed to get AV1 support this year or next year.

On newer versions of Android, dav1d is being used, so even in the absence of hardware decoding, AV1 should play quite well everywhere.

[benwaggoner]

On newer versions of Android, dav1d is being used, so even in the absence of hardware decoding, AV1 should play quite well everywhere.
...absent premium content with a hardware DRM requirement...

[Z2697]

...absent premium content with a hardware DRM requirement...

How is DRM tied in with decoder?

[rwill]

How is DRM tied in with decoder?

The pictures never leave the GPU I guess. Where 'GPU' is some protected hardware safe space that is not accessible even by villain hackery.

[kurkosdr]

On newer versions of Android, dav1d is being used, so even in the absence of hardware decoding, AV1 should play quite well everywhere.
Please tell me that Google hasn't started doing on the YouTube app and YouTube mobile that annoying thing they are doing on YouTube desktop, where they'll serve you AV1 even if your computer only has H.264 hardware acceleration? I had to install h264ify to prevent some old laptops from doing a vacuum cleaner impression while watching YouTube. On mobile it will drain the battery like no tomorrow.