AV1 hardware decoder implementations [Archive]

benwaggoner

30th October 2020, 21:17

As we're on the verge of lots of devices with hardware AV1 decode, I think it's time for a sticky about this in particular.

As Chrome and Firefox explicitly block HEVC decode even if the OS supports it, PC browsers are where AV1 is going to have the biggest improvement delta. Most other platforms (including Edge and Safari) have HEVC today, where the quality @ perf of HEVC and AV1 are pretty equivalent given relative maturity of encoders. While SW encoders do work, performance is going to be a limitation for 4K and 10-bit (which HDR requires). And the lack of HW DRM provides lots of limitations on premium content availability.

The Intel 11th gen Ice Lake CPUs are the first shipping PC implementation. Here's the best analysis of AV1 decode with Ice Lake I've seen: https://www.pcworld.com/article/3576298/tested-av1-performance-in-11th-gen-tiger-lake-vs-10th-gen-ice-lake-and-comet-lake-ryzen-4000.html.

I've not seen a test of DRM on Ice Lake, but I imagine it should work, and if it doesn't, a software bug fix is probably sufficient.

tl;dr works great, and as expected.

benwaggoner

25th November 2020, 00:02

Has anyone tested to see if any of the below support AV1 decode?
PlayStation 5
Xbox Series X|S
Apple M1 processor in the new Mac mini, MacBook Air, and MacBook Pro 13"

hajj_3

25th November 2020, 10:01

some info about apple m1 possibly have a 4k av1 hardware decoder or hybrid decoder: https://forums.macrumors.com/threads/apple-m1-vp9-av1-decoding.2269938/

Yups

28th November 2020, 00:02

The Intel 11th gen Ice Lake CPUs are the first shipping PC implementation.

Tigerlake not Icelake, big difference.

I have tested a few AV1 youtube videos in Chrome with my Tigerlake laptop and it works really good so far.

https://abload.de/img/av1zak7b.jpg

birdie

28th November 2020, 09:53

https://en.wikipedia.org/wiki/AV1#Hardware

benwaggoner

3rd December 2020, 01:33

Bad news: it appears that Qualcomm isn't doing HW AV1 in its 2021 chipsets.

https://www.anandtech.com/show/16271/qualcomm-snapdragon-888-deep-dive/3

AV1 decoding didn’t make the cut this year unfortunately, which means wide-spread adoption in mobile for the codec will be delayed for another year.

rubait

18th December 2020, 06:19

Has anyone tried benchmarking the AV1 decoder performance on Tigerlake using DXVA Checker?

rubait

18th December 2020, 06:20

Tigerlake not Icelake, big difference.

I have tested a few AV1 youtube videos in Chrome with my Tigerlake laptop and it works really good so far.

https://abload.de/img/av1zak7b.jpg

Did you try playing back on DXVA checker? It can't seem to recognize the clip.

Yups

19th December 2020, 20:40

Did you try playing back on DXVA checker? It can't seem to recognize the clip.

When I use D3D11 playback in DXVA Checker there are image artifacts on both LAV Decoder as well as Microsoft AV1 extension. DXVA2 runs on both but only with LAV Decoder it appears to use the GPU.

Using MPC Home Cinema playback runs fine using D3D11 AV1 in LAV.

Pepsi_Doggo

20th January 2021, 02:25

RTX 3060Ti decoding 4K HDR(10-bit)@60fps above just 2ms render time:

https://files.catbox.moe/i27eit.jpg

hajj_3

20th January 2021, 03:22

Android tv boxes featuring Amlogic S905X4 chips are now available for as little as £30/$40 delivered from china. These can hardware decode AV1.

quietvoid

20th January 2021, 04:06

Android tv boxes featuring Amlogic S905X4 chips are now available for as little as £30/$40. These can hardware decode AV1.

Personally waiting on certified devices for streaming with hwdec.. hopefully this year.

benwaggoner

20th January 2021, 18:05

hajj_3

20th January 2021, 19:14

Has anyone tried the Xbox Series X|S hybrid decoder yet?

Are you sure it can hybrid decode AV1?

benwaggoner

21st January 2021, 01:17

Are you sure it can hybrid decode AV1?
I am asking because I am not sure. But this sure sounded promising: https://forum.doom9.org/showthread.php?p=1929168&highlight=xbox#post1929168

soresu

4th February 2021, 09:02

Has anyone tried the Xbox Series X|S hybrid decoder yet? It's obviously complex to get working, but I can imagine the next gen game consoles are going to be a big share of the AV1 capable living room devices in 2021.

If you look at the system requirements section of the PDF it doesn't actually mention either XSS or XSX.

"Hardware: Xbox One, Xbox One S, or Xbox One X"

The googlesource repo is also labelled " GPU-accelerated AV1 decoder for the Xbox One".

So if someone can integrate this into Kodi or Plex for UWP, and Jaguar is enough to cover entropy decoding there are actually millions of devices in living rooms already.

I wouldn't expect MS to be any help on that score for obvious profit based reasons of pushing people to XSS and XSX.

Hopefully if it does get ported to the new console gen they use as much dav1d AVX2 asm as possible for the entropy decoding stage, I've no idea how much SSSE3 asm is in the current implementation to use on the Jaguar cores.

benwaggoner

4th February 2021, 20:28

If you look at the system requirements section of the PDF it doesn't actually mention either XSS or XSX.

"Hardware: Xbox One, Xbox One S, or Xbox One X"

The googlesource repo is also labelled " GPU-accelerated AV1 decoder for the Xbox One".

So if someone can integrate this into Kodi or Plex for UWP, and Jaguar is enough to cover entropy decoding there are actually millions of devices in living rooms already.
Entropy decoding isn't that complex, and well suited to fast single cores, and IIRC AV1's mandatory slices for high resolutions allow for parallelized entropy decode. This is a huge improvement from VP8-9, which were very hard to parallelize in decode.

I wouldn't expect MS to be any help on that score for obvious profit based reasons of pushing people to XSS and XSX.
I'm very confident that Microsoft is not worried about lack of demand for the new consoles! And they are putting a big emphasis on a uniform ecosystem. If they see material value in AV1, I'd expect them try to backport it. The real question is if they see value in AV1 over HEVC, which is supported in HW since Xbox One S.

Hopefully if it does get ported to the new console gen they use as much dav1d AVX2 asm as possible for the entropy decoding stage, I've no idea how much SSSE3 asm is in the current implementation to use on the Jaguar cores.
Jaguar doesn't have AVX2, alas.

https://en.wikipedia.org/wiki/Jaguar_(microarchitecture):
SSE4.1, SSE4.2, AES, CLMUL, MOVBE, AVX, F16C, BMI1

So, that would be friction for backporting dav1d to PS4 or Xbox One. SSSE3 may be sufficient for 4Kp60, though, which is all the previous generation supports anyway. I'm sure an AVX implementation would still be better than SSSE3, and probably less incremental work to develop once there are already SSSE3 and AVX2 implementations to upgrade/downgrade from.

Xbox Series and PS5 have AVX2.

RanmaCanada

6th February 2021, 05:26

Android tv boxes featuring Amlogic S905X4 chips are now available for as little as £30/$40 delivered from china. These can hardware decode AV1.

So I wonder how much longer we have to wait for Amazon to release a new fire device that can decode AV1 haha. I believe the MECOOL boxes are Google certified, so they should work with almost everything but Netflix..

quietvoid

8th October 2021, 16:37

FireTV Stick 4K Max:
AV1: Hardware accelerated up to 3840x2160p (4K) @60fps, 100Mbps, Main Profile Level 5.1, Color space 8-bit and 10-bit input with HDR10, HDR10+, and HLG.

Plays smoothly with Youtube 4K 60fps HDR clips on Kodi 20 Nexus.

benwaggoner

13th October 2021, 00:37

So I wonder how much longer we have to wait for Amazon to release a new fire device that can decode AV1 haha.
Turns out "seven months."

quietvoid

22nd October 2021, 01:36

Well this is unfortunate. The decoder in the FireTV Stick 4K Max is not applying film grain on playback.
Has anyone checked if others supporting the Main profile do apply it?

benwaggoner

22nd October 2021, 04:21

Well this is unfortunate. The decoder in the FireTV Stick 4K Max is not applying film grain on playback.
Has anyone checked if others supporting the Main profile do apply it?
It's not applying the FGS filter, even though the AV1 bitstream is flagged with it?

If you can get me a repro file, I can get someone to take a look.

hajj_3

22nd October 2021, 11:11

Amlogic A311D2 supports 8K AV1 @24p/4K 60p: https://www.cnx-software.com/2021/10/21/amlogic-a311d2-arm-processor-16gb-ram/

quietvoid

22nd October 2021, 13:17

It's not applying the FGS filter, even though the AV1 bitstream is flagged with it?

If you can get me a repro file, I can get someone to take a look.

Yes that's right. I tested it with a libaom encoded file that has film grain params in the stream, but there is only one set that is repeated for all the frames.

You can find the file here: https://mega.nz/file/wR0AmY7K#eG3IIcTbFkdUyBXn9TaOL-ImpEKDZgBOAcLEeVfJwh0

I've played it with both Kodi and https://github.com/moneytoo/Player, both of which use the hardware decoder (JustPlayer uses ExoPlayer) and result in no grain application.
While mpv for Android is in software and applies the grain.

I'll have to test with a complete grain table to see if it makes a difference.

benwaggoner

27th October 2021, 01:54

Yes that's right. I tested it with a libaom encoded file that has film grain params in the stream, but there is only one set that is repeated for all the frames.

You can find the file here: https://mega.nz/file/wR0AmY7K#eG3IIcTbFkdUyBXn9TaOL-ImpEKDZgBOAcLEeVfJwh0

I've played it with both Kodi and https://github.com/moneytoo/Player, both of which use the hardware decoder (JustPlayer uses ExoPlayer) and result in no grain application.
While mpv for Android is in software and applies the grain.

I'll have to test with a complete grain table to see if it makes a difference.
Thank you. It is being investigated. There was a Broadcom SDK issue (I believe fixed in the latest updates) that I know impacted some AV1 FGS decode, so it might be related to that.

soresu

27th October 2021, 03:12

Amlogic A311D2 supports 8K AV1 @24p/4K 60p: https://www.cnx-software.com/2021/10/21/amlogic-a311d2-arm-processor-16gb-ram/

If you read the article fully it mentions that 8K video output is not supported despite being able to decode video at that resolution.

Apparently 8K output is expected on the long awaited S908X which may be a bit beefier with A76/A55 cores instead vs A311D2's A73/A53 core configuration.

olduser217

28th October 2021, 08:27

Fire TV Stick 4K Max uses Broadcom SoC?
According to Wikipedia, its CPU is Mediatek MT8696.

Thank you. It is being investigated. There was a Broadcom SDK issue (I believe fixed in the latest updates) that I know impacted some AV1 FGS decode, so it might be related to that.

quietvoid

28th October 2021, 14:27

I'm pretty sure it's Mediatek, all the MediaCodec decoders advertised are MTK.

benwaggoner

29th October 2021, 18:21

If you read the article fully it mentions that 8K video output is not supported despite being able to decode video at that resolution.

Apparently 8K output is expected on the long awaited S908X which may be a bit beefier with A76/A55 cores instead vs A311D2's A73/A53 core configuration.
FWIW, it's not like 8K video is happening any year soon (no evidence yet that humans can see any difference between 4K and 8K in moving picture content). And 8K TVs typically have very advanced upsamplers in them, so it's probably better to feed native 4K to them and let the TV take it up to 8K.

Spyros

4th January 2022, 18:13

AMD's new mobile CPUs (with Zen 3+ and RDNA 2 architectures) support AV1 hardware decode.

https://i.imgur.com/CNkwEts.jpg

Blue_MiSfit

6th January 2022, 07:22

Yet their newly announced lower end 6500 series discrete GPUs lack hardware decode! :confused:

benwaggoner

7th January 2022, 23:43

Yet their newly announced lower end 6500 series discrete GPUs lack hardware decode! :confused:
No discreet decode hardware for anything?

That would break a lot of things, including anything with a hardware DRM requirement.

Blue_MiSfit

8th January 2022, 02:31

They lack AV1 decode specifically :)

Curiously they also gimped hardware AVC and HEVC encoding on these products, which didn't make any sense to me.

benwaggoner

11th January 2022, 22:43

They lack AV1 decode specifically :)

Curiously they also gimped hardware AVC and HEVC encoding on these products, which didn't make any sense to me.
How gimped?

rwill

11th January 2022, 23:17

How gimped?

I think "gimped" as in "removed".

Blue_MiSfit

12th January 2022, 07:22

It's not clear if the hardware encoders are still present, or restricted to 1080p..

hajj_3

12th January 2022, 16:43

How gimped?

i think i read that they removed h265 hardware encoding.

benwaggoner

12th January 2022, 18:27

It's not clear if the hardware encoders are still present, or restricted to 1080p..
Ugh.

I can kind of get why they might; HW encoders are mainly used for videoconferencing stuff in practice, where 1080p is a stretch goal. I've maybe used the HW encoders is my a6000 GPU three times ever.

quietvoid

27th April 2022, 15:56

ChaosKing

27th April 2022, 16:30

@quietvoid I would like to test it on my firetv max. Are there publicly available samples with film grain synthesis to download anywhere? AV1 works with Kodi 20 (dev version) btw.

EDIT
Found your link https://forum.doom9.org/showthread.php?p=1955516#post1955516 :)
Hmm yeah I don't see any generated noise in Kodi.

EDIT2
Looks correct in VLC, but not sure if HW accelerated...

Spyros

30th August 2022, 20:21

Beyond the core, the Ryzen 7000 Series Processors feature an all-new 6nm I/O die, which enables hardware-accelerated video encode/decode*, light-duty graphics work and multi-display support.

* Video codec acceleration (including at least the HEVC (H.265), H.264, VP9, and AV1 codecs) is subject to and not operable without inclusion/installation of compatible media players.

Press Release (https://ir.amd.com/news-events/press-releases/detail/1089/amdlaunches-ryzen-7000-series-desktop-processors-with)

ChaosKing

6th September 2022, 20:24

7 months have passed and the FireTV Stick 4K Max still doesn't do film grain synthesis correctly on playback.
At this point I've yet to watch anything in AV1 because of this.

I also don't think it supports AV1 through Netflix yet, but I haven't been subscribed for a couple months now.

I just updated my FireTV Stick Max and now I can see the generated grain by the av1 decoder in Kodi. It finally works!

When I hit pause in Kodi the grain still "moves" lol
I use Kodi 19 dev btw.

quietvoid

6th September 2022, 20:57

I just updated my FireTV Stick Max and now I can see the generated grain by the av1 decoder in Kodi. It finally works!

When I hit pause in Kodi the grain still "moves" lol
I use Kodi 19 dev btw.

Very cool, I just tested and it also works on my side. :thanks:

Beelzebubu

10th September 2022, 19:48

When I hit pause in Kodi the grain still "moves" lol

That sounds rather weird. Certainly not quite what you'd expect given how grain works in the spec...

benwaggoner

16th September 2022, 18:05

That sounds rather weird. Certainly not quite what you'd expect given how grain works in the spec...
Sounds like the FGS filter keeps running even when the video stops. Since it's an out-of-loop postprocessor, that sort of thing isn't unprecedented. In the implementations I'm aware of, the FGS filter isn't even part of the codec itself, and can be triggered by the same SEI metadata in other codecs as well.

Beelzebubu

17th September 2022, 18:10

Sounds like the FGS filter keeps running even when the video stops. Since it's an out-of-loop postprocessor, that sort of thing isn't unprecedented. In the implementations I'm aware of, the FGS filter isn't even part of the codec itself, and can be triggered by the same SEI metadata in other codecs as well.

True, except that the process is defined and the seed coded in the bitstream header. So if the seed is static (e.g. when paused, or because so coded), the grain looks the same every frame. So the above behaviour suggests the seed is changing even if the video is paused. Where does it get the new seeds from? And if from someplace else than the bitstream header, does that mean it looks anything like intended by the encoder?

(It's true that encoders like aomenc don't choose a specific seed based on reconstruction characteristics, they just randomly generate a number. But in theory, an encoder could select a seed expecting a particular reconstruction, and that's what I'd expect to see in the decoder as well then.)

ChaosKing

17th September 2022, 19:00

benwaggoner

18th September 2022, 02:37

True, except that the process is defined and the seed coded in the bitstream header. So if the seed is static (e.g. when paused, or because so coded), the grain looks the same every frame. So the above behaviour suggests the seed is changing even if the video is paused. Where does it get the new seeds from? And if from someplace else than the bitstream header, does that mean it looks anything like intended by the encoder?

(It's true that encoders like aomenc don't choose a specific seed based on reconstruction characteristics, they just randomly generate a number. But in theory, an encoder could select a seed expecting a particular reconstruction, and that's what I'd expect to see in the decoder as well then.)
Good points. Perhaps it is looping, or falls back to randomizing a seed if the seed remains the same for too many frames?