NVENC AV1 Discussion

[Blue_MiSfit]

Hello!

So I got a shiny new RTX 4000 series GPU and am messing about with the hardware AV1 encoder exposed by NVENC.

With some quick tire kicking on 1080p SDR content I was pleasantly surprised. Quality is NOT up to par with software AV1 implementations at high quality settings, but it's way better than their older HEVC and AVC hardware encoders, maybe even going toe to toe with x265's faster speed presets.

Anyone else taken this encoder for a drive?

[Selur]

Yup, I agree, it's really usable as a replacement for other HEVC and AVC encoding.
(also got an Intel Arc 380, but haven't really spend time comparing them in regard of av1 encoding quality)

[benwaggoner]

Originally Posted by Selur

Yup, I agree, it's really usable as a replacement for other HEVC and AVC encoding.
(also got an Intel Arc 380, but haven't really spend time comparing them in regard of av1 encoding quality)

I'd be curious to hear about Arc results. Intel's made some good encoders over the years.

[Selur]

Even when it's off-topic, I used ToS_1920x800_xdither.y4m and:

QSVEnc --y4m -i - --input-buf 16 --slices 0 --fps 24/1 --codec av1 --sar 1000:996 --icq 26 --quality best --bframes 7 --gop-len 120 --open-gop --colorrange limited --colormatrix bt709 --i-adapt --b-adapt --b-pyramid --colormatrix bt709 --output-depth 10 --output-file "J:\tmp\ToS_1920x800_xdither.av1"

with different icq values (23,25,26,27,29,31,33) and uploaded them to my GoogleDrive encoding speed was ~91-95fps (lower icq => slower speed).
I can do more encodes with different settings of sources if someone is interested.
(I encoded using a 'Asrock Intel Arc A380 Challenger ITX 6GB IC' card.)

Cu Selur

[Yups]

Originally Posted by benwaggoner

I'd be curious to hear about Arc results. Intel's made some good encoders over the years.

Rigaya compared Nvidia, Intel, AMD a while ago: https://rigaya.github.io/vq_results/


Nvidia and Intel fairly close, Nvidia seems to favor VMAF while Intel favors SSIM. AMD clearly below, although much better than H264 and HEVC.

On Intel to me HEVC CQP+16 bframes is better and probably best overall, the scores are extremely high, much higher than Intel+Nvidia AV1. In the past the second generation of a new h264 and h265 encoder improved quite a bit on GPUs, I hope its the same for AV1.

[Selur]

In the past the second generation of a new h264 and h265 encoder improved quite a bit on GPUs,...

That was due to the support for B-frames, I doubt there will be such great improvements for AV1, but who knows.

[Yups]

Originally Posted by Selur

That was due to the support for B-frames, I doubt there will be such great improvements for AV1, but who knows.

Nvidia yes. Intel supported B-frames from the beginning and there was a big improvement from Gen9 to Gen11 and smaller improvements to Gen12.1 and another small to Gen12.7. Check out 7700k and 12900k in the rigaya test.

H264 quality did improve over several generations on Intel iGPUs. Sandy Bridge to Ivy was quite big and another big one came with Haswell.

Meteor Lake iGPU gets a new Media IP which is upgraded from 12.55 to 13 including a 4x encode mode. There is a chance for quality improvements.

[Emulgator]

nVidia h265 @ BD and UHD-BD resolutions and bitrates:
I have noticed a good improvement from when I tested first (end 2021, beginning 2022) and the second time (Jan/Feb 2023),
so that I can consider it usable to work with now for quick UHD-BD projects not worth the meticulous CPU pace.
Both were on the same RTX3080, with B-frames. Thank you, rigaya !

[benwaggoner]

Originally Posted by Selur

That was due to the support for B-frames, I doubt there will be such great improvements for AV1, but who knows.

The more tools a codec can use, the more room there is for further improvements, particularly if more compute can be applied per pixel. ML mode decisions is an interesting area of research that will probably result in some big gains.

[writersblock29]

Originally Posted by benwaggoner

I'd be curious to hear about Arc results. Intel's made some good encoders over the years.

Hi, Ben! I'm using an ARC A750, so I think I can help you with that. I had posted a reply earlier, but later deleted it because:

1) I didn't want to hijack a thread meant to discuss NVENC AV1.

2) Most of my objective data from earlier testing has been deleted after I satisfied my own curiosities, meaning that most of what I had posted was purely subjective and of little use to you.

3) I discovered a fundamental flaw in my previous testing methodology which made my post even more useless to you.

4) Hardware encoding is damned fast, so how lazy do I have to be not to run a few tests?

It'll be limited in scope (I don't have much free time these days--and as I type this, I'm about to go to work), but I do think it'll cover a slice of real-world usage that it'll give you something to chew on. Might take me a few days to get it out there, though.

[writersblock29]

Alrighty, then. Here it goes:

File chosen: The X Files, S01E01 Pilot. Taken from the US release of the Season One bluray. Video bitrate of 24.7Mb/s.

Why? Because it's a good transfer with lots of detail. It also has a moderate amount of film grain, as one would find in older films.

Software Used: Handbrake 1.6.1.
Why? While Hardware AV1 encoders are finding their way into more of our favorite GUIs, Handbrake probably has the most exposure out in the field. They're also listed on Intel's advertising for AV1, thus increasing exposure for the average layperson.

Settings Used: 10-bit QSV AV1 and H265 codecs, both set to their default maximum quality profiles. DTS English audio was recompressed to 640kbs 5.1 EAC3.

Why? Limit the variables to ICQ values only, with everything else being at developer standards. Max quality settings seemed logical to me, so that's what I went with. As for the audio, I did that on autopilot since that's what I always do: I share a Plex server with multiple clients, and EAC3 is widely supported without triggering transcoding. Other than file size considerations, audio's not relevant here.

Results?

AV1

ICQ20 -- 9,534 kbs, 3.49GB. VMAF 95.13
ICQ22 -- 7,079 kbs, 2,65GB. VMAF 94.14
ICQ24 -- 4,347 kbs, 1.71GB. VMAF 92.38
ICQ26 -- 2,243 kbs, .99GB. VMAF 89.49
Stopped here since we're lower than VMAF 90.

H265

ICQ20 -- 5,515 kbs, 2.11GB. VMAF 93.17
ICQ22 -- 3,459 kbs, 1.40GB. VMAF 91.29
ICQ24 -- 2,449 kbs, 1.06GB. VMAF 89.96
Stopped here since we're lower than VMAF 90.

Disclaimers: I did not make use of AV1's film grain synth, which I suspect might give this codec an edge over H265. I wanted to limit variables and... okay, fine. I don't know what to type into Handbrake's encoder options to make that happen, or if Handbrake even supports it in the first place. So nope! It's not accounted for here.

Hope that helps!

[benwaggoner]

I've seen data that suggests AV1 got enough VMAF tuning in design that its VMAF scores somewhat outperform true subjective ratings.

[writersblock29]

Originally Posted by benwaggoner

I've seen data that suggests AV1 got enough VMAF tuning in design that its VMAF scores somewhat outperform true subjective ratings.

New kid on the block figured out how to cheat the system, eh? Well, that little...

[benwaggoner]

Originally Posted by writersblock29

New kid on the block figured out how to cheat the system, eh? Well, that little...

I don't think there was any intentional cheating. But AV1 was the first codec designed since VMAF came out, and VMAF was used a lot in late stage bitstream testing and in reference encoder development. It's pretty much inevitable that whatever metrics get used in encoder optimization the encoder gets optimized for. VMAF has its limitations, but it's better than PSNR at least.

VMAF has so many different versions and models and modes it's kind of hard to just give a number for and know what it means without a big footnote.

And in general, a metric that's a mean of the per-frame metrics can miss all sorts of temporal discontinuities that impact quality. We've not really found a good solution for that as an industry, but it's still a problem.