Versatile Video Coding (VVC) / H.266: HEVC successor

[rwill]

Well, do you have a source for your statement that x265 is better than vvenc at lower resolutions at visually lossless encoding? Besides your analogue eyes?

[birdie]

Whelp, here it is.

Open the PNG files and check how vvenc abso-fecking-lutely crushed the thumb fine details. The difference is night and day. And I've not even used any fancy x265 encoding flags.

And the encoding time for vvenc was about 20 times longer. It's a disaster.

And that's why I don't trust PSNR and other fancy automatic video quality assessment metrics. You may get better numbers but the net result will be utter crap.

[kurkosdr]

Why do people still use PSNR other than a complementary metric? It should be well-known by this point that PSNR doesn't accurately measure picture quality because it tends to favor blur over maintaining psychovisual quality. Use SSIM (or VMAF).

[birdie]

Originally Posted by kurkosdr

Why do people still use PSNR other than a complementary metric? It should be well-known by this point that PSNR doesn't accurately measure picture quality because it tends to favor blur over maintaining psychovisual quality. Use SSIM (or VMAF).

I'm not well versed in all these metrics but what I'm looking forward to is a metric that finds contrast edges and compares based on that.

Contrast edges are the image areas where you can clearly see lines and other abrupt changes that are not gradients. Video codecs have been able to encode gradients with a very high efficiency lately but that's not what video encoding IMO is in general about.

There have been a number of AI encoding companies that have popped up recently, and that's exactly what they're doing. They use AI techniques to identify the areas of the frame that are gradients or something that doesn't require a lot of bitrate and the areas that have fine detail. Then they tell the video codec how to use its bitrate allocation for that frame, and the result is spectacular.

SSIM is probably it but I've heard that encoders have learned to cheat this metric as well, so I still prefer to use my eyes to judge the end result.

[birdie]

And vmaf straight out lies about my encodes:

$ vmaf -r source.y4m -d vvenc.y4m 
VMAF version 2.3.0
30 frames ⡃⢐ 31.97 FPS
vmaf_v0.6.1: 99.889631

$ vmaf -r source.y4m -d x265.y4m 
VMAF version 2.3.0
30 frames ⡃⢐ 31.61 FPS
vmaf_v0.6.1: 99.843432

$ vmaf -r source.y4m -d source.y4m 
VMAF version 2.3.0
30 frames ⡃⢐ 31.01 FPS
vmaf_v0.6.1: 99.914262

1. It says vvenc did a better job while my eyes clearly see otherwise
2. It assigns near perfect scores to the encodes both of which are hardly visually lossless
3. When the source is equal to the result the score is not 100. That's just funny.

[birdie]

And SSIM doesn't seem to paint the right picture either Image similarity comparison simulating human perception :

$ dssim img-source.png img-vvenc.png img-x265.png 
0.002890	img-vvenc.png
0.003232	img-x265.png

0 meaning no difference and then going to infinity for something completely different.

It's only PNG that seems to get it right:

ls -l *png
587014 img-vvenc.png
657898 img-x265.png
689009 img-source.png

Where there's more complexity you get a bigger file size.

[kurkosdr]

Most modern codecs allow you to "tune" for a particular metric (at least for psnr or ssim), x265 does, does vvcenc have the ability to "tune" for a particular metric? If so, try comparing x265 and vvcenc both "tuned" for the same metric.

And then there is always the possibility that vvcenc sucks for 1080p SDR content compared to x265.

[birdie]

Originally Posted by kurkosdr

Most modern codecs allow you to "tune" for a particular metric (at least for psnr or ssim), x265 does, does vvcenc have the ability to "tune" for a particular metric? If so, try comparing x265 and vvcenc both "tuned" for the same metric.

I've not tuned x265 or vvenc for anything and that's exactly how most people use video codecs. I've used 100% default settings (except for disabling SAO for x265 which blurs out everything).

Originally Posted by kurkosdr

And then there is always the possibility that vvcenc sucks for 1080p SDR content compared to x265.

Exactly what I claimed earlier: modern codecs, AV1/VVC are optimized for 4K/HDR/10/12bit videos.

[kurkosdr]

Originally Posted by birdie

I've not tuned x265 or vvenc for anything and that's exactly how most people use video codecs. I've used 100% default settings (except for disabling SAO for x265 which blurs out everything).

By disabling SAO, you did change something in x265. I wonder what the equivalent of SAO for VVC is (and what the switch is on vvcenc)

But anyway, the correct procedure is to use the same tuning or disable any tunings, no matter what the default settings are.

[ShortKatz]

Originally Posted by rwill

@ShortKatz: Is this with an Intel or ARM MacBook Pro ?

It is an Intel MacBook Pro (would like to wait for M4 for my switch).

[birdie]

Originally Posted by kurkosdr

By disabling SAO, you did change something in x265. I wonder what the equivalent of SAO for VVC is (and what the switch is on vvcenc)

But anyway, the correct procedure is to use the same tuning or disable any tunings, no matter what the default settings are.

https://github.com/fraunhoferhhi/vvenc/wiki/Usage - I don't see anything SAO (Sample Adaptive Offset loop filter) related.

You'd think that a brand new codec with roughly three times more instruments (algos) to conserve bitrate doesn't need some special options not to blur everything out, no?

Anyways, I've reenabled SAO for x265, the result is slightly softer but still substantially better than for vvenc at the same bitrate.

All the files have been uploaded to the same shared mega folder.

VVC implements an additional new in-loop Adaptive Loop Filter (ALF) on top of the DBF and the SAO filter, the latter remaining unchanged compared to HEVC.

No idea how to disable it.

Edit:

Interesting. CTU32 is faster because of smaller search space, but for your command also because of the improved multi-threading, this explains the very high differences. To have a proper comparison you'd also need to compare the resulting quality.

To reduce blurriness I'd recommend trying one of: --ALF=0, --MCTF=0. It'll significantly increase the bitrate, so you might also just want to try to decrease QP/increase target bitrate.

Source.

[benwaggoner]

Originally Posted by birdie

And vmaf straight out lies about my encodes:

$ vmaf -r source.y4m -d vvenc.y4m 
VMAF version 2.3.0
30 frames ⡃⢐ 31.97 FPS
vmaf_v0.6.1: 99.889631

$ vmaf -r source.y4m -d x265.y4m 
VMAF version 2.3.0
30 frames ⡃⢐ 31.61 FPS
vmaf_v0.6.1: 99.843432

$ vmaf -r source.y4m -d source.y4m 
VMAF version 2.3.0
30 frames ⡃⢐ 31.01 FPS
vmaf_v0.6.1: 99.914262

1. It says vvenc did a better job while my eyes clearly see otherwise
2. It assigns near perfect scores to the encodes both of which are hardly visually lossless
3. When the source is equal to the result the score is not 100. That's just funny.

VMAF is just a machine learning MOS predictor trained originally on some pretty basic x264 variants and subjective ratings scores. I hope Netflix has added newer codecs and modes to its ground truth data, but I would be seriously surprised if the public VMAF model has ever seen a VVC bitstream, and so doesn't have any implicit knowledge of how people would subjectively rate VVC-style artifacts.

Like any ML model, VMAF does pretty well in interpolating values within the range of things tested in its ground truth data corpus, but is going to be a lot less predictable at extrapolating values for content that has attributes outside of what it has been trained on.

Still, it's about as good as we have for "objective" metrics. The p1204 metric in full reference mode but excluding bitstream data may do as well or better.

As always in the early days of encoder development and tuning, we need to rely on our eyeballs first and foremost until we get updated metrics and/or some experience with how to usefully apply existing metrics.

[rwill]

Originally Posted by oibaf

What is the "Mystery HEVC Encoder"?

That is... a Mystery!

Originally Posted by ShortKatz

It is an Intel MacBook Pro (would like to wait for M4 for my switch).

Hm, some of my Mac using colleagues reported thermal Problems when they did processing for more than 5 Minutes or so. And I hope yours is new enough to support AVX2.
M4 eh? When I look at the vvenc source the x86 specific folder has around 900kb of files, the arm specific folder 50kb. Good Luck ..

[MZ/X]

Just a notification about a VVC/H266 capable Windows video player.

The latest Mpxplay-MMC v3.21 uses FFmpeg v6.1.1 with VVdec Fraunhofer Versatile Video Decoder v2.3.x (20240625) to play VVC encoded content.
AV1 sw/hw decoding and DVB devices are also supported.
There are 32/64 bit and installer/portable versions.

Homepage: https://mpxplay.sourceforge.net
FB page: https://www.facebook.com/mpxplay

[FranceBB]

Looks like there's gonna be some joy for those running Linux as well as far as H.266 hardware decoding is concerned.
In the last release of libva (2.22) from Intel the changelog says:

- Added VVC decode LibVA interface.

This is almost definitely for the GPUs included in the upcoming Intel Lunar Lake processors, so come September of this year whoever is gonna buy those processors is gonna have VA-API support for H.266 hardware decoding, which means MPV hardware decoding straight out of the box. Now that's pretty cool.

[birdie]

Originally Posted by FranceBB

Looks like there's gonna be some joy for those running Linux as well as far as H.266 hardware decoding is concerned.
In the last release of libva (2.22) from Intel the changelog says:

- Added VVC decode LibVA interface.

This is almost definitely for the GPUs included in the upcoming Intel Lunar Lake processors, so come September of this year whoever is gonna buy those processors is gonna have VA-API support for H.266 hardware decoding, which means MPV hardware decoding straight out of the box. Now that's pretty cool.

Lunar Lake laptops are yet to be released.

[FranceBB]

Originally Posted by birdie

Lunar Lake laptops are yet to be released.

Yep, but by the end of the year, so things are starting to come together.

[nevcairiel]

Originally Posted by FranceBB

which means MPV hardware decoding straight out of the box. Now that's pretty cool.

It does not mean that. You still need a decoder to actually utilize the libva interface for that - eg. ffmpeg needs to implement it. And with expensive mobile-only hardware being the only testbed .. unless Intel sponsors a developer to do that by providing hardware, might take a while.

[FranceBB]

Ah... I see...

[MoSal]

Intel and AMD do support and play well with the open-source graphics stack. So, early support in libva is more of an expectation rather than a pleasant surprise.

VK_KHR_video_decode_vvc getting added to Vulkan, and subsequently getting implemented by some Mesa drivers, now that would be an interesting development (if/when it happens).