x266 VVC Encoder

[kurkosdr]

Originally Posted by GeoffreyA

Dark Shikari and akupenguin are sorely needed to come back and do some codec magic in 2025

What are the chances even fairly techy people owning VVC-capable hardware? Why would they spend effort improving an encoder for a standard they probably can't play? x264 benefited from all that H.264 decoding hardware in GPUs, as well as H.264 being the chosen format for HD resolutions in most consumer electronics. This meant volunteers had an incentive to improve x264 to see better pictures on their hardware. But for UHD resolutions (and HDR content), this spot is now occupied by HEVC and to a degree AV1.

In this sense, it's good that x266 is commercially sponsored, since we may eventually see it happening. Who knows, it might prove useful in the future.

[Z2697]

Originally Posted by kurkosdr

What are the chances even fairly techy people owning VVC-capable hardware? Why would they spend effort improving an encoder for a standard they probably can't play? x264 benefited from all that H.264 decoding hardware in GPUs, as well as H.264 being the chosen format for HD resolutions in most consumer electronics. This meant volunteers had an incentive to improve x264 to see better pictures on their hardware. But for UHD resolutions (and HDR content), this spot is now occupied by HEVC and to a degree AV1.

In this sense, it's good that x266 is commercially sponsored, since we may eventually see it happening. Who knows, it might prove useful in the future.

You knew software decoders do exist, right?
That being said, having HW decoder is certainly better.

[kurkosdr]

Originally Posted by Z2697

You knew software decoders do exist, right?
That being said, having HW decoder is certainly better.

Hmm... Can VVC be decoded in software without having the latest and greatest CPU? And with what software?

[FranceBB]

There are two main software decoders: VVDec and Libav.
The latter was created later compared to VVDec and was originally written in plain C, but then it got manually written intrinsics in assembly and it has been closing the gap over the last year.
In particular, the open source decoder inside libav was written by Nou Mi and Frank Plowman. I was able to chat to them both but in particular Frank recently held a workshop at Sky in which he detailed what led to the creation of the decoder and how to detect issues with it.




The reason why this is particularly important is that when they introduced the decoder, it ended up spreading everywhere, from Avisynth and VapourSynth indexers like LWLibavVideoSource() and FFVideoSource(), to players like MPV and VLC and of course FFMpeg itself which is used pretty much universally. It was a major thing. When it was first released, it surely struggled to decode videos on most CPUs, but then again, it was just plain C without any assembly, while nowadays the picture is very different.

Here's the decoding benchmark (and remember that you only really need the fps to match the one of the content when you're actually watching something and not indexing to re-encode):


Test 1:
Chicago Fire FULL HD H266 Main10 12Mbits 25p YV12 BT709 SDR 10bit AC3 2ch 48000Hz.ts

120 fps
Intel Xeon E5-2637 v4 3.50GHz dual socket 8c/16th, 64GB RAM

111 fps
Intel i7 5930K 3.50GHz single socket 6c/12th 32GB RAM


Test 2:
Judas and the Black Messiah SD H266 Main10 3Mbits 25p YV12 BT601 SDR 10bit AC3 2ch 48000Hz.ts

620 fps
Intel Xeon E5-2637 v4 3.50GHz dual socket 8c/16th, 64GB RAM

631 fps
Intel i7 5930K 3.50GHz single socket 6c/12th 32GB RAM

Test 3:
The Creator FULL HD H266 Main10 12Mbits 25p YV12 BT709 SDR 10bit AC3 2ch 48000Hz.ts

121 fps
Intel Xeon E5-2637 v4 3.50GHz dual socket 8c/16th, 64GB RAM

116 fps
Intel i7 5930K 3.50GHz single socket 6c/12th 32GB RAM


If you need the samples let me know and I'll send them over (trimmed down obviously, not the whole thing). To test it yourself you just need to download the latest version of MPV from master
I'll generate some UHD files too to see how well it copes, but the future definitely looks exciting, software wise.

[Z2697]

Originally Posted by FranceBB

Frank recently held a workshop at Sky in which he detailed what led to the creation of the decoder and how to detect issues with it.

I'm very curious about the details here, there's patchset for libvvdec (and libvvenc) inclusion as 3rd party components before the native vvc decoder, at least what we can see in public ffvvc fork.
Eventually libvvenc did get into official FFmpeg, but not libvvdec.

vvdec is still faster than libav vvcdec, by 1.37x (1T) or more depending on the requested threads, on Zen 5.
I think FFmpeg has a "tradition" of implementing MPEG decoders natively AND not support 3rd party ones?
(while VP series have libvpx and libdav1d as well as native VP9 and AV1(HW decoding only))

Is it OK for you to share some stories?

[GeoffreyA]

Originally Posted by Z2697

I'm very curious about the details here, there's patchset for libvvdec (and libvvenc) inclusion as 3rd party components before the native vvc decoder, at least what we can see in public ffvvc fork.
Eventually libvvenc did get into official FFmpeg, but not libvvdec.

vvdec is still faster than libav vvcdec, by 1.37x (1T) or more depending on the requested threads, on Zen 5.
I think FFmpeg has a "tradition" of implementing MPEG decoders natively AND not support 3rd party ones?
(while VP series have libvpx and libdav1d as well as native VP9 and AV1(HW decoding only))

Is it OK for you to share some stories?

There was some mild controversy a few years back about using vvdec. Nuo Mi argued that it would speed up work getting the infrastructure ready for the native decoder, which was already being developed at that time.

[GeoffreyA]

Originally Posted by FranceBB

But... you know, although highly unlikely, I'd say "never say never"... We've seen Avisynth legends like Didee and Dogway making a comeback, so who knows...

Regardless, he will forever be considered a legend on Doom9 for everything he has done (and not only around here but pretty much everywhere on the web).

I agree, it's unlikely. But never say never!

Originally Posted by ksec

I really want another encoder that takes the no nonsense approach. As a matter of fact I wish they could be back to work on x264 in a way that breaks H.264 limitation but in a completely patent free way.

Indeed, there's probably compression and quality to be had by backporting some modern techniques to x264.

[ksec]

Originally Posted by kurkosdr

Hmm... Can VVC be decoded in software without having the latest and greatest CPU? And with what software?

Hmm...... VVC is already being software decoded on Smartphone for nearly 3 years in Apps for certain regions.

[GeoffreyA]

Originally Posted by ksec

Hmm...... VVC is already being software decoded on Smartphone for nearly 3 years in Apps for certain regions.

With libdav1d, we've seen what continuous optimisation can achieve, so FFmpeg's VVC decoder could, in theory, reach something similar.

[Z2697]

Originally Posted by ksec

As a matter of fact I wish they could be back to work on x264 in a way that breaks H.264 limitation but in a completely patent free way.

When you break H.264 limitation in a certain way you've essentially created... H.265.
Or anything. It will be a new codec anyway.
New codec means incompatible with existing decoders, you need to at least hack the decoder for it to work.

[birdie]

Originally Posted by FranceBB

Test 1:
Chicago Fire FULL HD H266 Main10 12Mbits 25p YV12 BT709 SDR 10bit AC3 2ch 48000Hz.ts

120 fps
Intel Xeon E5-2637 v4 3.50GHz dual socket 8c/16th, 64GB RAM

111 fps
Intel i7 5930K 3.50GHz single socket 6c/12th 32GB RAM

Test 3:
The Creator FULL HD H266 Main10 12Mbits 25p YV12 BT709 SDR 10bit AC3 2ch 48000Hz.ts

121 fps
Intel Xeon E5-2637 v4 3.50GHz dual socket 8c/16th, 64GB RAM

116 fps
Intel i7 5930K 3.50GHz single socket 6c/12th 32GB RAM

1080p and such massive beasts of CPUs don't really look too good IMO.

Would be great if you compared ffmpeg's native H266 decoder with VVdec.

[BlueSwordM]

To be honest, these numbers are not great, but the CPUs aren't great anymore; they're not massive beasts anymore.

The 5930K is Haswell, and Xeon E5 v4 chips are Broadwell.

[benwaggoner]

Originally Posted by GeoffreyA

Indeed, there's probably compression and quality to be had by backporting some modern techniques to x264.

What do you mean by " no nonsense approach?"

Arguably, x264's brilliance was from ignoring PSNR and focusing on all sorts of nonsense like adaptive quantization, psychovisually weighted QP, tuning for anime content, etc. It was really a big departure from how encoders had traditionally been designed and tuned, by a very different set of people than normally worked on encoders historically.

We can forget what a brilliant weirdo x264 was sometimes because so many of its crazy ideas inspired so much that followed that they seem obvious and standard in retrospect.

But CRF and MB-Tree were not obvious before x264 implemented them. Single pass at constant perceptual quality but VBV enforcement was also very novel.

[rwill]

Originally Posted by benwaggoner

We can forget what a brilliant weirdo x264 was sometimes because so many of its crazy ideas inspired so much that followed that they seem obvious and standard in retrospect.

Did not take long for the industry to go full "VMAF approved" style after that. I blame people being too lazy to look at their videos after encoding, they just want the computer to say "its ok" and be done.

[LigH]

will not mention MVCD or KVCD ever again

[ksec]

Originally Posted by Z2697

When you break H.264 limitation in a certain way you've essentially created... H.265.
Or anything. It will be a new codec anyway.
New codec means incompatible with existing decoders, you need to at least hack the decoder for it to work.

Indeed incompatible, I am thinking a much higher resources usage, but patent free / patent expired codec. I think Main Profile is near Patent Free, we are a few more years till High Profile is mostly patent free as well.

With Larger block size, macroblock, more / higher aggressive transform adaptivity basically brute force everything we have in H.264. I am wondering if we could reach compression that is HEVC+.

[benwaggoner]

Originally Posted by rwill

Did not take long for the industry to go full "VMAF approved" style after that. I blame people being too lazy to look at their videos after encoding, they just want the computer to say "its ok" and be done.

Well, it sure would be so convenient to have an AI that tunes quality based on ratings from an AI that rates quality!

Alas, quality is only as good as humans perceive it to be, not AIs, and subjective correlation still isn't great. P1204.3 can outdo VMAF, but they are both ML-trained metrics and such reliant on the quality of their ground truth training data. VMAF was quite limited originally by being tested only on a relatively small set of x264 variations (mainly just varying CRF and frame size).

But AI optimizing for AI scoring can yield some weird results. For example, as VMAF excludes chroma, enabling chroma qp offset turning for VMAF will make chroma look much worse in exchange for slightly improved luma.

And no one has a good, practical metric for noise/grain similarity yet.

[kurkosdr]

Originally Posted by ksec

Indeed incompatible, I am thinking a much higher resources usage, but patent free / patent expired codec. I think Main Profile is near Patent Free, we are a few more years till High Profile is mostly patent free as well.

With Larger block size, macroblock, more / higher aggressive transform adaptivity basically brute force everything we have in H.264. I am wondering if we could reach compression that is HEVC+.

It seems like you are looking to make something like EVC Baseline. Can't you use EVC Baseline instead? An open-source encoder and decoder called "xeve" exists.

Problem is hardware support for standalone players. OEMs add support for formats only when they are needed to implement some headline feature.

[Z2697]

Originally Posted by kurkosdr

It seems like you are looking to make something like EVC Baseline. Can't you use EVC Baseline instead? An open-source encoder and decoder called "xeve" exists.

Problem is hardware support for standalone players. OEMs add support for formats only when they are needed to implement some headline feature.

I think the point is x264-level quality, both the encoding outcome and the design of encoder itself.
XEVE is not of that level of quality... yet? Or will it?

[kurkosdr]

Originally Posted by Z2697

I think the point is x264-level quality, both the encoding outcome and the design of encoder itself.
XEVE is not of that level of quality... yet? Or will it?

The authors of xeve claim better-than-x264-level quality for EVC Baseline: https://github.com/mpeg5/xeve

So it can probably achieve x264-level quality.