Versatile Video Coding (VVC) / H.266: HEVC successor

[FranceBB]

On a more positive news, it looks like Nou Mi is still working on libav's H.266 VVC decoder as he just added manually written AVX2 assemblies: Link
If I find the time (and that's a big "if") this week and I'll compile a new version of FFMpeg to test how the decoder is doing in my series of sample test clips in terms of speed now that AVX2 have been introduced.
Hopefully it will reduce the gap between libav and VVDec.

[birdie]

Originally Posted by FranceBB

On a more positive news, it looks like Nou Mi is still working on libav's H.266 VVC decoder as he just added manually written AVX2 assemblies: Link
If I find the time (and that's a big "if") this week and I'll compile a new version of FFMpeg to test how the decoder is doing in my series of sample test clips in terms of speed now that AVX2 have been introduced.
Hopefully it will reduce the gap between libav and VVDec.

Last time I checked vvdec was three times faster. I don't think ffmpeg's implementation is even close to closing the gap :-) And palette support is yet to be merged.

[Z2697]

They said they were not going to accept libvvdec and that's why Nuo Mi had to write a "native" decoder from scratch (no one forced him of course, but if anyone wants vvc decode in ffmpeg, someone has to write a native one).
It's not like they "forced" people to write native av1 decoder, they just use libdav1d, but hey, what can I say, perhaps the fact that the dav1d is developed by videolan makes them happier.

[GeoffreyA]

Originally Posted by Z2697

They said they were not going to accept libvvdec and that's why Nuo Mi had to write a "native" decoder from scratch (no one forced him of course, but if anyone wants vvc decode in ffmpeg, someone has to write a native one).
It's not like they "forced" people to write native av1 decoder, they just use libdav1d, but hey, what can I say, perhaps the fact that the dav1d is developed by videolan makes them happier.

I remember reading the discussion about that. There seemed to be politics going on, where one of the developers was opposing that anything VVC should be included. There was also opposition to the native decoder that Nuo Mi and others had been working on for some time. If I remember correctly, Mi reasoned that, if not the native decoder, using libvvdec for the time being would allow them to work on the CBS module, etc.

[nevcairiel]

Originally Posted by Z2697

They said they were not going to accept libvvdec and that's why Nuo Mi had to write a "native" decoder from scratch (no one forced him of course, but if anyone wants vvc decode in ffmpeg, someone has to write a native one).

You got the timeline mixed up here, vvdec was not accepted because the native decoder was in the works.

[FranceBB]

Yep, the native decoder was in the works and therefore they pushed back on the inclusion of VVDec much to Adam Wieckowski's disappointment.
I was also a bit disappointed and many people were, 'cause the whole thing wasn't 100% clear back then and many people thought that the patches were being refused 'cause there were some hardcore AV1 supporter that wanted to see H.266 VVC fail. That wasn't true, of course, but the fact that people didn't know that there was a native decoder in the works led to some very speculative (and often rude) comments in the mailing list and on various channels. What's ironic is that the refinement and improvement of the H.266 VVC decoder were done as one of the projects of the Google Summer of Code 2023 Link by Nuo Mi, Anton Khirnov and of course Frank Plowman, so, ironically, Google, the one behind AV1, sponsored that. This obviously brought all the flame and the controversies to an end, but still the amount of disinformation that spread earlier on was astonishing and the whole thing was very sad to read...

[Z2697]

I didn't really paid attention to the timeline when I saw that conversation, that's true. I can't find it again, anyone has link to that mailing list thread? (where they also discussed libdav1d being integrated and libopus being the only non-experimental encoder for opus, but the main topic is libvvdec)

[GeoffreyA]

Originally Posted by Z2697

I didn't really paid attention to the timeline when I saw that conversation, that's true. I can't find it again, anyone has link to that mailing list thread? (where they also discussed libdav1d being integrated and libopus being the only non-experimental encoder for opus, but the main topic is libvvdec)

https://patchwork.ffmpeg.org/comment/60589/

[Z2697]

Originally Posted by GeoffreyA

https://patchwork.ffmpeg.org/comment/60589/

That's exactly what I was looking for! Thanks!

So the conversation was indeed happend long before we see the work of native vvc decoder in public, but after being reminded by nevcairiel and FranceBB I realized I shouldn't just imagine some conspiracy based on that. I mean maybe there're non-public developments.

Nuo Mi said in that thread:
We will not have a useable native vvc decoder in 0.5~1 years.

I don't know if that "we" refers to a team that Nuo Mi is/was in and was already working on native vvc decoder that moment.

[GeoffreyA]

I think it will be hard to know without asking Nuo Mi. If we look at the ffvvc fork where they developed the decoder, the first commits by Mi were after the date of that discussion (except for one related to HEVC).

https://github.com/ffvvc/FFmpeg/comm...354ef8d27c2+34

[hajj_3]

Allegro DVT Launches The Industry’s First Real-Time VVC/H.266 Encoder IP

https://www.allegrodvt.com/news/alle...66-encoder-ip/

[birdie]

Originally Posted by hajj_3

Allegro DVT Launches The Industry’s First Real-Time VVC/H.266 Encoder IP

https://www.allegrodvt.com/news/alle...66-encoder-ip/

That's old news and I mentioned it in the WP article several weeks ago.

[ShortKatz]

The experimental flag for VVC was removed from FFmpeg today. If you now build mpv with the most recent FFmpeg, you can play VVC files. I've tested this with 5 different VVC samples, all played without issue.

[ksec]

Originally Posted by ShortKatz

The experimental flag for VVC was removed from FFmpeg today. If you now build mpv with the most recent FFmpeg, you can play VVC files. I've tested this with 5 different VVC samples, all played without issue.

Which means we should expect this to officially land in 7.1, hopefully this October if not November.

Now we just need an encoder to play with. Unfortunately that is slower than expected. Normally we have experimental encoder first before decoder being officially a feature on FFmpeg.

[Z2697]

Originally Posted by ksec

Which means we should expect this to officially land in 7.1, hopefully this October if not November.

Now we just need an encoder to play with. Unfortunately that is slower than expected. Normally we have experimental encoder first before decoder being officially a feature on FFmpeg.

libvvenc is available now, or actually, available for quite some time now.

[ShortKatz]

Yes, you can build FFmpeg with --enable-libvvenc to get VVC encoding support. But encoding speed is very very slow.

[frankplow]

Originally Posted by birdie

Last time I checked vvdec was three times faster. I don't think ffmpeg's implementation is even close to closing the gap :-) And palette support is yet to be merged.

VVdeC also does not support the Main 10 4:4:4 profile, which includes the Palette tool.

I think Nuo Mi is focussing on performance now, seeing as the decoder is nearly Main 10 compliant. I haven't seen a difference quite as significant as 3x for some time and the gap is closing.



(i7-8700k, 12 hyperthreads)

[Jamaika]

Codec VVenc quality does not pass the test with ffmpeg
https://www.sendspace.com/file/5nfqhn
frame= 0 fps=0.0 q=0.0 size= 1KiB time=N/A bitrate=N/A speed=N/A
ERROR: In function "lastPOCInCache" in RateCtrl.h:185: Accessing empty cache
[vost#0:0/libvvenc @ 000001c546ea4310] Error submitting video frame to the encoder
[vost#0:0/libvvenc @ 000001c546ea4310] Error encoding a frame: Generic error in an external library
[vost#0:0/libvvenc @ 000001c546ea4310] Task finished with error code: -542398533 (Generic error in an external library)
[vf#0:0 @ 000001c54705b300] All consumers returned EOF
[vost#0:0/libvvenc @ 000001c546ea4310] Terminating thread with return code -542398533 (Generic error in an external library)

[birdie]

Originally Posted by frankplow

VVdeC also does not support the Main 10 4:4:4 profile, which includes the Palette tool.

I think Nuo Mi is focussing on performance now, seeing as the decoder is nearly Main 10 compliant. I haven't seen a difference quite as significant as 3x for some time and the gap is closing.

(i7-8700k, 12 hyperthreads)

I tested the builtin decoder months ago before optimizations and even in your example VVDec is almost twice as fast for the 4K sample so there's plenty room for improvement.

I'm quite sure the builtin decoder will surpass VVDec performance wise eventually but we are not there yet.

[LigH]

New uploads: [Windows][GCC 14.2.0][64 bit]

Fraunhofer VVC Encoder ver. 1.12.0 a1996a8

Fraunhofer VVC Decoder ver. 3.0.0-rc1 eb1944e