Versatile Video Coding (VVC) / H.266: HEVC successor

[ksec]

Originally Posted by birdie

I've never heard anyone say or imply that. Citations needed.



And it was merged, no prob.

Apart from Nuo Mi's commit, all the other VVC has nothing to do with H.267 itself but Dolby Vision Configuration Box aka 'dvvC'.

But I do remember it was a question from Nuo Mi on integration with VVC and FFmpeg. So I will probably look it up later.


Edit: https://patchwork.ffmpeg.org/project...ail.com/#60590

Lynne Dec. 22, 2020, 4:10 p.m. UTC | #4
Dec 21, 2020, 07:07 by nuomi2021@gmail.com:

> you can download test clips here:
> https://www.itu.int/wftp3/av-arch/jv...test/VTM-11.0/
>

Honestly, I'm not sure about this patch. Its an external library decoder, which
doesn't help us very much, and its for a codec competing with another codec
(AV1) for which we don't really have a native decoder for yet. Not to mention
some of us were involved with the effort to standardize and/or write a decoder
for AV1, so there's also a personal element to not wanting VVC to have any
success.

So I'm kind of against this particular patch as-is. I'd be okay with it being a native
decoder built into our codebase. But as-is, this serves nothing to help the
project, but only corporations who invested into the codec and want to see
some return on their investment from enforcing licensing on our users.
And no one is even really using this codec at the moment.



Nuo Mi Dec. 22, 2020, 5:02 p.m. UTC | #5
Hi Lynne,
Thanks for the comments.
We only have a small patch for the vvdec wrapper. All other codes (raw
demuxer, cbs reader, parser, and cbs writer in the future) will benefit the
native decoder and the vvc ecosystem.

I do not think av1 is the savior. No matter how it promised you, if we only
have one codec to choose from, it will be a disaster for all of us.
But, if you and the ffmpeg tech committee decided to reject all vvc patches
in the future, I can accept it and stop work on this.

BTW, do you know the detailed plan for av1 native decoder? will we port
dav1d back or implement a new one?

thanks

[FranceBB]

Originally Posted by ksec

Not to mention
some of us were involved with the effort to standardize and/or write a decoder
for AV1, so there's also a personal element to not wanting VVC to have any
success.

this was a personal opinion which would be fine, however this:

Originally Posted by ksec

So I'm kind of against this particular patch as-is.

is not.
I do understand people working on AV1 and such, but coming to the point of blocking an H.266 VVC decoding patch is too much. We should really be able to easily decode any codec, no matter what. With this logic, we would have never had reverse engineered decoders for codecs like DNxHR and Apple ProRes for video or indeed DolbyE and DolbyED2 for audio. Sure, one could make a reasoning on why he might want to see one codec succeed and another one fail, but not having the decoding isn't gonna help anyone, if anything it's gonna make things worse!

Think about DolbyE for instance.
You might want to say "Ah, FLAC should succeed so I'm not gonna add decoding".
Will corporations move to FLAC? Nope, they will just buy software decoders like DolbyDP600 and potentially make things worse as it would feed money straight to Dolby.
The same applies to video codecs, not having a video decoder for Apple ProRes would lead to people giving money to Apple 'cause they wouldn't be able to refuse such a masterfile, same thing for DNxHR with AVID and the same thing with MPEG-LA for H.266 VVC.

In a nutshell, I'm glad that nowadays Libav can decode pretty much anything (including proprietary codecs) and I really wouldn't want to see it stopping right now 'cause it would only make things harder for those who use those in professional settings like the various broadcasters.

Originally Posted by ksec

I do not think av1 is the savior. No matter how it promised you, if we only
have one codec to choose from, it will be a disaster for all of us.

This is exactly right and I think exactly the same.

Originally Posted by ksec

But, if you and the ffmpeg tech committee decided to reject all vvc patches
in the future, I can accept it and stop work on this.

This is even more dangerous.
We had 1 good guy working on the VVDec integration and they're making him go away and stop working on it, potentially. I mean, honestly, I really hope the people in the FFMpeg committee change their mind 'cause right now this way of thinking it's really not good and it's not gonna happen.
I mean, we can all be proud and thankful to google for what it has done with AV1 and everything, but we shouldn't really limit decoding to just open codecs 'cause it would mean the END of the open source tools in professional settings.

[nevcairiel]

Note that the main complaint is not against VVC in general, and such a complaint would never be able to stand anyway, but rather the patch merely being a thin wrapper around an external decoding library controlled by a corporation, rather then actual efforts to bring open and native decoding to FFmpeg.
If someone were to develop a native decoder for FFmpeg, and not an external library, none of these arguments would even exist. All these other proprietary codecs you mention have native decoders in FFmpeg.

I would recommend those "professional" users deriving revenue from FFmpeg to chip in and further such a project. FFmpeg does not make money, but developers need to eat.

PS:
The argument against "wrappers" is that they don't really further the open-source multimedia ecosystem. Even decoders benefit from competition, which is why dav1d was created by a group of ffmpeg developers (and others), instead of just using aom.

[Wiabol]

A decoder developed by a corporation. Bold opinion for a bunch of Google fanboys.

VVC, just as AV1 are becoming too complex to be integrated into the main codebase. That's why there's David.

[birdie]

Originally Posted by nevcairiel

Note that the main complaint is not against VVC in general, and such a complaint would never be able to stand anyway, but rather the patch merely being a thin wrapper around an external decoding library controlled by a corporation, rather then actual efforts to bring open and native decoding to FFmpeg.
If someone were to develop a native decoder for FFmpeg, and not an external library, none of these arguments would even exist. All these other proprietary codecs you mention have native decoders in FFmpeg.

I would recommend those "professional" users deriving revenue from FFmpeg to chip in and further such a project. FFmpeg does not make money, but developers need to eat.

PS:
The argument against "wrappers" is that they don't really further the open-source multimedia ecosystem. Even decoders benefit from competition, which is why dav1d was created by a group of ffmpeg developers (and others), instead of just using aom.

Tons of codecs are supported exactly this way in FFMpeg - by using "thin" wrappers for external libraries.

This includes AV1 as well.

[rwill]

Originally Posted by nevcairiel

The argument against "wrappers" is that they don't really further the open-source multimedia ecosystem. Even decoders benefit from competition, which is why dav1d was created by a group of ffmpeg developers (and others), instead of just using aom.

But this "argument" can easily be debunked.

VVdeC is under 3-clause BSD. IT hardly can be more open source than that. By that "argument"s logic x265 should be removed too as it is integrated with a wrapper and developed by a corporation.

[birdie]

I've just talked to FFMpeg developers in their IRC channel.

No one opposes VVC, it's just no one is currently interested in it, thus no one wants to review the patches to merge them. There are no other reasons. As simple as that.

At least yesterday's FFMpeg GIT snapshot works fine with them, so there's that.

I'm pretty sure if companies behind VVC actually cared, they'd sponsor someone to get this code merged. Darn.

[nevcairiel]

Originally Posted by rwill

But this "argument" can easily be debunked.

VVdeC is under 3-clause BSD. IT hardly can be more open source than that.

No where did I mention the license. Everyone relying on the exact same codebase to do decoding also doesn't debunk the competition I mentioned.

Originally Posted by rwill

By that "argument"s logic x265 should be removed too as it is integrated with a wrapper and developed by a corporation.

We are talking about decoders here. Encoding is a very different task, and a much more complex thing to implement (well).

--

At the end of the day, no format is being blocked based on it being disliked or in competition to anyones favorite format. People may express that sentiment, as is their right, but it will not be taken into account by other developers to decide if its suitable for merging. The main reason something doesn't get merged is either one of two things - patchset needs work, and/or no developer is available (or interested) in dealing with it.

[rwill]

Originally Posted by nevcairiel

No where did I mention the license. Everyone relying on the exact same codebase to do decoding also doesn't debunk the competition I mentioned.

Yes, you did not mention the license, you did mention the "open-source multimedia ecosystem" though. And one wrapped decoder is better than none.

Originally Posted by nevcairiel

We are talking about decoders here. Encoding is a very different task, and a much more complex thing to implement (well).

Well decoding seems to be a complex thing too because last time I checked the FFmpeg H.262 and H.264 decoders failed in decoding conformance streams correctly.

HEVC support is much better but they are "wrapping" OpenHEVC there?

[nevcairiel]

OpenHEVC was/is a fork of libavcodec, which then got re-integrated into ffmpeg eventually. It does not require any external dependencies to work.

[FranceBB]

Originally Posted by nevcairiel

I would recommend those "professional" users deriving revenue from FFmpeg to chip in and further such a project. FFmpeg does not make money, but developers need to eat.

Not FFMpeg but open source in general and believe me, they do. Think about AVC Intra classes in x264, if it wasn't for NRK funding the development, we wouldn't have those.
Same goes for DolbyE and DolbyED2 decoders written by Nicolas Gaullier who works for a French broadcasting company, Sky/Comcast funding x265 development with Multicoreware and I could go on, but there are lots of TVs helping out as they can in open source development. Even me, in my little, I try to contribute as much as I can to the community.

Originally Posted by birdie

I'm pretty sure if companies behind VVC actually cared, they'd sponsor someone to get this code merged. Darn.

Easier said than done.
I'm still trying to get the code for the DolbyED2 Atmos decoder merged to the main FFMpeg codebase and I begged the mailing list to no avail.
If someone has the power to merge, please merge: https://trac.ffmpeg.org/ticket/9864

[MartinEesmaa]

I updated libvvenc codes for new vvenc patch:
https://github.com/MartinEesmaa/FFmp...6f2f89c3462381

Original patches:
https://patchwork.ffmpeg.org/project...t/?series=7922

Builds of latest FFmpeg VVCEasy:
https://github.com/MartinEesmaa/VVCE...-martin-eesmaa

- Martin Eesmaa

[birdie]

A patch series to support VVC encoding/decoding in ffmpeg via vvenc/vvdec has again been resubmitted. There are minor fixes here and there:

This patch set adds H266/VVC support.
This includes parsing, muxing, demuxing, decoding and encoding.
Decoding is done using the external library VVdeC
(https://github.com/fraunhoferhhi/vvdec.git) and can be enabled with
--enable-libvvdec.
Encoding is done using the external library VVenC
(https://github.com/fraunhoferhhi/vvenc.git) and can be enabled with
--enable-libvvenc.

Changes since v4:

General:
    - Add original author to patches 01-03
    - Add co-author to patches 04 and 05

PATCH 02/10 configure / cbs_h266_syntax_template
    - bugfix: add cbs_h266, cbs_h266_select (moved from patch 03/10)
    - bugfix: parsing multilayer in cbs_h265_syntax_template

PATCH 03/10 libavcodec/vvc_parser.c
    - bugfix: check startcode length (3 or 4 byte) to 
              find correct frame end position

PATCH 06/10 libavformat/movenc.c
    - merge movenc.c with upstream

PATCH 07/10 libavcodec/libvvdec.c
    - bugfix: set correct avframe pts for decoded picture
    - bugfix: print additional info if decoding fails

Nuo Mi (3):
  avcodec: add enum types for H266/VVC
  avcodec: add cbs for H266/VVC
  avcodec: add bitstream parser for H266/VVC

Thomas Siedel (7):
  avcodec: add MP4 to annexb support for H266/VVC
  avformat: add demuxer and probe support for H266/VVC
  avformat: add muxer support for H266/VVC
  avcodec: add external decoder libvvdec for H266/VVC
  avcodec: add external encoder libvvenc for H266/VVC
  avformat: add ts stream types for H266/VVC
  avcodec: increase minor version for H266/VVC

[FranceBB]

Yep.
Looks like Thomas Siedel from Spin Digital is trying again to get it approved and merged to master and at this point I really hope it is.

[FranceBB]

Hey guys,
there's gonna be a webinar with the multicoreware guys about x266, at what point the project is etc.
I strongly encourage everyone to participate.

https://us02web.zoom.us/webinar/regi...SDKIvWPmw2jUxw

It's gonna be January 31st at 5PM UK Time.

[Jamaika]

Decoder VVC
https://github.com/ffvvc/FFmpeg

[Jamaika]

VVCSoftware_VTM 19.2-06a9a140
https://www.sendspace.com/file/jmqw85

[birdie]

I've discovered a large library of uncompressed 4K videos under the Creative Commons Attribution 4.0 International License from the European Space Observatory. You can get them here. Will try to compress them visually losslessly using x264, x265 and VVENC and see who wins.

The codec used is hqx (CHQX / 0x58514843), yuv422p16le(10 bpc) - no idea what it is, but it must be uncompressed.

Grass Valley HQX, as an intermediate codec, is specifically designed for editing and post-production: Can withstand re-encoding without significant quality loss. Is built for high speed and low-CPU usage. Allows accurate inter-frame cuts (not grouped around keyframes). Has well-defined chroma characteristics. It is available for both Windows and Macintosh platforms, handles many different video resolutions up to and including 8K (DCI), incorporates an alpha channel for graphics handling, and is available in 8-bit and 10-bit versions. The Grass Valley Codec Pack includes/allows you to import and export video files that use the Grass Valley HQ, Grass Valley HQX, Grass Valley Lossless, and Grass Valley DV codecs.

[FranceBB]

Originally Posted by birdie

The codec used is hqx (CHQX / 0x58514843), yuv422p16le(10 bpc) - no idea what it is, but it must be uncompressed.

Nope.
The GV codec has three profiles: HQ, HQX, Lossless.
In this case they used the HQX profile which is lossy, so it's not lossless nor uncompressed, it's compressed and lossy.
It's a very good mezzanine file, though, generally used for those who need to edit or indeed store it as TX Master for playout in Grass Valley based systems. It's basically the Grass Valley's rival of AVID's DNxHR.

[birdie]

Originally Posted by FranceBB

Nope.
The GV codec has three profiles: HQ, HQX, Lossless.
In this case they used the HQX profile which is lossy, so it's not lossless nor uncompressed, it's compressed and lossy.
It's a very good mezzanine file, though, generally used for those who need to edit or indeed store it as TX Master for playout in Grass Valley based systems. It's basically the Grass Valley's rival of AVID's DNxHR.

I've not noticed any perceivable compression artifacts, the bitrate is outright insane, for the file that I downloaded, it's 1 332 878 kb/s, which surely looks like compression, if there is any, is minimal.

4K Bluray has around 75Mbps bitrate - files from the observatory are ~ 18 times denser.