Versatile Video Coding (VVC) / H.266: HEVC successor [Archive] - Page 8

hajj_3

17th December 2020, 13:12

AV1 threw out lots of baby with the bathwater in trying for a total do-over of codec development, instead of just building off of VP9, because it ended up so different. (Not the VP9 was any peach.) Other AV1 implementations have tons of freedom to do much better, which is both better and unfortunate. A future AV2 will have the benefit of whatever portion of aom's code and optimizations are reusable.

Some of the nice things about a future AV2 codec are:

1. lots of patents will have expired.
2. sisvel published a list of patents they claim that av1 violates therefore av2 can be developed to make sure that those aren't violated.
3. unifiedpatents.com have been successfully invalidating some av1 and hevc patents which means that some of those patents will be able to be used.

I do hope that AV2 isn't ratified within 5yrs of AV1 though, otherwise it might make a difficult for AV1 to become popular.

Jamaika

6th January 2021, 22:07

Untested info
No video player can show this. Full compatibility can be expected in a year's time!

mp4box.exe -new -info -add import.vvc output.mp4

Codec VVC in conteiner mp4.

Track Importing HEVC - Width 1280 Height 720 FPS 25000/1000 SAR 0/0
VVC Import results: 55 samples (144 NALUs) - 55 SEI - 3 IDR
VVC Stream uses forward prediction - stream CTS offset: 55 frames
VVC Max NALU size is 63768 - stream could be optimized by setting nal_length=2
# Movie Info - 1 track - TimeScale 600
Duration 00:00:00.000 (recomputed 00:00:04.240)
Fragmented: no
Major Brand iso4 - version 1 - compatible brands: iso4
Created: GMT Wed Jan 6 21:02:22 2021

# Track 1 Info - ID 1 - TimeScale 25000
Media Duration 00:00:00.000 (recomputed 00:00:02.200)
Track has 1 edit lists: track duration is 00:00:04.240
Media Info: Language "Undetermined (und)" - Type "vide:vvc1" - 55 samples
Visual Sample Entry Info: width=1280 height=720 (depth=24 bits)
Visual Track layout: x=0 y=0 width=1280 height=720
VVC Video - Visual Size 1280 x 720
VVC Info: Profile 1 @ Level 64 - Chroma Format YUV 4:2:0

NAL Unit length bits: 32
Parameter Sets: 1 SPS 1 PPS
Bit Depth 8 - 5 temporal layers
SPS#1 hash: CD7EE785B89AF3327C233B50CA28C755D751EAFA
PPS#1 hash: FF54700DA8418365C4BB4330DEE1EF986AA37AAC

RFC6381 Codec Parameters: vvc1.1.L64
Average GOP length: 27 samples
Max sample duration: 0 / 25000

0.500 secs Interleaving

FranceBB

7th January 2021, 12:30

Sweet, so it's possible to mux in mp4 as well and not just mkv. Good.
Apart from the container, is there any player at all (beside the DecoderApp.exe) that can play VVC files?
And last but not least: any news from Multicoreware about x266?

quietvoid

7th January 2021, 15:17

There seems to be a x266 repo on MCW's Bitbucket since October or something, but it's still private.

FranceBB

8th January 2021, 13:11

There seems to be a x266 repo on MCW's Bitbucket since October or something, but it's still private.

I see! So they're waiting to get things going before opening to open source. Well, it kinda makes sense, I think...

birdie

9th January 2021, 10:35

ffmpeg hasn't even begun adding support for VVC. That looks weird, the codec uptake seems to be extremely slow.

Jamaika

9th January 2021, 11:18

What is this talking about?
GPAC did what it could. I exerted pressure. There aren't resources for modifying new codecs in GPAC jpeg2000, avif, heif.
I remind you that in jpeg.org propaganda heif photos can contain and convert to HTJPEG2000. It is fantasy right now. There are no funds or there are paid programs somewhere.
https://jpeg.org/jpeg2000/index.html
JPEGXS will not come out free this year. It was supposed to be the successor to jpeg2000.
https://www.iis.fraunhofer.de/en/ff/amm/content-production/jpegxs.html
https://www.intopix.com/ffmpeg
Google codecs are for google and the rest don't count. The rest as competition will not support google.
The mpeg5 container had new features mainly anti-piracy in the network under vvc, evc codecs. The strange thing is we have mp4 and matroska from vvc. SONY is sure to be happy as he fought with matroska.
https://github.com/MPEGGroup/isobmff
The creators of m-ad-s laughed at me with the answer. Write better, create your own library, but they advertise google codec aom.
https://github.com/m-ab-s/media-autobuild_suite/commits/master

Jamaika

9th January 2021, 11:28

x266 Open-Source Encoder for VVC by MulticoreWare
https://ottverse.com/x266-vvc-multicoreware-opensource-encoder/
If you have any questions for the MulticoreWare team, please get in touch with Shivakumar Narayanan at x266@multicorewareinc.com.

Jamaika

9th January 2021, 11:33

Untested
Main Features
Spliting video by scenes for parallel encoding because AV1 encoders are currently not good at multithreading, encoding is limited to single or couple of threads at the same time.

PySceneDetect used for spliting video by scenes and running multiple encoders.
Fastest way to encode AV1 without losing quality, as fast as many CPU cores you have :).
Target Quality mode. Targeting end result reference visual quality. VMAF used as substructure
Resuming encoding without loss of encoded progress.
Simple and clean console look.
Automatic detection of the number of workers the host can handle.
Building encoding queue with bigger files first, minimizing waiting for the last scene to encode.
Both video and audio transcoding with FFmpeg.
Logging of progress of all encoders.
Install
Prerequisites:

Install Python3
When installing under Windows, select the option add Python to PATH in the installer
Install FFmpeg
Recommended to install vapoursynth with lsmash for faster and better processing
Encoder of choice:

Install AOMENC
Install rav1e
Install SVT-AV1
Install SVT-VP9
Install vpx VP9, VP8 encoding
Install VTM VVC encoding test model
Optional :

Vapoursynth
ffms2
lsmash
mkvmerge
If someone compiles, let him praise it. :)
https://github.com/master-of-zen/Av1an

FranceBB

9th January 2021, 13:40

x266 Open-Source Encoder for VVC by MulticoreWare
https://ottverse.com/x266-vvc-multicoreware-opensource-encoder/

Well, it really just says:

For the larger open source community, it will be made available when we have reached critical mass required for its first release. As we near readiness, we will update here on how you can get hold of it.

so, again, they're working on it, but we can't see it yet, so all we have is the encoder_app.exe reference encoder

Jamaika

9th January 2021, 13:52

so, again, they're working on it, but we can't see it yet, so all we have is the encoder_app.exe reference encoder
There is actually something, but I don't know what it is.
For sure gcc won't compile.
What language will it be in? It is not known.
The creator dreams of x266, but currently works in vvc.
I don't know what single file x266.cpp files are. There is probably some assembler.
https://github.com/chenm001/x266
https://github.com/fraunhoferhhi/vvdec/commits?author=chenm001
The ?266's RISC-V NON-Standard Vector Extension ONLY compile and disassembly by ?266 project modified llvm suite. You may build bu yourself or please contract to get prebuilt Centos-6 binary package.

FranceBB

9th January 2021, 14:08

Well, I contacted Multicoreware anyway with my Sky email, asking them for some info.
Let's see if they reply.

ksec

9th January 2021, 20:15

ffmpeg hasn't even begun adding support for VVC. That looks weird, the codec uptake seems to be extremely slow.

The spec is barely out. Finalised last August, but profiles were still not set then. VTM Reference Software was barely catching up in October and VTM 11.0 only released two days ago. Conformance test wasn't finalised either.

I mean normally these sort of "cleaning up" takes up to a year. So I think they are doing well so far.

MeteorRain

10th January 2021, 21:24

Cutting edge technologies take time to be implemented.
22.2 channel aac audio spec was released years ago, and we barely just had a preliminary support in the latest version, implemented by jeeb.
Sometimes you have to be patient ;)

Jamaika

14th January 2021, 03:12

ffplay's humble beginnings
https://github.com/FFmpeg/FFmpeg/commit/ebdd33086adc9899959840b616480c42c80653ce#diff-8d19749c083bc0060b37a7e4e22171de76da06f51836f41f96ee630d53c30d3f

FranceBB

14th January 2021, 09:16

ffplay's humble beginnings
https://github.com/FFmpeg/FFmpeg/commit/ebdd33086adc9899959840b616480c42c80653ce#diff-8d19749c083bc0060b37a7e4e22171de76da06f51836f41f96ee630d53c30d3f

That is indeed something!
Having a decoder is basically the first thing you want when supporting a codec, so those commits are more than welcomed. :)

FranceBB

14th January 2021, 12:43

https://github.com/fraunhoferhhi/vvdec
https://github.com/fraunhoferhhi/vvenc
plus my fix
I have added codecs and compilation issues in my threads

I'm clearly doing something wrong.

Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19042.
The C compiler identification is MSVC 19.27.29112.0
The CXX compiler identification is MSVC 19.27.29112.0
Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe
Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe - works
Detecting C compiler ABI info
Detecting C compiler ABI info - done
Detecting C compile features
Detecting C compile features - done
Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe
Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe - works
Detecting CXX compiler ABI info
Detecting CXX compiler ABI info - done
Detecting CXX compile features
Detecting CXX compile features - done
CMAKE_MODULE_PATH: updating module path to: C:/Users/Frank/Desktop/vvenc-master/cmake/modules
Looking for pthread.h
Looking for pthread.h - not found
Found Threads: TRUE
Configuring done
Generating done

https://i.imgur.com/XdXGsCB.png
https://i.imgur.com/2wdONHk.png
https://i.imgur.com/AxLcvTC.png
https://i.imgur.com/NPgTz4m.png
https://i.imgur.com/AP5xg3z.png

Jamaika

16th January 2021, 23:29

New codecs VVC
https://www.sendspace.com/file/09xyse

FranceBB

16th January 2021, 23:41

New codecs VVC
https://www.sendspace.com/file/09xyse

Thank you for building it for me! I'll test it on Monday as soon as I'll get back to work! :D

Jamaika

17th January 2021, 10:08

The Streaming Codec Landscape in 2021 (https://www.wowza.com/blog/streaming-codec-landscape-2021)
What Happened in 2020?
This past year saw the launch of two MPEG codecs — VVC and Low Complexity Enhancement Video Coding (LCEVC) — and the continued development of Essential Video Coding (EVC). You can read more about VVC’s technical background here. I reviewed Fraunhofer’s VVC encoder and decoder, respectively called VVenC and VVdeC, for Streaming Media Magazine here. As Table 2 shows, VCVenC proved 39% more efficient than x265 in terms of quality, but it took 10x longer to encode, and it was 3.7x slower during decode.

Regarding royalties, the intellectual property that was contributed to VVC came from many parties, and VVC will be royalty-bearing, although we probably won’t know what the royalty policy will look like until mid-2021 or later.

benwaggoner

17th January 2021, 23:40

Jamaika

18th January 2021, 08:09

An error in the quoted statement here - LCEVC isn't a codec. It's a standardized postprocessing with metadata technology that sits on top of codecs. It could be applied to H.264, HEVC, and/or VVC.
Is LCEVC mpeg5 paid? I don't see free libraries, and the final standard was in October 2020.
https://www.lcevc.org/
Weird. nothing is said about the new AV2 codec.
https://www.programmersought.com/images/110/fcf500c777166b3532682cd3f536ec0e.png

hajj_3

18th January 2021, 11:19

Is LCEVC mpeg5 paid? I don't see free libraries, and the final standard was in October 2020.
https://www.lcevc.org/
Weird. nothing is said about the new AV2 codec.
https://www.programmersought.com/images/110/fcf500c777166b3532682cd3f536ec0e.png
Interestingly that image from ateme shows that AV2 will be ratified in ~2026. I haven't seen a timeline for AV2 before. I'm glad they aren't going to try and release a new one every 3 years which i think was the original intention. This way AV1 will have time to become popular. Also nearly all the h264 patents will have expired by then. I'm sure that some of the patents that Sisvel claim AV1 violates will have expired by then too.

ksec

18th January 2021, 19:40

Interestingly that image from ateme shows that AV2 will be ratified in ~2026. I haven't seen a timeline for AV2 before. I'm glad they aren't going to try and release a new one every 3 years which i think was the original intention. This way AV1 will have time to become popular. Also nearly all the h264 patents will have expired by then. I'm sure that some of the patents that Sisvel claim AV1 violates will have expired by then too.

Giving too much credit for Google or Alliance Open Media, their original goal was new codec every "2" years. /s

Speaking of H264 patents, I am eagerly waiting for the patent free EVC baseline to test. Which is pretty much all H.264 expired or soon to be expired patents.

benwaggoner

19th January 2021, 18:14

Giving too much credit for Google or Alliance Open Media, their original goal was new codec every "2" years. /s

Speaking of H264 patents, I am eagerly waiting for the patent free EVC baseline to test. Which is pretty much all H.264 expired or soon to be expired patents.
There are certainly people interested in getting AV2 standardized before 2026. VVC's technical superiority over AV1 is pretty substantial. And essential patents are expiring every year, so there are plenty of techniques that couldn't be implemented in AV1 that are already available for AV2.

hajj_3

27th January 2021, 22:02

BAD NEWS GUYS... The Media Coding Industry Forum (MC-IF) whose members make up the majority of VVC patents have been unable to achieve concencus in picking a patent licensing pool administrator so there will be 2 PATENT POOLS!!!!! One of the pools is 'Access Advance', the other is 'MPEG-LA'. Source: https://a7dce6fd-e8f0-45f7-b0b0-255c5c9a28e1.filesusr.com/ugd/0c1418_a055f01edeba4e0abe7a1c0482debab7.pdf

Clearly they haven't learnt their lesson from the HEVC patent licensing problems!

ksec

27th January 2021, 22:47

LOL, It is Access Advance ( Previously HEVC Advance ) and MPEG LA again. I am starting to think there are some bad blood between the two.

And even Google is part of Access Advance Licensor.

Despite the public hatred against MPEG-LA, they have been the one offering cheaper HEVC licensing.

FranceBB

28th January 2021, 09:04

Yeah... this is not good news for general availability...
Let's see how it's gonna play out...

ksec

28th January 2021, 15:27

Yeah... this is not good news for general availability...
Let's see how it's gonna play out...

Well We are only 1 year away from 3nm. Whatever cost, power or other obstacles AV1 had with mobile hardware decoder will diminish with the advance of computing and design.

excellentswordfight

28th January 2021, 19:42

Well We are only 1 year away from 3nm. Whatever cost, power or other obstacles AV1 had with mobile hardware decoder will diminish with the advance of computing and design.
TSMC 5nm node has higher transistor cost then 7nm, and 3nm will probably follow that trend, so not sure how 3nm will help if the reason for the lack of av1 decoders was cause of roi on transistor budget.

Edit. initially ofc, but the trend is that transistor cost isnt drastically reduced anymore from die shrinks.

ksec

28th January 2021, 20:10

TSMC 5nm node has higher transistor cost then 7nm, and 3nm will probably follow that trend, so not sure how 3nm will help if the reason for the lack of av1 decoders was cause of roi on transistor budget.

Edit. initially ofc, but the trend is that transistor cost isnt drastically reduced anymore from die shrinks.

If you are comparing non EUV 7nm aka N7, then barely. Otherwise no. ( EUV 7nm and EUV 5nm. ) Wafer Price are on the rise and per transistor cost isn't yet. It is not hitting the same cost reduction but per transistor cost has never risen per generation during their time of introduction.

Design Cost is the main factor in cost increase, but that bare little relevance to unit cost without specific volume accounted for.

5nm and 3nm are relevant not only because of cost but Die Space and Power Usage. Where in Mobile devices you are expecting a budget of sub 100mW for Decoding. In Laptop you have the luxury of budget increase by 10x / 1W if not more on a GPU.

hajj_3

28th January 2021, 20:56

MPEG-LA has released a press release about their VVC patent pool: https://www.mpegla.com/wp-content/uploads/FINAL-VVC-PID-PrsRls-2021-01-27.pdf

excellentswordfight

28th January 2021, 22:51

If you are comparing non EUV 7nm aka N7, then barely. Otherwise no. ( EUV 7nm and EUV 5nm. ) Wafer Price are on the rise and per transistor cost isn't yet. It is not hitting the same cost reduction but per transistor cost has never risen per generation during their time of introduction.

Design Cost is the main factor in cost increase, but that bare little relevance to unit cost without specific volume accounted for.

5nm and 3nm are relevant not only because of cost but Die Space and Power Usage. Where in Mobile devices you are expecting a budget of sub 100mW for Decoding. In Laptop you have the luxury of budget increase by 10x / 1W if not more on a GPU.
I was talking about the non euv N7 node, as its rumored that 5nm wafer price was 1.8x higher, while comparing something like apples a12 to a14, density ”only” increased by 1.5x.

But thats intresting, but to me it sounds a bit odd... ”We cannot make an sub 100mW decoder on current silicon so lets use software decoding instead!” :)

I though that the key factor was that they didnt wanna waste money on silicon for an feature that might not see wide adoption, and that av1 needed quite a bit of silicon budget.

benwaggoner

29th January 2021, 01:25

I though that the key factor was that they didnt wanna waste money on silicon for an feature that might not see wide adoption, and that av1 needed quite a bit of silicon budget.
I've heard estimates that an AV1 decoder can take twice the silicon budget of a VVC decoder. That extra cost of AV1 HW support is likely higher than any savings from not having a per-decoder licensing fee.

GTPVHD

30th January 2021, 07:52

https://www.streamingmedia.com/Articles/News/Online-Video-News/VVC-Patent-Pools-And-Then-There-Were-Two-144949.aspx

VVC is dead on arrival as many have predicted.

Funky080900

30th January 2021, 10:44

https://www.streamingmedia.com/Articles/News/Online-Video-News/VVC-Patent-Pools-And-Then-There-Were-Two-144949.aspx

VVC is dead on arrival as many have predicted.

“Note that two pools aren't necessarily a bad thing; many successful technology standards involve multiple pools”

The article doesn't say that VVC is dead only that “potential licensees won't know total VVC royalty costs until both pools release their royalty terms”

ksec

30th January 2021, 18:16

“Note that two pools aren't necessarily a bad thing; many successful technology standards involve multiple pools”

The article doesn't say that VVC is dead only that “potential licensees won't know total VVC royalty costs until both pools release their royalty terms”

Access Advance already have some preliminary announcement [1] with regards to pricing a long time ago. Considering they are making it a HEVC + VVC pool, you can bet it won't be cheaper than HEVC. It will be at best, the same as current HEVC pricing but also includes VVC.

And again, ValosMedia, or specifically Qualcomm. Given all other members of Valos are already in either MPEG LA or Access, Qualcomm are not in any the pools ( yet ).

[1] https://accessadvance.com/hevc-advance-releases-draft-vvc-licensing-program-overview-includes-a-joint-vvc-and-hevc-license/

benwaggoner

31st January 2021, 05:10

foxyshadis

2nd February 2021, 15:06

I've heard estimates that an AV1 decoder can take twice the silicon budget of a VVC decoder. That extra cost of AV1 HW support is likely higher than any savings from not having a per-decoder licensing fee.

I wonder how much of the spec they implement, since nothing has trickled down for consumers to test yet. AV1 bucks the general trend of having a very stripped-down baseline MPEG profile that hardware can implement; its baseline includes most of the more complex tools and colorspaces that hardware pretends don't exist and therefore the state of the art never really budges as much as you'd expect.

benwaggoner

2nd February 2021, 18:14

I wonder how much of the spec they implement, since nothing has trickled down for consumers to test yet. AV1 bucks the general trend of having a very stripped-down baseline MPEG profile that hardware can implement; its baseline includes most of the more complex tools and colorspaces that hardware pretends don't exist and therefore the state of the art never really budges as much as you'd expect.
Yeah, it's a tradeoff. It's awesome we can do 444 10-bit in the baseline decoder, but that definitely bumps up complexity. That alone requires 2.5x the sample fill rate of 420 8-bit.

MPEG has been great at making very thoughtful choices to optimize for HW decoder complexity, but AV1 didn't get nearly the breadth of input about HW implementations, and efforts were very much focus on SW decoders for a long time.

hajj_3

17th February 2021, 01:32

VTM 12.0 is out.

LigH

18th February 2021, 09:28

New upload: VTM Encoder Version 12.0 [Windows][GCC 10.2.0][64 bit] (http://www.mediafire.com/file/6ithw8upfn331et/VVCSoftware_VTM-12.0_091d3bb5.7z/file) 091d3bb5

birdie

30th April 2021, 12:28

vvenc developers have no plans (https://github.com/fraunhoferhhi/vvenc/issues/59) for supporting lossless compression or anything other than YUV420. Bummer.

FranceBB

30th April 2021, 15:15

Oh that's disappointing...
I guess we will have to wait for Multicoreware to release x266 then...
Any news on that regard?

Marsu42

13th May 2021, 10:49

vvenc developers have no plans (https://github.com/fraunhoferhhi/vvenc/issues/59) for supporting lossless compression or anything other than YUV420. Bummer.

Actually, it says "The support is not planned in the near future." - which makes sense because they would want to get basic vvc support ready first, and not code/ optimize/ debug multiple bit depths or lossless at the same time.

Jamaika

14th May 2021, 10:27

New codec VVC 12.3 / VVENC 0.3.1 2021.05.14 + JVET_V0061 + JVET_V0106 + test files video
https://www.sendspace.com/file/aayebk

birdie

18th May 2021, 14:35

Actually, it says "The support is not planned in the near future." - which makes sense because they would want to get basic vvc support ready first, and not code/ optimize/ debug multiple bit depths or lossless at the same time.

There are relatively few users of lossless and they don't care much e.g. about speed, so I'd be glad if they just implemented it based on the algos already in place and continue to optimize the code base for lossy encoding.

benwaggoner

18th May 2021, 21:24

There are relatively few users of lossless and they don't care much e.g. about speed, so I'd be glad if they just implemented it based on the algos already in place and continue to optimize the code base for lossy encoding.
Yeah, lossless mode is quite different and also less complex than lossy encoding. https://x265.readthedocs.io/en/master/lossless.html

A nice feature of HEVC and VVC is the ability to use lossless coding units selectively in a lossy stream. Lossless can compress stuff like a black/white checkerboard more efficiently than any frequency transform.