Versatile Video Coding (VVC) / H.266: HEVC successor

[birdie]

The SpinDigital latest/last VVC patch series allows ffmpeg to encode and decode (read play in mpv) VVC/H.266 but it's not complete yet.

For instance I couldn't mux an elemental H.266 stream with AAC audio into mkv but it worked for mp4 (or was it vice versa - I don't remember now).

Also, VVC decoding [using vvdec] requires a lot of CPU resources - it's comparable to software decoded AV1.

vvdec really could use more optimizations.

If anyone's interested I can upload/share my ffmpeg/mpv builds for Fedora 37 x86_64. Don't ask for your own distro build as I'm on vacation now and my laptop is not suitable for firing up VMs with your favourite distro, let alone building software.

[FranceBB]

It's your lucky day, Artem, 'cause I'm on Fedora xD

[Jamaika]

Originally Posted by birdie

For instance I couldn't mux an elemental H.266 stream with AAC audio into mkv but it worked for mp4 (or was it vice versa - I don't remember now).

Now I see. The funny thing now is that the unfinished decoder is on ffvvc. The MP4 decoder bass plugin for all VVC codecs was made by Martin. When it connects, muxer/demuxer should work thearetically.

[birdie]

Originally Posted by FranceBB

It's your lucky day, Artem, 'cause I'm on Fedora xD

mpv / ffmpeg with VVC decoding support for Fedora 37.

The installation instructions are in the archive.

sha256sum ffmpeg+mpv-with-vvc-decoding-support-f37.tar.xz 
c6fc12b058da084e2695a01e6ac595f5f57bb4c6bc2d3864c0e065531cd79dc0  ffmpeg+mpv-with-vvc-decoding-support-f37.tar.xz

[FranceBB]

Thank you!
Tested right up and it's actually working on my Fedora 37

[birdie]

Originally Posted by FranceBB

Thank you!
Tested right up and it's actually working on my Fedora 37

Could you test 4K 60fps video playback? Specifically this one? Other clips.

Can your rig play it without dropping frames? ;-) Press Shift + I too see the statistics - most importantly "Dropped frames".

My Core i5 6200U can barely handle a 1080p clip. Near 100% CPU use and almost zero headroom.

[Jamaika]

Test ffmpeg + vvc
ffmpeg.exe -i "iphone6s_4k.mov" -c:v libuvg266 -vb 3000k -c:a ac3 -ac 2 -ab 128k -s 1920x1080 -frames 100 -pix_fmt yuv420p output_uvg266.mts --> OK
ffmpeg.exe -i "iphone6s_4k.mov" -c:v libuvg266 -vb 3000k -c:a ac3 -ac 2 -ab 128k -s 1920x1080 -frames 100 -pix_fmt yuv420p output_uvg266.mov --> decoder ffvvc + vvc_mp4toannexb_bsf.c MartinEesmaa Error {no header}
ffmpeg.exe -i "iphone6s_4k.mov" -c:v libuvg266 -vb 3000k -c:a ac3 -ac 2 -ab 128k -s 1920x1080 -frames 100 -pix_fmt yuv420p output_uvg266.mkv --> no file

ffmpeg.exe -i "iphone6s_4k.mov" -c:v libxeve -vb 3000k -c:a ac3 -ac 2 -ab 128k -s 1920x1080 -frames 100 -pix_fmt yuv420p output_xeve.mts --> no file
ffmpeg.exe -i "iphone6s_4k.mov" -c:v libxeve -vb 3000k -c:a ac3 -ac 2 -ab 128k -s 1920x1080 -frames 100 -pix_fmt yuv420p output_xeve.mov --> OK
ffmpeg.exe -i "iphone6s_4k.mov" -c:v libxeve -vb 3000k -c:a ac3 -ac 2 -ab 128k -s 1920x1080 -frames 100 -pix_fmt yuv420p output_xeve.mkv --> no file

ffmpeg.exe -f avs3 -i "SFTI_1920x1080.bin" -c:v libx264 -vb 3000k -an -s 1920x1080 -pix_fmt yuv420p output_x264.mov --> error {no frames}
ffmpeg.exe -f vvc -i "NovosobornayaSquare_1920x1080.bin" -c:v libx264 -vb 3000k -an -s 1920x1080 -pix_fmt yuv420p output_x264.mov --> error {no file}
[vvc @ 0000025b0ee2b5f0] No start code is found.
[vvc @ 0000025b0ee2b5f0] Error splitting the input into NAL units.

[benwaggoner]

Originally Posted by birdie

Also, VVC decoding [using vvdec] requires a lot of CPU resources - it's comparable to software decoded AV1.

vvdec really could use more optimizations.

Honestly "comparable to SW AV1" is a pretty big accomplishment this early in VVC's life cycle, and speaks to a real good job of balancing decoder cost/complexity with compression efficiency gains.

AV1 is a bear to implement in hardware.

[hajj_3]

VeriSilicon delivered multi-format hardware video decoder Hantro VC9000D supporting 8K@120FPS VVC/H.266 to customers

source: https://www.verisilicon.com/en/Press.../HantroVC9000D

[FranceBB]

Originally Posted by birdie

Could you test 4K 60fps video playback?

Dude, you wanna kill my CPU? XD

Jokes aside, I run Fedora on my personal computer at home, so it's not really powerful like the Intel Xeon workstations I have at work (20c/40th) or the servers I have there (56c/112th), it's just a puny little i7 6700HQ with 4c/8th at 2.60GHz and no AVX-512 just AVX2 (yes, it could go higher like 3.5GHz single core or 3.2GHz all cores, but it would start throttling like crazy, so...)
I use it mainly to read my email, browse the internet, post random stuff on Doom9, Facebook, Linkedin, occasionally watch YouTube (mainly Sky News or Linus Tech Tips) etc.
I mean, all very basic stuff.


Anyway, FULL HD H.266 VVC playback, no problem, but with UHD 60p it was begging me to pull the plug as it just wanted to die at 95°C, all cores 100%.


TL;DR no, I cannot play UHD 60p H.266 VVC without dropping frames on a 4 core laptop (nor I expected to).

Originally Posted by benwaggoner

Honestly "comparable to SW AV1" is a pretty big accomplishment this early in VVC's life cycle

It's not comparable, by the way.
In my test AV1 is lighter than H.266 VVC, probably because they had a lot of time to optimize software decoding.

[birdie]

Originally Posted by FranceBB

Dude, you wanna kill my CPU? XD

Jokes aside, I run Fedora on my personal computer at home, so it's not really powerful like the Intel Xeon workstations I have at work (20c/40th) or the servers I have there (56c/112th), it's just a puny little i7 6700HQ with 4c/8th at 2.60GHz and no AVX-512 just AVX2 (yes, it could go higher like 3.5GHz single core or 3.2GHz all cores, but it would start throttling like crazy, so...)
I use it mainly to read my email, browse the internet, post random stuff on Doom9, Facebook, Linkedin, occasionally watch YouTube (mainly Sky News or Linus Tech Tips) etc.
I mean, all very basic stuff.


Anyway, FULL HD H.266 VVC playback, no problem, but with UHD 60p it was begging me to pull the plug as it just wanted to die at 95°C, all cores 100%.


TL;DR no, I cannot play UHD 60p H.266 VVC without dropping frames on a 4 core laptop (nor I expected to).



It's not comparable, by the way.
In my test AV1 is lighter than H.266 VVC, probably because they had a lot of time to optimize software decoding.

LOL. OK ;-) Thanks.

Yeah, sorry, I stand corrected, currently software VVC decoding using VVdeC is considerably more CPU intensive than AV1 decoding.

Sorry for misleading people earlier. My Ryzen 7 5800X chewed VVC 4K 60fps no problem and I didn't quite pay attention to power consumption or CPU usage. It must have been a lot higher than AV1.

[birdie]

From the lead Fraunhofer HHI VVC developer, Adam Wieckowski :

AV1 decoding is really much more optimized. The dav1d decoder has more lines of assembly code than Linux kernel. We don't have the knowledge or the resources to put in this kind of work.

To be fair, the best way to improve vvdec right now would be to rewrite it, because the performance is bound by the software structure that we inherited from VTM, even if heavily optimized. And we are not going to do this. We are still working on minor improvements, but don't expect really big leaps for vvdec anytime soon. ffvvc might someday be more efficient than vvdec, but by that time I expect HW support to be fairly broad.

The only bigger thing being worked on right now is the error resilience.

[FranceBB]

Originally Posted by birdie

My Ryzen 7 5800X chewed VVC 4K 60fps no problem

ehehehe that's an 8c/16th 3.8GHz (max 4.7GHz) desktop CPU from 2020, no wonder it managed to decode it unlike my 4c/8th 2.6GHz laptop CPU from 2015 :P

Even benchmark values here get to the point of ridiculous in favor of your desktop CPU xD Intel i7 6700HQ vs AMD Ryzen 7 5800X

[benwaggoner]

Sounds like we need the dav1d equivalent for VVC. Aomdec was similar to the Fraunhofer work here: optimizing the reference decoder, but not attempting to build a ground-up performance optimized decoder for real-world use.

[LigH]

Originally Posted by birdie

From the lead Fraunhofer HHI VVC developer, Adam Wieckowski :

Does this quote have a shareable source?

[birdie]

Originally Posted by LigH

Does this quote have a shareable source?

This.

[DTL]

Originally Posted by birdie

From the lead Fraunhofer HHI VVC developer, Adam Wieckowski :

Simple (and cheap to develop) C-reference implementation may be only for quality comparison. It will be truly free opensource working demo of a new codec. Github friendly. And to show the benefit from new version in compare with old products. For general industry it may be completely zero reason to invest into SIMD-optimized very hard to develop implementation for general purposes CPUs with SIMD co-processors just to make freeware version with no commertial benefit. For the quickly shrinking number of PC real users. The commertial developemnt is oriented into real market with real number of users to get money. May you can found ASIC for VVC decoding in some new smartphone in a few months later and it will outperform many massively-multicore expensive Xeons.

If codec will be accepted by industry it will be implemented in ASIC with all requirement performance for enduser setup (set-top-box or internal display decoder or a chip for handheld smartphone and so on).

[birdie]

VVenC 1.8 has been released.

Changes over v1.8.0-rc1

• deprecated obsolete chroma QP control mechanism (to be removed in the next release)
• removed deprecated parameters for partitioning control

Changes over v1.7.0

• vvencFFapp removed from default installation
• Speedups (~15% for faster/~5% for slower/~10% for other presets)
• Improved error messages
• Added temporal ALF APS prediction, closing the efficiency gap between single- and multi-threaded operation
• Extended BIM by extrapolating parameters to non MCTF-filtered frames
• Various fixes
• Usability improvements
• Various improvements

[MartinEesmaa]

I just want to let note, that I updated new FFmpeg master and MPV with VVCEasy new builds. I tried ffvvc in Windows, but it was error, so I switched from ffvvc to external VVC decoder of my own repository.

https://github.com/MartinEesmaa/VVCE...acc45942bb0420
https://github.com/MartinEesmaa/VVCE...eff0809530813e

- Martin Eesmaa

[Gravitator]

Всем привет!
Martin Eesmaa, help?

ffmpeg -i space_6s.mkv -f yuv4mpegpipe - | vvencapp --y4m -i - --fps 24000/1001 --preset fast -q 30 -o y4m.266

av_interleaved_write_frame(): Broken pipe
[out#0/yuv4mpegpipe @ 000001fdd23bac00] Error muxing a packet
[out#0/yuv4mpegpipe @ 000001fdd23bac00] Error writing trailer: Broken pipe

> sample