x266 VVC Encoder

[ksec]

The last webinar questions have been answered (http://bit.ly/3EB0Hdk).

I'm a little bit dumbfounded because seemingly some questions have been misunderstood, some others have very basic and uninformative answers.

Well I don’t think they misunderstood any question. I think they intentionally avoid answering it directly. It is just part of PR.

I would love to test x266. Both as still image like HEIC and video. At least both Mediatek and Qualcomm are on board with VVC. If Apple follows through that is potentially 95%+ of Mobile market.

[benwaggoner]

The last webinar questions have been answered (http://bit.ly/3EB0Hdk).

I'm a little bit dumbfounded because seemingly some questions have been misunderstood, some others have very basic and uninformative answers.
Is Ultraziq a rebranded/enhanced implementation of UHDKit?

[Blue_MiSfit]

It sure seems to be.

[kurkosdr]

The last webinar questions have been answered (http://bit.ly/3EB0Hdk).
So, we finally have a date: H2 2023. This means that everyone will have to withhold complaints about when x266 will be released to 1st Jan 2024.

BTW I also think some of the answers are being evasive, but people need to understand that corporations can share only what they can share. But they shared a release date, so that's good.

At least both Mediatek and Qualcomm are on board with VVC. If Apple follows through that is potentially 95%+ of Mobile market.
Out of curiosity, what will be the case for this? Firefox and Chrome (which are the majority of the browser market) have decided to not adopt any ISO/ITU standard beyond H.264, so that rules web video out, and broadcast and Blu-Ray UHD content belongs to HEVC (too much of an installed base to revert now, imagine if MPEG4 ASP had become the standard for FullHD, that's what essentially happened with UHD and HEVC), which means VVC-encoded videos will be unplayable in the majority of UHD televisions. My only guess is 8K UHD content, which is generally assumed to be VVC-encoded, but this raises the question if smartphones can encode 8K UHD content, in VVC.

[benwaggoner]

Out of curiosity, what will be the case for this? Firefox and Chrome (which are the majority of the browser market) have decided to not adopt any ISO/ITU standard beyond H.264, so that rules web video out, and broadcast and Blu-Ray UHD content belongs to HEVC (too much of an installed base to revert now, imagine if MPEG4 ASP had become the standard for FullHD, that's what essentially happened with UHD and HEVC), which means VVC-encoded videos will be unplayable in the majority of UHD televisions. My only guess is 8K UHD content, which is generally assumed to be VVC-encoded, but this raises the question if smartphones can encode 8K UHD content, in VVC.
Chrome added support for HEVC in the fall, if a system HEVC decoder is available. So it's plausible they'll do the same for VVC in a few year (and plausible they won't).

[kurkosdr]

Chrome added support for HEVC in the fall, if a system HEVC decoder is available.
So, Google decided to screw over Linux and Mozilla after initially joining them in their decision to not implement patent-encumbered formats newer than H.264. Why I am not surprised?

[benwaggoner]

So, Google decided to screw over Linux and Mozilla after initially joining them in their decision to not implement patent-encumbered formats newer than H.264. Why I am not surprised?
I think it is more a matter of prioritizing users over ideological purity. AV1 hasn't really caught on for HDR content, nor have practical benefits of it over HEVC been demonstrated. Everything but browsers can do HEVC these days, and the browser HDR market hasn't been big enough to justify spending 2-4x more time to encode AV1 HDR files for them. Allowing HEVC passthrough to system decoders doesn't require any patent engagement by Google, and adds compatibility with a whole lot of premium content that would otherwise not be available in a browser versus a standalone app.

And Linux systems certainly do support HEVC playback. There's not a build-in software player, but CPU/GPU HW HEVC decoders can certainly be accessed under lots of Linux distributions.

I can't speak to Mozilla's plan, but they certainly could add the "we won't have a decoder, but we'll pass HEVC on to one that exists."

Chrome and Firefox supported HEVC passthrough around a decade ago, as codec passthrough used to be done by default if a decoder was available. That functionality was explicitly blocked by the browsers later.

Supporting VVC in the same way will be quite straightforward once we start seeing PC systems with VVC decoders (I'd expect some PC CPU or GPUs with support to launch by late 2024.).

[ksec]

Out of curiosity, what will be the case for this? Firefox and Chrome (which are the majority of the browser market) have decided to not adopt any ISO/ITU standard beyond H.264, so that rules web video out, and broadcast and Blu-Ray UHD content belongs to HEVC (too much of an installed base to revert now, imagine if MPEG4 ASP had become the standard for FullHD, that's what essentially happened with UHD and HEVC), which means VVC-encoded videos will be unplayable in the majority of UHD televisions. My only guess is 8K UHD content, which is generally assumed to be VVC-encoded, but this raises the question if smartphones can encode 8K UHD content, in VVC.

Future TV broadcast standard like Brazil have chosen VVC. HDR and 4K ( It doesn't even need to be 8K ) Broadcast will also likely be using a new video codec standard. There are also other Streaming Services which doesn't rely on Web. Especially outside of EUR and North America. Even in 4K HDR Content VVC will still be much better than HEVC.

[FranceBB]

Allowing HEVC passthrough to system decoders doesn't require any patent engagement by Google, and adds compatibility with a whole lot of premium content that would otherwise not be available in a browser versus a standalone app.

Yep, that's exactly it, in fact I really don't see anything wrong with this approach. After all, if a user has an H.265 capable hardware decoder in his GPU, it means that the GPU manufacturer already paid the royalty to be able to decode it (and therefore included it in the final GPU price that the user paid). In other words, given that the user has *theoretically* already paid the fee, having such a functionality artificially blocked inside the browser wouldn't make any sense, so I'm totally in favor of this approach. I, myself, have an H.265 capable GPU in terms of decoding and indeed chrome://gpu shows it:

https://i.imgur.com/8Twnv6E.png

and indeed if I try to play an H.265 stream it works like a charm:

https://i.imgur.com/Nn9BkfD.png


I'm using the DASH reference player: https://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html to test with the following stream: https://dash.akamaized.net/dash264/TestCasesUHD/2a/5/MultiRate.mpd

https://i.imgur.com/8leWwei.png

[kurkosdr]

And Linux systems certainly do support HEVC playback. There's not a build-in software player, but CPU/GPU HW HEVC decoders can certainly be accessed under lots of Linux distributions.
No, not consistently at least. HEVC hardware decoding acceleration on Desktop Linux depends on the driver used, since GPU vendors are unclear on whether the royalty is paid on the hardware sale or on the driver download, which scares away open-source driver authors. Fedora had to disable hardware decoding of patent-encumbered formats for that very reason. And then there is the problem of perfectly functional GPUs which however don't have HEVC hardware decoding acceleration.

Google decided to screw Desktop Linux users over and treat them as lesser Chrome users after promising they wouldn't do that (in order to drum up support for VP9). Why am I not surprised? The next step is Mozilla getting blamed for not treating Desktop Linux users as lesser Firefox users too.

Remember when the web was universally accessible and not subject to whether you've gone through the MPEG LA and Access Advance tollbooths? I do.

[birdie]

Remember when the web was universally accessible and not subject to whether you've gone through the MPEG LA and Access Advance tollbooths? I do.

I do, the only video format at the time being numb 8bit palette GIF.

Neither APNG, nor MNG ever took off.

[rwill]

I do, the only video format at the time being numb 8bit palette GIF.

Neither APNG, nor MNG ever took off.

That was because GIF being a completely open format and totally not patent encumbered was a perfect match for the Web of the People. Right comrades ?

*edit*
So whats going on with x266? Cant be that hard to write a H.266 encoder. Looking at their feature roadmap I think it would take me just 2-3 man-months to get to a working release....

[kurkosdr]

That was because GIF being a completely open format and totally not patent encumbered was a perfect match for the Web of the People.
I kind of expected this comment, despite being plain wrong: The GIF compression patent was a patent ambush. In other words, the patent holder came to assert and collect after the format had been already widely implemented. The royalty-free PNG was invented shortly afterwards precisely because the web was always meant to be universally accessible, but inertia was king (as always). After the GIF compression patent expired, universal accessibility was restored. Then YouTube made the Flash Player plugin and H.264 mandatory. Even today, the legacy of Flash Player lives on because H.264 was implemented at the browser level in order to encourage websites to move on from Flash Player. But what's the point of HEVC on the web? A slightly worse format than AV1 that is patent-encumbered? And has the patent mess of HEVC been sorted out yet or there are still multiple patent pools that you need to negotiate with? And how many patent pools exactly are essential to HEVC this week? What a mess. But again, I am not surprised Google proved to be a backstabbing company. They are an ad agency masquerading as a tech company after all.

[FranceBB]

Fedora had to disable hardware decoding of patent-encumbered formats for that very reason. And then there is the problem of perfectly functional GPUs which however don't have HEVC hardware decoding acceleration.


This is a video shot with my Google Pixel 6 Pro in H.265 the other day:


General
Complete name : /home/FranceBB/Downloads/PXL_20230309_182128484.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 102 MiB
Duration : 13 s 592 ms
Overall bit rate : 63.1 Mb/s
Encoded date : UTC 2023-03-09 18:21:43
Tagged date : UTC 2023-03-09 18:21:43
xyz : +45.4339+9.2417/

Video
ID : 3
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L6.1@Main
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 13 s 589 ms
Bit rate : 62.8 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 60.000 FPS
Minimum frame rate : 45.662 FPS
Maximum frame rate : 93.652 FPS
Real frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.126
Stream size : 102 MiB (100%)
Title : VideoHandle
Language : English
Encoded date : UTC 2023-03-09 18:21:43
Tagged date : UTC 2023-03-09 18:21:43
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : hvcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 13 s 592 ms
Bit rate mode : Constant
Bit rate : 192 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 319 KiB (0%)
Title : SoundHandle
Language : English
Encoded date : UTC 2023-03-09 18:21:43
Tagged date : UTC 2023-03-09 18:21:43

Other
Type : meta
Duration : 13 s 589 ms
Bit rate mode : Variable



This is me playing it with hardware decoding through MPV on my Fedora 37 x64: https://i.imgur.com/YXrv2At.png


as you can see, being an 8bit yv12 (4:2:0 planar) Full Range BT709 60p UHD video, it's being hardware decoded by the GPU in nv12 using vaapi.


To do this, you need to add the following lines in mpv.conf

#Here we enable hardware decoding for everything
hwdec=auto
hwdec-codecs=all
vd-lavc-check-hw-profile=no

#Here we fallback to software if there's no hardware decoding
vd-lavc-software-fallback=yes

#Here we set the video sync (useful with GNOME)
video-sync=display-resample-desync


in case anyone needs it, here's my full mpv.conf that needs to be placed in /etc/mpv: Link (https://mega.nz/file/OYtW1ZgQ#7-LdOvps-E33NptK2JsryjsaC-bKgolf5FrCCi1srXo)


As far as chrome is concerned, unless you need to stay on the official Google Chrome, you can use chromium, in particular chromium-freeworld from RPM Fusion which still includes all the decoders which are not included in the standard version of chromium afaik.

https://github.com/rpmfusion/chromium-freeworld

just

sudo dnf install chromium-freeworld

and that's it.

[kurkosdr]

This is a video shot with my Google Pixel 6 Pro in H.265 the other day:




This is me playing it with hardware decoding through MPV on my Fedora 37 x64: https://i.imgur.com/YXrv2At.png


as you can see, being an 8bit yv12 (4:2:0 planar) Full Range BT709 60p UHD video, it's being hardware decoded by the GPU in nv12 using vaapi.


To do this, you need to add the following lines in mpv.conf

#Here we enable hardware decoding for everything
hwdec=auto
hwdec-codecs=all
vd-lavc-check-hw-profile=no

#Here we fallback to software if there's no hardware decoding
vd-lavc-software-fallback=yes

#Here we set the video sync (useful with GNOME)
video-sync=display-resample-desync


in case anyone needs it, here's my full mpv.conf that needs to be placed in /etc/mpv: Link (https://mega.nz/file/OYtW1ZgQ#7-LdOvps-E33NptK2JsryjsaC-bKgolf5FrCCi1srXo)


As far as chrome is concerned, unless you need to stay on the official Google Chrome, you can use chromium, in particular chromium-freeworld from RPM Fusion which still includes all the decoders which are not included in the standard version of chromium afaik.

https://github.com/rpmfusion/chromium-freeworld

just

sudo dnf install chromium-freeworld

and that's it.
Ok, how did you do this? Proprietary Nvidia or AMD drivers? Not everyone has an Nvidia or AMD GPU, most systems have Intel GPUs. Fedora with the default drivers can't decode HEVC even if the GPU has hardware acceleration:
https://www.reddit.com/r/Fedora/comments/xpt34f/fedora_37_drops_vaapi_accelerated_hardware_video/
This is due to GPU vendors' unwillingness to clarify whether the royalty is paid on the hardware sale or on the driver download.

Also, Chromium can't sync to Google Account anymore so it's useless for most people (another little bit of backstabbing by Google, surprise surprise).

Please note that while I don't hate HEVC or VVC per se, I hate those HEVC and VVC fanboys that try to downplay the massive IPR problems of those codecs (and how this could affect the universal accessibility of the web) by pointing to system codecs as the solution, conveniently ignoring the fact system codec support is inconsistent and hence not universal. And yes, HEVC and VVC do have massive IPR problems: Again, how many patent pools exactly are essential to HEVC this week? Same for VVC?

[FranceBB]

Ok, how did you do this? Proprietary Nvidia or AMD drivers?

I totally get the pain of using proprietary NVIDIA drivers on Linux, particularly 'cause for me NVIDIA drivers are almost always broken at every kernel update and the fact that NVIDIA isn't willing to open source a damn thing is pretty annoying as it left the Nouveau project guys in the dark, with GPUs running at the lowest possible clock speeds with no hope for re-clocking. Sure, the latest changes bring a bit of hope as they would allow for some GPUs to be reclocked, however it only works for newer GPUs so anyone with a 900 series like me will be left in the dark with either a buggy proprietary driver or a slow unusable open source one.

Leaving this sad chapter aside, that thing was actually done from my old 2016 era laptop, which has an Intel i7 6700HQ (which has the HD Graphics 530 inside it) and an NVIDIA GTX950M. The latter runs on Nouveau drivers and is almost never used as it's totally useless: official NVIDIA drivers are almost always broken while Nouveau drivers don't allow reclocking so...

https://i.imgur.com/isKUQQO.png

Anyway, long story short, the test was actually run using the HD Graphics 530 which can decode H.265 8bit (but not 10bit, why Intel, whyyyyyyyyy?!) via hardware and it did actually work. Of course, if I use the official Google Chrome and not the freeworld chromium version, on the other hand, the screen will stay pitch black with just the audio playing.


Fedora with the default drivers can't decode HEVC even if the GPU has hardware acceleration


I asked DBelton from the Fedora community and he helped me long time ago. I can't remember which Intel drivers and decoders he and Marko made me install through both the command line and Fedy, but the whole thing worked and it's still working today. You should definitely go to the Fedora Forum and ask, they'll help you just like they helped me back in 2016.


Also, Chromium can't sync to Google Account anymore so it's useless for most people (another little bit of backstabbing by Google, surprise surprise).


I'm totally with you on this, to be fair.
That has been one of the things that annoyed me incredibly 'cause it was a bit of a stab in the back to all the open source developers who worked on the various chromium forks.
I have some friends of mine who actively contribute and when that thing happened they didn't really take it well.
I, myself, didn't take it well either 'cause on the little Windows XP community I'm part of we have some people providing some constantly backported version of chromium (the last one being 108) and of course I haven't been able to sync my account ever since Google blocked it for third party forks... :(



Please note that while I don't hate HEVC or VVC per se, I hate the HEVC and VVC fanboys that try to downplay the massive IPR problems of those codecs

I mean, this is Doom9 not a reddit subforum, so professional people are here, so it's unlikely you'll ever meet any fanboy and I can assure you neither me nor Ben is.
But again, MPEG codecs have always been here and they're here to stay, so even if you were to get a couple more people on Doom9 jumping on the AV1 (and possibly AV2) bandwagon, it would change literally nothing (I know it's hard, but it's true). I mean, think about every 8K broadcaster, every future 8K BD manufacturer etc, what do you think it's gonna happen? Love it or hate it, H.266 VVC will be a thing just like H.265, H.264 and MPEG-2 have and there will be hardware encoders, hardware decoders, playback ports for playout systems and indeed GPUs manufacturers supporting those for consumer-tier profiles (NVIDIA almost definitely and then Intel and probably AMD) as well as hardware decoding support in mobile phones.

This is inevitable, it is going to happen and, in my opinion, it's not a bad thing.


By the way, to prove that I'm not biased in any way, even though my job is literally to create mezzanine files which are by definition going to be MPEG codec based (like XDCAM-50 and XAVC Intra Class 300) and will be played on hardware playback ports in playout systems, I did create (very recently) some AV1 files too for distribution as I was asked to do so for a few trailers (around 50 clips) that were gonna be played over the internet on web pages of our website. (https://forum.doom9.org/showthread.php?t=183907) So, you see, although AV1 (and in the future AV2) might become a web thing, I can almost definitely guarantee you that it will never happen for linear broadcasting anywhere in the world and that's just the way it is.

[kurkosdr]

Future TV broadcast standard like Brazil have chosen VVC. HDR and 4K ( It doesn't even need to be 8K ) Broadcast will also likely be using a new video codec standard. There are also other Streaming Services which doesn't rely on Web. Especially outside of EUR and North America. Even in 4K HDR Content VVC will still be much better than HEVC.
With some rare exceptions like Brazil, most countries chose HEVC for their UHD broadcasts years ago, so VVC for broadcasting is irrelevant for most parts of the world. The following is a good example:
https://www.google.com/search?q=site:digitalbitrate.com+vvc
https://www.google.com/search?q=site:digitalbitrate.com+hevc

The only broadcasting area I see VVC succeeding is 8K UHD content, which will probably be 2-3 channels per satellite or so, considering how scarce bitrate is in most satellites. Irrelevant in the big scheme of things, but still technically interesting.

Streaming boxes could be a big win for VVC, but it faces competition from AV1 and the existing HEVC installed base. That's a space to watch.

[kurkosdr]

I asked DBelton from the Fedora community and he helped me long time ago. I can't remember which Intel drivers and decoders he and Marko made me install through both the command line and Fedy, but the whole thing worked and it's still working today. You should definitely go to the Fedora Forum and ask, they'll help you just like they helped me back in 2016.
That's why I asked what are you using btw, because you have to do at least some things to get HEVC working in Fedora (and other Desktop Linux distros). Having to install codecs is the problem here, because officially recommending those codecs to people can be considered as "inducing" patent infringement. It's why Fedora can't show a pop-up saying "run the following commands to install such and such codec".

This is the problem I am highlighting here: Once you have to tell people that they need to have this and that, and can't even tell them how, the whole universal accessibility concept of the web is lost.

[benwaggoner]

No, not consistently at least.
Well, it is Linux we're talking about there's not a lot it has done consistently across distributions in terms of advanced digital media and graphical stuff.

Google decided to screw Desktop Linux users over and treat them as lesser Chrome users after promising they wouldn't do that (in order to drum up support for VP9). Why am I not surprised? The next step is Mozilla getting blamed for not treating Desktop Linux users as lesser Firefox users too.

Remember when the web was universally accessible and not subject to whether you've gone through the MPEG LA and Access Advance tollbooths? I do.
I certainly don't remember that for digital video, ever. I spent years making a lot of money encoding RealVideo, Windows Media, and QuickTime versions of the same content for the web as plenty of customers only had one of the three. When we got to more universal playback, it was due to the ubiquity of Flash's H.263 decoder support. We did have a while where browsers became powerful enough at H.264 + AAC-LC because universal enough that a single file could play on >99% of web browsers, but it was only a few years between that becoming reliable and HDR (with a 10-bit encoding requirement) becoming important.

Browsers have been about the only place where HEVC hasn't been universal the last five years.

[ksec]

With some rare exceptions like Brazil, most countries chose HEVC for their UHD broadcasts years ago, so VVC for broadcasting is irrelevant for most parts of the world. The following is a good example:
https://www.google.com/search?q=site:digitalbitrate.com+vvc
https://www.google.com/search?q=site:digitalbitrate.com+hevc

The only broadcasting area I see VVC succeeding is 8K UHD content, which will probably be 2-3 channels per satellite or so, considering how scarce bitrate is in most satellites. Irrelevant in the big scheme of things, but still technically interesting.

Streaming boxes could be a big win for VVC, but it faces competition from AV1 and the existing HEVC installed base. That's a space to watch.

Because you are comparing the current state and future state. The world is much bigger than North America and EUR. VVC are already in use in India. And China is getting ready too.

[ksec]

That's why I asked what are you using btw, because you have to do at least some things to get HEVC working in Fedora (and other Desktop Linux distros). Having to install codecs is the problem here, because officially recommending those codecs to people can be considered as "inducing" patent infringement. It's why Fedora can't show a pop-up saying "run the following commands to install such and such codec".

This is the problem I am highlighting here: Once you have to tell people that they need to have this and that, and can't even tell them how, the whole universal accessibility concept of the web is lost.

As far as I am aware. The HEVC and AVC decoding issue is only a concern using third party open source drivers. So if you have something like Ubuntu which installs drivers from Nvidia and Intel you would have no problem. the argument is basically that what ever patent there is, those official drivers are being accounted for.

But considering this is Fedora / Redhat, the company which declare AAC-LC as patent free ( Thank God ). This comes a little bit of a surprise. Especially after this has been the norm for years before declaring it unsafe. But they are now under IBM, I guess legal have a different interpretation.

Anyway we only have wait a few more years before AVC patents expires. We will have a truly parent free half decent video codec.

[kurkosdr]

Well, it is Linux we're talking about there's not a lot it has done consistently across distributions in terms of advanced digital media and graphical stuff.
When it comes to video, it's because of legal roadblocks, not technical reasons. Stop trying to deflect.


I certainly don't remember that for digital video, ever. I spent years making a lot of money encoding RealVideo, Windows Media, and QuickTime versions of the same content for the web as plenty of customers only had one of the three. When we got to more universal playback, it was due to the ubiquity of Flash's H.263 decoder support. We did have a while where browsers became powerful enough at H.264 + AAC-LC because universal enough that a single file could play on >99% of web browsers, but it was only a few years between that becoming reliable and HDR (with a 10-bit encoding requirement) becoming important.
From the browser's perspective, RM, WMV and QT/MOV were binary blobs to be downloaded, so it was out of scope. The problem started with the Flash Player plugin and then continued with HTML5 video tag leaving the format unspecified (nice implementable specification there folks).


Browsers have been about the only place where HEVC hasn't been universal the last five years.
For a good reason. The web is supposed to be universally accessible and universally implementable, which means it avoids patent-encumbered formats (more so patent-encumbered formats with multiple patent pools). Eventually, this boils down to whether you consider access to the web a fundamental human right. I do.

[birdie]

That was because GIF being a completely open format and totally not patent encumbered was a perfect match for the Web of the People. Right comrades ?

*edit*
So whats going on with x266? Cant be that hard to write a H.266 encoder. Looking at their feature roadmap I think it would take me just 2-3 man-months to get to a working release....

I guess they don't want to make public something which is a lot worse (both in quality and performance) than already existing codecs such as VVEnc.

And VVEnc is quite good actually albeit slow but not much slower than e.g. libaom.

[rwill]

Well, it is Linux we're talking about there's not a lot it has done consistently across distributions in terms of advanced digital media and graphical stuff.


Remember when the web was universally accessible and not subject to whether you've gone through the MPEG LA and Access Advance tollbooths? I do.


I certainly don't remember that for digital video, ever. I spent years making a lot of money encoding RealVideo, Windows Media, and QuickTime versions of the same content for the web as plenty of customers only had one of the three. When we got to more universal playback, it was due to the ubiquity of Flash's H.263 decoder support. We did have a while where browsers became powerful enough at H.264 + AAC-LC because universal enough that a single file could play on >99% of web browsers, but it was only a few years between that becoming reliable and HDR (with a 10-bit encoding requirement) becoming important.

Browsers have been about the only place where HEVC hasn't been universal the last five years.

I support that on the Web the baseline should be a 160x120 Theora video and those platforms that want better have to support H.264/HEVC or VVC. So the < 1% market share Linux Desktop guys get their stamp sized video and have no reason to complain anymore and the rest ( > 99% market share ) can finally move on, which they have already anyway.

Personally I don't get the Linux Desktop people. They are a clear minority but defend and make demands for their fragmented platform like some Otaku does for his waifu.

[rwill]

I guess they don't want to make public something which is a lot worse (both in quality and performance) than already existing codecs such as VVEnc.

And VVEnc is quite good actually albeit slow but not much slower than e.g. libaom.

Well should this be the case they have to be somewhat careful, lest they will never have anything to release ever.

[kurkosdr]

I support that on the Web the baseline should be a 160x120 Theora video and those platforms that want better have to support H.264/HEVC or VVC. So the < 1% market share Linux Desktop guys get their stamp sized video and have no reason to complain anymore and the rest ( > 99% market share ) can finally move on, which they have already anyway.

Personally I don't get the Linux Desktop people. They are a clear minority but defend and make demands for their fragmented platform like some Otaku does for his waifu.
As a person who uses Desktop Linux professionally, I can attest that I don't think I am entitled to "make demands" for Blu-Ray playback or access to every AAA PC game that gets released or anything like that. But access to the web is a human right. It's necessary for people to do things like filing taxes or searching for information so they can do their jobs. This means the web has to be universally accessible.

Oh, and your 160x120 Theora video thing, besides being stupid (downscaling loses a ton of information which can be important for things like tutorials) is something that's not even guaranteed to be there. Theora mandated as a minimum format with the same resolution as the other available formats would be a good idea because at least it makes for an implementable HTML5 spec.

But anyway, you don't have to care about my opinion. Have you ever wondered why W3C didn't propose a format for the HTML5 video tag? It's because they have a principle to not include patented-encumbered technology in W3C standards, which means their views are closer to mine than yours. But since they couldn't agree on a royalty-free standard, they left it unspecified (because that totally makes sense, I guess).

Also, people here forget a good 5% of Windows users run versions older than Windows 8, which means they aren't guaranteed to have HEVC OS codecs and probably run old PCs with old GPUs without hardware HEVC decoding either. But I know, those are poor people. Who cares if they can access the web?

[rwill]

<..> I can attest that I don't think I am entitled to "make demands" <..> But access to the web is a human right. <..> Theora mandated as a minimum format with the same resolution as the other available formats would be a good idea <..>

Wow - access to video on the web a human rights issue - this sure escalated fast.

[FranceBB]

Also, people here forget a good 5% of Windows users run versions older than Windows 8, which means they aren't guaranteed to have HEVC OS codecs and probably run old PCs with old GPUs without hardware HEVC decoding either.

There will always be a compatibility fallback in streaming platforms, though.
I mean, nowadays you would probably get H.265 HEVC for HDR UHD contents and H.264 AVC for SDR FULL HD 8bit and lower and I think that's going to stay for a very very very long time. I mean, if you watch a TV Series on streaming platform x and you go there with, let's say, Windows XP and a backported version of Chromium (currently Chromium 108 is the latest that has been backported), it will almost definitely serve you the H.264 version at all resolutions and bitrate combinations, ranging from FULL HD to HD to SD and possibly lower streams with AAC audio which is perfectly decodable. Speaking of which, even via software only decoding, H.264 nowadays is pretty well handled, so I don't really think it's gonna be a problem anytime soon as H.264 is here to stay. Ironically AV1 would be much harder to decode for an x86 SSE4.1 max operating system, so much so that even Google itself offers VP9 + Opus rather than AV1 to the Windows XP users even for UHD contents.

This is me on Windows XP Professional x86 with the Extended Microsoft Support which ended on July 2019 and Chromium 108:

- YouTube - VP9 + Opus (https://i.imgur.com/v4uAGee.png)
- BBC - H.264 + AAC (https://i.imgur.com/qxOcdz2.png)


I guess they don't want to make public something which is a lot worse (both in quality and performance) than already existing codecs such as VVEnc.


I guess that's the point.

Well should this be the case they have to be somewhat careful, lest they will never have anything to release ever.

Well, the current situation is the following while comparing with VTM, however no comparisons were done against VVEnc:

https://i.imgur.com/zIXwr46.png

[kurkosdr]

There will always be a compatibility fallback in streaming platforms, though.
I mean, nowadays you would probably get H.265 HEVC for HDR UHD contents and H.264 AVC for SDR FULL HD 8bit and lower and I think that's going to stay for a very very very long time. I mean, if you watch a TV Series on streaming platform x and you go there with, let's say, Windows XP and a backported version of Chromium (currently Chromium 108 is the latest that has been backported), it will almost definitely serve you the H.264 version at all resolutions and bitrate combinations, ranging from FULL HD to HD to SD and possibly lower streams with AAC audio which is perfectly decodable.
First of all, even H.264 system codecs are not guaranteed to exist on Windows Vista and earlier, so some browsers will fail H.264 decoding on those OSes too. Also, even this H.264 fallback is not guaranteed to exist on all websites (this is why W3C's decision to not include some mandatory format for the HTML5 video tag was a big omission). This is why I think Mozilla's and Google's initial decision to restrict decoding on formats with a palatable IPR situation was the right one, given the nature of the spec. Even H.264 was problematic enough from an IPR standpoint (but that was grandfathered in, I guess).

[benwaggoner]

When it comes to video, it's because of legal roadblocks, not technical reasons. Stop trying to deflect.
There is no legal roadblock to passing off bitstreams to drivers that pass them on to hardware components that I am aware of.

From the browser's perspective, RM, WMV and QT/MOV were binary blobs to be downloaded, so it was out of scope. The problem started with the Flash Player plugin and then continued with HTML5 video tag leaving the format unspecified (nice implementable specification there folks).
Correct. And that period covers the majority of the history of video in browsers, circa 1995-2016 or so, depending on how one defines it.

For a good reason. The web is supposed to be universally accessible and universally implementable, which means it avoids patent-encumbered formats (more so patent-encumbered formats with multiple patent pools). Eventually, this boils down to whether you consider access to the web a fundamental human right. I do.
Is access to the web at stake here somehow? It seems we're more talking about "access to anything that anyone could publish to the web."

The fundamental challenge is that digital media is really complex, with heavy compute and real-time requirements. That's very different that the original goals of the web, which focused on publishing content in a way that different clients could render in ways that maximally preserve the information being presented. Stuff that requires constant heavy compute with real-time requirements was way out of scope.

Media codecs are also really complex beasts, with an enormous number of tools that need to be combined in very complex ways to provide enough improvement to merit the high costs of adopting a new format. So far, the MPEG/ISO process has, despite its many drawbacks, created codecs sufficiently superior to alternatives that they become dominant despite those limitations.

This is because Moore's Law and hand-tuned assembly and hardware DRM and vsync and lots of stuff that other web stuff doesn't require become essential.

H.263 has been patent free for decades. It was good enough for Flash and YouTube to become juggernauts. But the advancements around video codecs are too important and complex for content publishers to be happy to stick with a "good enough" format like we have with JPEG and PNG for so long.

This is how it has always been, and I don't see a clear alternative that would be long-term sustainable. Maybe if AV2 winds up being just better than VVC, sure. But we're still not going to see anyone publishing premium HDR content to web browsers without DRM, because that would cannibalize other revenue streams way more than the Linux market could add. And if somehow movie studios were forced to release their high value content in ways they can't preserve a good share of that value, we'll just have less and cheaper content made.

We have Peak TV because streaming enabled new business models that made it possible to monetize depth of engagement; before the only way to increase revenue was to increase number of viewers of ads, however apathetic they are. Anyone who prefers to make content under different business models is welcome to do so, and plenty do. Anyone who only wants to watch content produced via business models or distributed with technology they approach of is welcome to do so.

None of us have a right to make content creators make the content we want the way we want it under terms that we define.

[benwaggoner]

But anyway, you don't have to care about my opinion. Have you ever wondered why W3C didn't propose a format for the HTML5 video tag? It's because they have a principle to not include patented-encumbered technology in W3C standards, which means their views are closer to mine than yours. But since they couldn't agree on a royalty-free standard, they left it unspecified (because that totally makes sense, I guess).
There are and were able patent-free video codecs available to pick from. MPEG-1, H.263, and Theora, for example. I presume they weren't picked because they weren't good enough to be competitive, but they certainly did exist. H.263 was the dominant web codec for several years, before Flash added H.264.

Also, people here forget a good 5% of Windows users run versions older than Windows 8, which means they aren't guaranteed to have HEVC OS codecs and probably run old PCs with old GPUs without hardware HEVC decoding either. But I know, those are poor people. Who cares if they can access the web?
There is no version of Windows guaranteed to have HEVC out of the box. Pretty much all OEM systems do and have for years. But a system builder could certainly build a system that doesn't, with some effort.

[kurkosdr]

Oh ffs, what do VP8 and Theora have to do with v-sync? And when did they have any kind of issues with assembly hand-tuning (which all software decoders and encoders have)? If you are gonna post such long posts, at least try to make sense.

Also, VP8 was certainly good enough, and Theora was good enough as a fallback, and W3C could have chosen either as a mandatory format for the video tag to define an implementable HTML5 spec (which is kind of an important attribute for a spec, if you haven't noticed), but some "important" members with lots of patents in the patent pools (Apple and Microsoft) blocked that. Design-by-committee at its worst.

And no, the primary purpose of W3C is not to please content creators or care about hardware DRM, the primary purpose of W3C is to define implementable standards for the web, in case you also failed to notice.


There is no legal roadblock to passing off bitstreams to drivers that pass them on to hardware components that I am aware of.
[...]
There is no version of Windows guaranteed to have HEVC out of the box. Pretty much all OEM systems do and have for years. But a system builder could certainly build a system that doesn't, with some effort.
That's the problem here. With no mandatory format defined in the spec, the browser can't guarantee the implementation of the spec no matter what software decoders it bundles (I hope this plain fact is apparent to everyone). And passing to system codecs doesn't guarantee successful decoding either due to what you said there and the other issues mentioned in this thread. The fact Chrome and Firefox used to agree on what formats to decode offered some hope that the mess the W3C farted out as a purported standard would at least be defacto standardised, and formats with particularly bad IPR issues avoided in the process as a bonus, but nope. it's a mess that will stay that way. Can't wait until some evolution of WMV becomes an important codec for the web. Why not? If it comes into existence, there will be a system decoder for it in most Windows PCs (and Macs, since Apple will probably license it). Just pass it to the system codecs!

[kurkosdr]

Is access to the web at stake here somehow? It seems we're more talking about "access to anything that anyone could publish to the web."
According to this line of thinking, IE with its incompatible JavaScript and ActiveX wasn't breaking compatibility with other web browsers, just with something that someone could publish. But according to this line of thinking, what's the point of even having web standards?

[kurkosdr]

H.263 has been patent free for decades.
No, and certainly not for decades.
https://meta.wikimedia.org/wiki/Have_the_patents_for_h.263_expired_yet%3F

[rwill]

Well, the current situation is the following while comparing with VTM, however no comparisons were done against VVEnc: <..>

This is how VVenC stacks up against x265...

https://github.com/fraunhoferhhi/vvenc/wiki/Encoder-Performance

There seems to be no mischief going on with x265 settings as far as I can tell from the x265 command lines.
If you look at "VVenC: Runtime vs PSNR BD-rate" x265 placebo has 30% higher BD-rate than HM-16.24.
They specify the x265 command line to be "--preset {0,1,2,3,…,9} --tune psnr --crf {17,22,27,32} --keyint 1s --min-keyint 1s --profile main10 --output-depth 10", so thats PSNR tune with CRF bitrate control.

Now either HM got really good in the last couple years or x265 aged like milk or.. well I don't know. 'placebo' preset should be closer to HM really.

VTM having some SIMD optimizations now makes it less of a pushover performance wise but it still lacks threading. Adding some conservative mode decision + threading and you should end up where VVenC 'slow' is.

But I am really surprised by how bad x265 is compared against HM-16.24. And the x265 guys (those that are still left anyway) now want to make a competitive VVC encoder?

Well good luck then.

[rwill]

There are and were able patent-free video codecs available to pick from. MPEG-1, H.263, and Theora, for example. I presume they weren't picked because they weren't good enough to be competitive, but they certainly did exist. H.263 was the dominant web codec for several years, before Flash added H.264.

I am still hoping for EVC Baseline Profile to somehow get off the ground. While not patent free its supposed to be royalty free. It allows for a really tiny efficient implementation because of its small toolset.

Hoping for something remotely usable that is truly patent free in the year 2023, where almost everything promising regarding video has been at least patented twice, appears to be some sort of pipe dream to me.

[benwaggoner]

This is how VVenC stacks up against x265...

https://github.com/fraunhoferhhi/vvenc/wiki/Encoder-Performance

There seems to be no mischief going on with x265 settings as far as I can tell from the x265 command lines.
If you look at "VVenC: Runtime vs PSNR BD-rate" x265 placebo has 30% higher BD-rate than HM-16.24.
They specify the x265 command line to be "--preset {0,1,2,3,…,9} --tune psnr --crf {17,22,27,32} --keyint 1s --min-keyint 1s --profile main10 --output-depth 10", so thats PSNR tune with CRF bitrate control.
HM has always offered better quality within its limitations than x265, with epochal patience. x265 was always better in quality @ perf, and supports all the rate control, adaptive quantization, etcetera stuff needed for practical real-world encoding.

HM can absolutely be better at delivering better mean PSNR with fixed-QP, fixed GOP encoding without VBV. But that's not a thing anyone actually watches.

But I am really surprised by how bad x265 is compared against HM-16.24. And the x265 guys (those that are still left anyway) now want to make a competitive VVC encoder?
It's "bad" at stuff no x265 customers or users want to do. This is always true about reference encoders versus commercial encoders.

I expect that VVCEnC already beats the reference encoder using real-world scenarios and performance requirements. And x266 certainly would need to as well for a meaningful 1.0 release.

[benwaggoner]

I am still hoping for EVC Baseline Profile to somehow get off the ground. While not patent free its supposed to be royalty free. It allows for a really tiny efficient implementation because of its small toolset.

Hoping for something remotely usable that is truly patent free in the year 2023, where almost everything promising regarding video has been at least patented twice, appears to be some sort of pipe dream to me.
The only thing I've heard about EVC in the last few years is the base codec for the ML extensions in the MPAI-EVC project: https://mpai.community/standards/mpai-evc/about-mpai-evc/

Although MPAI seem to be pivoting focus to the newer end-to-end ML MPAI-EEV codec: https://mpai.community/standards/mpai-eev/about-mpai-eev/

[kurkosdr]

I am still hoping for EVC Baseline Profile to somehow get off the ground. While not patent free its supposed to be royalty free. It allows for a really tiny efficient implementation because of its small toolset.

Hoping for something remotely usable that is truly patent free in the year 2023, where almost everything promising regarding video has been at least patented twice, appears to be some sort of pipe dream to me.
Unfortunately, nobody wants EVC. Mozilla and Google don't want it due to the "Baseline" vs "Additional tools enabled" thing (sure, you can choose to decode without using the additional tools, but at what quality cost?), and Microsoft and Apple don't want it because it's a threat to HEVC and by extension the patents they have in HEVC.

[ksec]

I am still hoping for EVC Baseline Profile to somehow get off the ground. While not patent free its supposed to be royalty free. It allows for a really tiny efficient implementation because of its small toolset.

Hoping for something remotely usable that is truly patent free in the year 2023, where almost everything promising regarding video has been at least patented twice, appears to be some sort of pipe dream to me.

Yes. While The E stands for Essential, I could actually argued it should stand for Efficiency. How on earth did they manage to achieve something better than AVC Part 10, while simultaneously being computationally less expensive to decode. It also uses mostly H.264 era patents. And would actually becomes patents free relatively quickly. If it was combined with LC-EVC ( Another set of standard ) it would give HEVC level plus quality with a much lower complexity. But yes... a pipe dream to me as well.

[kurkosdr]

It also uses mostly H.264 era patents. And would actually becomes patents free relatively quickly. If it was combined with LC-EVC ( Another set of standard ) it would give HEVC level plus quality with a much lower complexity.
Do we have a patent list for EVC Baseline and Non-Baseline? LC-EVC?

[birdie]

https://i.ibb.co/drBKJk9/message.png

[benwaggoner]

Unfortunately, nobody wants EVC. Mozilla and Google don't want it due to the "Baseline" vs "Additional tools enabled" thing (sure, you can choose to decode without using the additional tools, but at what quality cost?), and Microsoft and Apple don't want it because it's a threat to HEVC and by extension the patents they have in HEVC.
Lots of incumbent players, including Apple, already have HEVC encode and decode capabilities, and generally a new codec has to offer >40% compression efficiency reductions to justify the huge transitional costs of shifting an ecosystem. It seems like HEVC licensing hit the "good enough" threshold while EVC didn't have path to big gains over HEVC. VVC clearly does have that path, with the >40% bitrate reduction quite feasible within the next couple of years.

Apple's done very will the the HEIC (HEVC still frame) format for photography, cutting file sizes a bunch over JPEG, and offering better quality with sharp edged and synthetic content. EVC would have had to offer something really compelling for Apple to introduce a new non-backwards compatible format.

[nakTT]

https://i.ibb.co/drBKJk9/message.png
Nice one!:thanks:

I have been waiting for x266 for while now.

[kurkosdr]

Their faq says:
What's your very rough and approximate ETA for the public x266 1.0 release?
H2 2023 is what we are looking at for the release of x266. An update will be given for the same

To me this means that they will release until 31st December 2023 or an update will be given on 1st January 2024.

Anyway, the GPL obligates them to provide source code if they release, so the only way for them to avoid releasing source code is to avoid releasing at all, basically abandon the whole project. I don't think they will do that considering the effort they've already put in.

[oibaf]

Unfortunately, nobody wants EVC. Mozilla and Google don't want it due to the "Baseline" vs "Additional tools enabled" thing (sure, you can choose to decode without using the additional tools, but at what quality cost?)...

Any source for this? I didn't find any.

[birdie]

Doesn't look like we're getting x266 this year:

Hello Artem,

Thanks for reaching out and for your continued enthusiasm. The x266 video codec development is on-going. If you have any specific requirement, please let us know. I am sure our team can help you out with the right solution. And when the x266 codec is nearing completion, we will get in touch with you!

Best Regards,
Suchithra Thyagarajan

[rwill]

(sure, you can choose to decode without using the additional tools, but at what quality cost?)

Any source for this? I didn't find any.

I think he pulled that one out of his rear as you cannot decode EVC Main Profile streams with a EVC Baseline decoder.

[rwill]

Doesn't look like we're getting x266 this year:

What a fail.

He noted your enthusiasm though. Maybe you should start a Go-Fund-Me to support their development effort. I mean Duke Nukem Forever took a while too. BTW anyone got news of Star Citizen?

[FranceBB]

I don't think they will do that considering the effort they've already put in.

This and the fact that they would upset the companies funding the development via the current x266 consortium partnership program (like the one I work for).
In other words, we will get x266, that's for sure, the only question is "when".