Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th November 2023, 04:13   #1  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 352
Netflix Loses Patent Dispute to Broadcom in Germany, Ordered to Stop Using HEVC

Turns out HEVC does have "content fees" after all:

https://www.nexttv.com/news/achtung-...c-to-stream-4k

https://www.broadbandtvnews.com/2023...inging-patent/

Quoting from the second article:
Quote:
The European patent at issue, EP 2 575 366 (366 Patent), covers key features of digital video processing often used in HEVC video coding. The Munich District Court ruled that Netflix infringes the 366 Patent through its transmission of HEVC video, which Netflix uses to provide Ultra HD content to its users.
Personally, I can't say I am too surprised. There are several patent-licensing groups for HEVC (see here for details), and we can't just assume all of them are fine with not charging "content fees", especially when some of them have not made their license terms public.

This is also probably why free-to-view platforms such as YouTube and Twitch have steered clear of HEVC and have adopted VP9 as their post-H.264 codec... Netflix will probably sign a license and pass the "content fee" to its subscribers, free-to-view platforms can't do that. This means that a ruling like the above for a free-to-view platform would mean being stuck with tons of HEVC-encoded content (that cost time and electricity to encode) that can't be streamed to users. The whole "one format to rule them all" idea is nice as an engineer's fantasy, but economic realities are dictating what actually happens.

Anyway, looks like we have two post-H.264 codecs: HEVC (for subscription content and broadcast) and VP9 (for free-to-view). And then there's AV1 which is theoretically competing with VVC but actually competing with HEVC on internet subscription content and is also used for free-to-view.

Last edited by kurkosdr; 26th November 2023 at 05:53.
kurkosdr is offline   Reply With Quote
Old 26th November 2023, 05:45   #2  |  Link
Sunspark
Registered User
 
Join Date: Nov 2015
Posts: 511
Microsoft has charged end-users $1 for the HEVC codec for some time now, sometimes it is possible to get it for free.
Sunspark is offline   Reply With Quote
Old 26th November 2023, 06:04   #3  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 352
Quote:
Originally Posted by Sunspark View Post
Microsoft has charged end-users $1 for the HEVC codec for some time now, sometimes it is possible to get it for free.
That's for the decoder though. What determines whether a format gets adopted by free-to-view services is the absence of "content fees". H.264 got adopted by free-to-view services after famously waiving "contents fees" for free-to-view internet video.

Basically, royalties apply for decoders, encoders, and transmission of encoded content (the last referred to as "content fees").

I trust YouTube (Google) have done their research and realised that HEVC can't be considered to be clear of "content fees" for free-to-view, and they are better of going to VP9 and AV1 (and streaming 1080p H.264 to UltraHD SmartTVs that support HEVC but not VP9).

What we know is that there Broadcom is collecting "content fees", and there is no known waiver for free-to-view from Broadcom's side. And since the whole reason those patent holders broke off from MPEG LA was to milk their patents more aggressively, I wouldn't be too optimistic on such a waiver existing.

Last edited by kurkosdr; 26th November 2023 at 06:07.
kurkosdr is offline   Reply With Quote
Old 26th November 2023, 13:44   #4  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 332
Quote:
Originally Posted by kurkosdr View Post
Turns out HEVC does have "content fees" after all:
Well it speaks more for that there wasnt any, at least netflix didnt pay broadcom any But whats at least for sure, is that this proves the skepticals right, you are defiantly open to patent issues when having a service using HEVC.

From the looks of it broadcom has had Netflix in it sights for quite some time, filing patent infringements all over the place, saying that they are using their technology while undermining their STB/cable business.

Just look at some of them on this filing: "Dynaimc Network Load Balacing Heterogeneous Link Speed", "System and method for Procesing Data Using a Network", "Multiple Pathway Session Setup to Support QoS Sercices" etc. Actually only two of the nine are video-compression related.

https://www.documentcloud.org/docume...oadcom-Netflix

As thay also have a bunch of HEVC related once they have been using those as well (from what I can see is that they also been filing h264 related ones), and from the looks of it, one of those worked.

Here is patent in question: https://data.epo.org/publication-ser...8&iepatch=.pdf

It will be very interesting to see that fallout of this, could have a huge impact that goes beyond Netflix not offering HEVC in Germany. The question is also if Broadcom is specificly targeting HEVC licensing fees, or if this is more a tactic cause Netflix didnt want to sign a license agreement with them, that seemed to cover a lot of more technologies than just video-compression.

Last edited by excellentswordfight; 26th November 2023 at 13:53.
excellentswordfight is offline   Reply With Quote
Old 26th November 2023, 18:31   #5  |  Link
Sunspark
Registered User
 
Join Date: Nov 2015
Posts: 511
I don't know that there will be a huge impact. From what I have been able to observe, most places/services have avoided HEVC which has resulted in a chicken & egg scenario where it isn't offered because nobody has it because it isn't offered.
Sunspark is offline   Reply With Quote
Old 26th November 2023, 22:23   #6  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 352
Quote:
Originally Posted by Sunspark View Post
I don't know that there will be a huge impact. From what I have been able to observe, most places/services have avoided HEVC which has resulted in a chicken & egg scenario where it isn't offered because nobody has it because it isn't offered.
If all the HEVC patent holders wanted to waive "content-fees" (at least for free-to-view internet content), they would all do it tomorrow, and publicly so. The fact they don't is highly suspicious, it reeks of "let them adopt it first (and spend their time and electricity encoding to it), and then we'll come and collect".
kurkosdr is offline   Reply With Quote
Old 27th November 2023, 02:23   #7  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 1,135
If i were google i'd remove h265 support from chrome. That would act as a good punishment by harming adoption of hevc.
hajj_3 is offline   Reply With Quote
Old 27th November 2023, 03:05   #8  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,824
Quote:
Originally Posted by Sunspark View Post
I don't know that there will be a huge impact. From what I have been able to observe, most places/services have avoided HEVC which has resulted in a chicken & egg scenario where it isn't offered because nobody has it because it isn't offered.
All streaming services that do HDR use HEVC. Some may also use other codecs, but HEVC is the best codec supported on the majority of HDR capable devices.

Apple's also been all-in on HEVC for their devices, with it as their default video recording format AND image format (HEIC).
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 27th November 2023, 04:19   #9  |  Link
Sunspark
Registered User
 
Join Date: Nov 2015
Posts: 511
Thanks Ben, I didn't know this as I don't have HDR or an Apple device. Good info.
Sunspark is offline   Reply With Quote
Old 27th November 2023, 04:41   #10  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 352
Quote:
Originally Posted by benwaggoner View Post
All streaming services that do HDR use HEVC. Some may also use other codecs, but HEVC is the best codec supported on the majority of HDR capable devices.

Apple's also been all-in on HEVC for their devices, with it as their default video recording format AND image format (HEIC).
Nope, definitely nope. YouTube uses VP9 and AV1 to deliver HDR (and yes it also supports PQ not just HLG)
https://support.google.com/youtube/answer/7126552
and doesn't use HEVC at all.

They even have an HDR channel with HDR PQ videos:
https://www.youtube.com/@TheHDRChannel

I've attached the specs of the VP9 and AV1 streams of a video from the above channel below.

The issue with Google's approach is that practically all UHD TVs support HEVC with HDR10, while not all UHD TVs support VP9 with HDR10 (minimum), which means that on TVs that don't support at least VP9 with HDR10, YouTube will top out either at 1080p H.264 or 2160p VP9 SDR (depending on what the TV does support).

Netflix is a paid service though, and not providing HDR10 on UHD TVs could lead some people to stop paying for the premium tier of the subscription, so they have to do HEVC.

Also, YouTube's streams don't do HDR10+ (don't know if Netflix does), not to say YouTube's HDR10 is bad for free-to-view video (also, HDR10+ coming sometime I am told).

So, it boils down to: Is supporting HDR on older UHD TVs and providing HDR10+ worth paying the "content fee" for your streaming service? That depends on the streaming service. And that's how economic realities decide the formats.

Code:
General
Complete name                            : <path>/Real 4K HDR 60fps: Sony Food Fizzle in HDR VP9.webm
Format                                   : WebM
Format version                           : Version 4
File size                                : 276 MiB
Duration                                 : 1 min 22 s
Overall bit rate                         : 28.2 Mb/s
Frame rate                               : 59.940 FPS
Writing application                      : google/video-file
Writing library                          : google/video-file

Video
ID                                       : 1
Format                                   : VP9
Format profile                           : 2
Format level                             : 5.1
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_VP9
Duration                                 : 1 min 22 s
Bit rate                                 : 27.1 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 0)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.054
Stream size                              : 265 MiB (96%)
Language                                 : English
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : R: x=1.000000 y=1.000000, G: x=1.000000 y=1.000000, B: x=1.000000 y=1.000000, White point: x=1.000000 y=1.000000
Mastering display luminance              : min: 0.1000 cd/m2, max: 0 cd/m2
Code:
General
Complete name                            : <path>/Real 4K HDR 60fps: Sony Food Fizzle in HDR AV1.mp4
Format                                   : dash
Codec ID                                 : dash (iso6/av01/mp41)
File size                                : 266 MiB
Duration                                 : 1 min 22 s
Overall bit rate                         : 27.2 Mb/s
Frame rate                               : 59.940 FPS
Encoded date                             : 2022-09-17 16:55:47 UTC
Tagged date                              : 2022-09-17 16:55:47 UTC

Video
ID                                       : 1
Format                                   : AV1
Format/Info                              : AOMedia Video 1
Format profile                           : Main@L5.1
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : av01
Duration                                 : 1 min 22 s
Bit rate                                 : 27.2 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 (60000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.055
Stream size                              : 266 MiB (100%)
Title                                    : ISO Media file produced by Google Inc.
Encoded date                             : 2022-09-17 16:55:47 UTC
Tagged date                              : 2022-09-17 16:55:47 UTC
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : R: x=1.000000 y=1.000000, G: x=1.000000 y=1.000000, B: x=1.000000 y=1.000000, White point: x=1.000000 y=1.000000
Mastering display luminance              : min: 0.1000 cd/m2, max: 1 cd/m2
Codec configuration box                  : av1C

Last edited by kurkosdr; 27th November 2023 at 05:01.
kurkosdr is offline   Reply With Quote
Old 28th November 2023, 02:15   #11  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,824
Quote:
Originally Posted by kurkosdr View Post
Nope, definitely nope. YouTube uses VP9 and AV1 to deliver HDR (and yes it also supports PQ not just HLG)
https://support.google.com/youtube/answer/7126552
and doesn't use HEVC at all.
I was speaking more of premium streaming services, not user generated content like YouTube, which has extremely different financial constraints. And of course, YouTube is Google's dog that the codec efforts wag, so it's always way more aggressive with non-MPEG codecs than anyone else.

All premium services support HDR on lots of TVs that don't have decoders for AV1. Everyone launched HDR on HEVC, and will still deliver somewhere between 100%-40% of HDR as HEVC. AV1 is still a couple years out from critical mass, and there wasn't ever a x265-competitive VP9 HDR encoder in terms of quality @ perf @ bitrate.

Quote:
They even have an HDR channel with HDR PQ videos:
https://www.youtube.com/@TheHDRChannel
I'm not aware of people streaming HDR other than PQ except maybe for some live sports that is originated in HLG. HLG is really a RF solution solving a problem that IP delivery doesn't have.

Quote:
The issue with Google's approach is that practically all UHD TVs support HEVC with HDR10, while not all UHD TVs support VP9 with HDR10 (minimum), which means that on TVs that don't support at least VP9 with HDR10, YouTube will top out either at 1080p H.264 or 2160p VP9 SDR (depending on what the TV does support).


Netflix is a paid service though, and not providing HDR10 on UHD TVs could lead some people to stop paying for the premium tier of the subscription, so they have to do HEVC.
Quote:
Also, YouTube's streams don't do HDR10+ (don't know if Netflix does), not to say YouTube's HDR10 is bad for free-to-view video (also, HDR10+ coming sometime I am told).
Yeah, the economics of user generated content mean that YouTube spends a lot fewer MIPS per pixel than premium services, as the large majority of their titles are watched a single digit number of times.

Quote:
So, it boils down to: Is supporting HDR on older UHD TVs and providing HDR10+ worth paying the "content fee" for your streaming service? That depends on the streaming service. And that's how economic realities decide the formats.
HDR10+ is just some extra SEI messages, so there's no downside to including it. It's an open spec with no implementation charge as well, so YouTube certainly could support it if they want.

Lots of streaming services include HDR10+ in all their HDR-10 streams by default.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th November 2023, 11:06   #12  |  Link
oibaf
Registered User
 
Join Date: Nov 2008
Posts: 71
Quote:
Originally Posted by hajj_3 View Post
If i were google i'd remove h265 support from chrome. That would act as a good punishment by harming adoption of hevc.
Firefox also added HEVC decoding support in version 120 released few days ago: https://bugzilla.mozilla.org/show_bug.cgi?id=1849392
oibaf is offline   Reply With Quote
Old 28th November 2023, 19:03   #13  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 352
Quote:
Originally Posted by oibaf View Post
Firefox also added HEVC decoding support in version 120 released few days ago: https://bugzilla.mozilla.org/show_bug.cgi?id=1849392
Semi-related note: This is related to another engineer's fantasy some people on the interwebs had (including me): That a certain format should be playable by a browser with and without the presence of system codecs, or not be playable at all. The idea is that a given HEVC stream is not guaranteed to have an H.264 or VP8 fallback stream, so in those cases, the website's video player will work on certain system configurations but not on others (yes, using the same browser). Making HEVC unplayable in all cases would theoretically push HEVC out of the web and solve the issue that way. However, while an H.264 or VP8 fallback stream is not guaranteed to exist, it almost always exists, so this problem doesn't really manifest, which means it's a problem nobody cares about. I guess this is what Google and Mozilla realized and enabled the passing of HEVC decoding to system codecs to enable its use by certain subscription services (and avoid HEVC playback becoming a competitive edge for Edge and Safari, since most new systems now ship with HEVC system codecs). Again, economic realities beat engineer's fantasy.

And one fun fact: On Firefox, even H.264 is not playable without system codecs, but almost every computer made from 2009 and onwards has an H.264 system codec (save for some Desktop Linux distros), so again, it's a problem no non-engineer cares about.

Last edited by kurkosdr; 28th November 2023 at 19:19.
kurkosdr is offline   Reply With Quote
Old 28th November 2023, 20:10   #14  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 1,135
Quote:
Originally Posted by kurkosdr View Post
And one fun fact: On Firefox, even H.264 is not playable without system codecs, but almost every computer made from 2009 and onwards has an H.264 system codec (save for some Desktop Linux distros), so again, it's a problem no non-engineer cares about.
i'm pretty sure firefox bundles cisco's openh264 decoder.
hajj_3 is offline   Reply With Quote
Old 28th November 2023, 20:40   #15  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 352
Quote:
Originally Posted by hajj_3 View Post
i'm pretty sure firefox bundles cisco's openh264 decoder.
I thought of that too, but turns out OpenH264 only does Baseline profile (most H.264 videos are High profile and practically all 720p and above H.264 videos are High profile). OpenH264 is there for WebRTC.
kurkosdr is offline   Reply With Quote
Old 28th November 2023, 21:21   #16  |  Link
oibaf
Registered User
 
Join Date: Nov 2008
Posts: 71
Quote:
Originally Posted by kurkosdr View Post
I thought of that too, but turns out OpenH264 only does Baseline profile (most H.264 videos are High profile and practically all 720p and above H.264 videos are High profile). OpenH264 is there for WebRTC.
According to this: https://github.com/cisco/openh264/pu...76f83ad642e944
they says it is just supporting Constrained Baseline, but it should actually also supports Main and High profiles.
oibaf is offline   Reply With Quote
Old 28th November 2023, 21:22   #17  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,824
Quote:
Originally Posted by hajj_3 View Post
i'm pretty sure firefox bundles cisco's openh264 decoder.
I believe all the top four browsers bundle a software H.264 decoder.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th November 2023, 21:34   #18  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,824
Quote:
Originally Posted by kurkosdr View Post
Semi-related note: This is related to another engineer's fantasy some people on the interwebs had (including me): That a certain format should be playable by a browser with and without the presence of system codecs, or not be playable at all. The idea is that a given HEVC stream is not guaranteed to have an H.264 or VP8 fallback stream, so in those cases, the website's video player will work on certain system configurations but not on others (yes, using the same browser). Making HEVC unplayable in all cases would theoretically push HEVC out of the web and solve the issue that way. However, while an H.264 or VP8 fallback stream is not guaranteed to exist, it almost always exists, so this problem doesn't really manifest, which means it's a problem nobody cares about. I guess this is what Google and Mozilla realized and enabled the passing of HEVC decoding to system codecs to enable its use by certain subscription services (and avoid HEVC playback becoming a competitive edge for Edge and Safari, since most new systems now ship with HEVC system codecs). Again, economic realities beat engineer's fantasy.
Spot on. In particular, lots of services only have HEVC for HDR content, so lack of HEVC support was increasingly limiting browsers to SDR.

And even when a AV1 software decoder exists, premium content DRM requirements typically mandate HW DRM for HDR and UHD. Battery life playing a two hour movie in 4K 10-bit SW decode can also be materially lower than with a HW decoder.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th November 2023, 21:36   #19  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,824
Quote:
Originally Posted by oibaf View Post
According to this: https://github.com/cisco/openh264/pu...76f83ad642e944
they says it is just supporting Constrained Baseline, but it should actually also supports Main and High profiles.
Yeah, I don't think I've seen anyone make a Constrained Baseline stream in well over a decade. High has been a safe default for a long, long time now.

(even when targeting really slow SW decoders, Constrained Baseline + 8x8 blocks from High Profile is still better and a tiny bit faster to decode).
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 3rd December 2023, 03:23   #20  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,975
Quote:
Originally Posted by benwaggoner View Post
I'm not aware of people streaming HDR other than PQ except maybe for some live sports that is originated in HLG. HLG is really a RF solution solving a problem that IP delivery doesn't have.
Not just sports, anything that airs on a linear channel is HLG, encoded in H.265 and transmitted as a .ts stream.
This means that if a movie or a documentary goes on air on linear TV it is also gonna be in HLG (90% of the times it's converted from PQ by basically taking the input in, finding the MaxCLL value in the metadata or a sidecar xml, mapping whatever that was to 1000 nits HLG - unless it's lower than 1000 nits - and then encoding).
Sometimes it becomes a tedious process as it also involves getting the conversion process (and specially created LUTs) signed off by the majors owning the content that gets delivered.

Running linear channels nowadays isn't easy as you quite literally need to produce the UHD HDR HLG BT2020 50p version for the UHD channel that gets encoded in H.265, the FULL HD BT709 SDR 25i version for the FULL HD channel that gets encoded in H.264 and the SD BT601 SDR 25i version for the SD channel that gets encoded in MPEG-2.

I still dream of the day in which we'll be able to get rid of SD and MPEG-2 from the face of the Earth once and for all...

Anyway the point being that those linear channel companies most often than not also have a (limited) streaming service and you can bet anything you want that anything UHD in there is also gonna be HLG 'cause the last thing they want is asking the staff to create yet another version, only in PQ perhaps, just for the streaming service, 'cause that would require time, effort and would also cost money as it would need to go down the whole processing chain again, I mean encoding, loudness correction, QC, distribution etc.

Last edited by FranceBB; 3rd December 2023 at 03:28.
FranceBB is offline   Reply With Quote
Reply

Tags
content-fees, h.265, hevc, licensing, patents

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:58.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.