Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd August 2023, 18:34   #41  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,980
A service I worked on called Movies Anywhere uses HEVC wherever possible. This is a free service dealing with premium Hollywood content. It was free because it was tied to the digital rights you get when you buy a title on disc or at a streaming service.

We had no HEVC licensing costs, and were comfortable taking the risk. We delivered, 4K, with HDR, Dolby Vision, Atmos, etc.
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Old 22nd August 2023, 19:12   #42  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,653
Quote:
Originally Posted by kurkosdr View Post
Copying and pasting from above:
- Facebook
- Twitch
- Various X-rated sites
- Microsoft Stream
- IBM Video
- various YouTube alternatives (Rumble etc)

All of them H.264-only, except Twitch which uses H.264 and VP9. None of them uses HEVC.
Facebook also has big Feels about Patents. They've really backed away from their AV1 goals, however. I can't speak about Twitch in detail as they are a partner company. I don't have much experience with the other providers/markets. It doesn't sound like content sees would apply in those cases, and companies like Microsoft and IBM have extensive patent cross-licensing portfolios which should reduce or eliminate encoder/decdoer fees.

Real world, it gets down to the potential benefits of reduced bandwidth costs versus the extra storage costs of having multiple codecs and the complexity of encoding and managing a multi codec service. Plus if a service is live and not being hugely profitable, the instinct is to leave it as it is rather than make a big capital investment in long-term operational cost savings. And HEVC in browsers is relatively new and kinda came out of the blue. Services could be working on multi-codec but not have launched yet.

Also, a lot of services you've mentioned aren't really monetizing quality, and so reducing bitrate, even with a quality hit, is a much easier knob to control storage and bandwidth costs.

And if a service didn't archive the source, but just kept distribution encodes, reencoding a mediocre H.264 into HEVC doesn't offer nearly the same benefits as encoding fresh from the source. This has been pretty common to save on storage costs.

Real-time HEVC encoding takes a few times more compute/pixel, and would require hardware upgrades if a service is using low-power fixed function H.264 encoders. We're not seeing much live AV1 for similar reasons (although it is several times slower yet than HEVC). The extra capital and/or operational costs per channel may not pencil out for low-viewership streams. And, at least, would require a new budget cycle.

Quote:
Again, if HEVC made financial sense for free-to-view web video, one of those services would be using it.
There are lots of ways it may not have made sense for a given free-to-view service, as above. Sometimes it doesn't make sense as the leadership of a service may not be even thinking in these directions. It's a complex thing to model! And capital investments have a much higher bar these days with VC funding trying up and interest rates so high.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 22nd August 2023, 20:00   #43  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 275
Quote:
Originally Posted by benwaggoner View Post
Facebook also has big Feels about Patents.
Please try to get this into your skull: it's not about "feels", it's about financial realities. I personally lol'ed hard at the graph on the previous page: 2 patent pools, 8 unaffiliated entities. Nice. Unless you have a reliable source from all 10 entities pledging there won't be "content fees", jumping into HEVC carries the huge risk that one or those 10 entities will come to collect content fees years after you've spent time and electricity encoding your content to HEVC. Now, if you are a pay-to-view service, those fees can be rolled into the cost the customer pays to view the video, but free-to-view services serving tons of video to anyone who asks and making tight margins per video from ads don't have that option.

Quote:
Originally Posted by benwaggoner View Post
and companies like Microsoft and IBM have extensive patent cross-licensing portfolios which should reduce or eliminate encoder/decdoer fees.
Irrelevant, unless they have patent cross-licensing portfolios with every patent holder in HEVC. The patent holders they do not have patent cross-licensing portfolios with can always come to collect content fees.

Quote:
Originally Posted by benwaggoner View Post
Real world, it gets down to the potential benefits of reduced bandwidth costs versus the extra storage costs of having multiple codecs and the complexity of encoding and managing a multi codec service. Plus if a service is live and not being hugely profitable, the instinct is to leave it as it is rather than make a big capital investment in long-term operational cost savings. And HEVC in browsers is relatively new and kinda came out of the blue. Services could be working on multi-codec but not have launched yet.

Also, a lot of services you've mentioned aren't really monetizing quality, and so reducing bitrate, even with a quality hit, is a much easier knob to control storage and bandwidth costs.
Reality is, any free-to-view service choosing to use some more modern format alongside H.264 has gone to VP9. So there is a desire to use formats more modern than H.264 by some of those services, just not HEVC. If there was some kind of desire to use HEVC by some free-to-view service, we would have heard it by now. Pay-to-view services have already done it.

Last edited by kurkosdr; 22nd August 2023 at 20:29.
kurkosdr is offline   Reply With Quote
Old 22nd August 2023, 20:10   #44  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 275
Quote:
Originally Posted by Blue_MiSfit View Post
A service I worked on called Movies Anywhere uses HEVC wherever possible. This is a free service dealing with premium Hollywood content. It was free because it was tied to the digital rights you get when you buy a title on disc or at a streaming service.

We had no HEVC licensing costs, and were comfortable taking the risk. We delivered, 4K, with HDR, Dolby Vision, Atmos, etc.
Then it's not a free service, it's part of the package the customer buys when buying the disc of streaming service in question, which means it's actually a bundled service. This means any potential content fee (I guess that's what you mean when you say "risk") can be bundled into the purchase price (the margins the Hollywood studio makes per disc or subscriber likely allow it).

Last edited by kurkosdr; 22nd August 2023 at 20:13.
kurkosdr is offline   Reply With Quote
Old 22nd August 2023, 23:07   #45  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,653
Quote:
Originally Posted by kurkosdr View Post
Please try to get this into your skull: it's not about "feels", it's about financial realities. I personally lol'ed hard at the graph on the previous page: 2 patent pools, 8 unaffiliated entities. Nice. Unless you have a reliable source from all 10 entities pledging there won't be "content fees", jumping into HEVC carries the huge risk that one or those 10 entities will come to collect content fees years after you've spent time and electricity encoding your content to HEVC. Now, if you are a pay-to-view service, those fees can be rolled into the cost the customer pays to view the video, but free-to-view services serving tons of video to anyone who asks and making tight margins per video from ads don't have that option.
You may feel there is a problem in theory, but lots of companies have adopted HEVC without any problems in practice. I concur that getting a clear straight answer from all participants is way more complicated than it should ever have been. But unless you have some examples of someone actually having to pay content licenses for free-to-play content, this is largely conjecture.

Quote:
Irrelevant, unless they have patent cross-licensing portfolios with every patent holder in HEVC. The patent holders they do not have patent cross-licensing portfolios with can always come to collect content fees.
In practice, the patent cross-licensing can have a big effect on these concerns, as it allows a whole lot of patents that could apply be ruled out. IANAL, but this seems to be the general case. There's no patent issue without a patent holder wiling to sue.

Quote:
Reality is, any free-to-view service choosing to use some more modern format alongside H.264 has gone to VP9. So there is a desire to use formats more modern than H.264 by some of those services, just not HEVC. If there was some kind of desire to use HEVC by some free-to-view service, we would have heard it by now. Pay-to-view services have already done it.
That's like saying premium content must have a big reason not to adopt VVC as no one has yet. Market timing is a big factor here. Bear in mind that VP9 has been broadly playable in web browsers for seven years now, while HEVC has been broadly supported for just shy of a year.

VP9 also got a lot of very optimistic promises made about bitrate savings over H.264 that never really paid off, largely due to x264, but also because of some poor design decisions that forced a lot of serialization for encoding and decoding. The VP9 toolchain was fine for YouTube splitting up lots of single-threaded chunks across lots of instances, but not for a lot of core x264 scenarios.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 18th September 2023, 06:40   #46  |  Link
Spyros
Registered User
 
Join Date: Jun 2019
Posts: 16
August 2023 patent list

Small update, a newer patent list is available (August 1, 2023).

I didn't notice earlier, because the website changed (from mpegla.com to via-la.com).

19 pages with unexpired patents, 37 pages with expired ones.
Spyros is offline   Reply With Quote
Old 26th September 2023, 20:32   #47  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,653
There's a few pages worth that are expiring this or next month as well. Dolby really dominates the remaining ones.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 27th September 2023, 15:29   #48  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 275
Quote:
Originally Posted by Spyros View Post
Small update, a newer patent list is available (August 1, 2023).

I didn't notice earlier, because the website changed (from mpegla.com to via-la.com).

19 pages with unexpired patents, 37 pages with expired ones.
In the US, the last patent for the first version of the H.264 standard will expire on 29 nov 2027:
https://scratchpad.fandom.com/wiki/M...#H.264_patents
So it's not worth holding your breath (when it comes to US patents).

I am interested how many EU patents remain active after 17 August 2004 (the date the first version of the H.264 standard was published), since the EU doesn't have a habit of granting multi-year patent extensions willy-nilly like the US does. For example, MPEG4 ASP is still patented in the US (and in Brazil, due to an one-time extension given to some patents in that country) but is royalty-free everywhere else.

MPEG LA updates their patents lists every 3 months btw.

Last edited by kurkosdr; 27th September 2023 at 15:36.
kurkosdr is offline   Reply With Quote
Old 4th October 2023, 23:24   #49  |  Link
Avclover
Registered User
 
Join Date: Sep 2023
Posts: 5
Quote:
In the US, the last patent for the first version of the H.264 standard will expire on 29 nov 2027
It seems the current VIA LA patent list features at least one early US patent that expires only at the end of 2030:
US9356620B2.

It is also worth noting that the ubiquitous high profile was introduced only after version 1 of AVC.
Avclover is offline   Reply With Quote
Old 5th October 2023, 18:15   #50  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 275
Quote:
Originally Posted by Avclover View Post
It seems the current VIA LA patent list features at least one early US patent that expires only at the end of 2030:
US9356620B2.

It is also worth noting that the ubiquitous high profile was introduced only after version 1 of AVC.
VIA LA's patent list includes extensions such as SVC and MVC which are technically part of the AVC standard but are not essential to implement (both SVC and MVC are backwards-compatible extensions).
https://www.via-la.com/licensing-2/avc-h-264/

You are correct about the High Profile though: The High profile was added on 1st March 2005, so that's the patent priority date to look for, I guess.
kurkosdr is offline   Reply With Quote
Old 5th October 2023, 20:06   #51  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,653
Quote:
Originally Posted by kurkosdr View Post
VIA LA's patent list includes extensions such as SVC and MVC which are technically part of the AVC standard but are not essential to implement (both SVC and MVC are backwards-compatible extensions).
https://www.via-la.com/licensing-2/avc-h-264/

You are correct about the High Profile though: The High profile was added on 1st March 2005, so that's the patent priority date to look for, I guess.
Since High mainly added 8x8 blocks, which were the pre-H.264 standard, I wonder if there are all that many essential patents for it. How to choose between and single 4x4 versus 8x8, perhaps?
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 6th October 2023, 07:21   #52  |  Link
rwill
Registered User
 
Join Date: Dec 2013
Posts: 318
Quote:
Originally Posted by kurkosdr View Post
VIA LA's patent list includes extensions such as SVC and MVC which are technically part of the AVC standard but are not essential to implement (both SVC and MVC are backwards-compatible extensions).
You write that MVC is backwards compatible. I have a question. If I write 128 as Profile IDC into a SPS to signal MVC Stereo High Profile, how is this backwards compatible to Profile IDC = 100 which is High Profile? If a Decoder knows nothing about Profile IDC = 128 or 132 and the corresponding change in syntax, would it not flat our fail to decode?

Quote:
Originally Posted by benwaggoner View Post
Since High mainly added 8x8 blocks, which were the pre-H.264 standard, I wonder if there are all that many essential patents for it. How to choose between and single 4x4 versus 8x8, perhaps?
That would be news to me that the very specific 8x8 Transform of AVC was a standard pre-H.264.
rwill is offline   Reply With Quote
Old 6th October 2023, 20:54   #53  |  Link
Avclover
Registered User
 
Join Date: Sep 2023
Posts: 5
Quote:
VIA LA's patent list includes extensions such as SVC and MVC which are technically part of the AVC standard but are not essential to implement (both SVC and MVC are backwards-compatible extensions).
While that's true, the specific patent originated in 2002, got a ridiculous term extension and is not related to SVC or MVC.

Quote:
Since High mainly added 8x8 blocks, which were the pre-H.264 standard, I wonder if there are all that many essential patents for it. How to choose between and single 4x4 versus 8x8, perhaps?
High profile also introduced quantization scaling matrices and separate C_B and C_R QP control to the specification.

Last edited by Avclover; 6th October 2023 at 21:04.
Avclover is offline   Reply With Quote
Old 6th October 2023, 22:41   #54  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,653
Quote:
Originally Posted by Avclover View Post
High profile also introduced quantization scaling matrices and separate C_B and C_R QP control to the specification.
Oh, right!

Not all encoders and bitstreams use those tools, but some certainly do, or at least can.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 6th October 2023, 23:05   #55  |  Link
Avclover
Registered User
 
Join Date: Sep 2023
Posts: 5
I guess the amount of decoders with support is what counts from a licensing perspective.

Last edited by Avclover; 6th October 2023 at 23:22.
Avclover is offline   Reply With Quote
Old 9th October 2023, 18:26   #56  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 275
Quote:
Originally Posted by rwill View Post
You write that MVC is backwards compatible. I have a question. If I write 128 as Profile IDC into a SPS to signal MVC Stereo High Profile, how is this backwards compatible to Profile IDC = 100 which is High Profile? If a Decoder knows nothing about Profile IDC = 128 or 132 and the corresponding change in syntax, would it not flat our fail to decode?
There is a simple answer to that: You won't do that, you will instead do what Blu-ray 3D wants, because it's the only use of MVC that matters (and fortunately it is backwards compatible with High profile).

And even if you don't care about Blu-ray 3D playback (say you are making MVC in MP4), then you will do whatever it takes to to not break High Profile anyway. I am sure there are multiple ways to produce a stream that's technically compliant with the latest version of the H.264/AVC standard but breaks the first High profile decoders, but don't do that if you want to get the benefits of the huge H.264/AVC installed base out there. That's true even if we are not talking about hardware players but software players. No real-world H.264/AVC stream out there breaks High Profile, not ones anyone would care about anyway.

So, as a person writing a software player, implementing the first version of the H.264/AVC standard that includes the High profile achieves compatibility with practically all real-world H.264/AVC streams out there.

Last edited by kurkosdr; 9th October 2023 at 23:19.
kurkosdr is offline   Reply With Quote
Old 9th October 2023, 18:42   #57  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,653
Quote:
Originally Posted by kurkosdr View Post
There is a simple answer to that: You won't do that, you will do what Blu-ray 3D wants, because that's the only use of MVC that matters (and is backwards compatible with High profile).
Except 3D Blu-ray is H.264, and Apple specifies using MV-HEVC. There will need to be some work to get it working for HEVC, >1080p, and for HDR. It's all pretty obvious how to do it, and the HC reference encoder includes MV-HEVC support.
Quote:
So, implementing the first version of High Profile achieves compatibility with practically all real-world H.264/AVC streams.
Yeah. Thank goodness the days of "subset of Main and Baseline + B-frames" defacto subsets are behind us.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th October 2023, 19:17   #58  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 275
Quote:
Originally Posted by benwaggoner View Post
Except 3D Blu-ray is H.264, and Apple specifies using MV-HEVC. There will need to be some work to get it working for HEVC, >1080p, and for HDR. It's all pretty obvious how to do it, and the HC reference encoder includes MV-HEVC support.
MV-HEVC has nothing to do with H.264/AVC, so it doesn't belong in this thread.
kurkosdr is offline   Reply With Quote
Old 31st October 2023, 14:28   #59  |  Link
oibaf
Registered User
 
Join Date: Nov 2008
Posts: 44
Quote:
Originally Posted by Spyros View Post
Small update, a newer patent list is available (August 1, 2023).

I didn't notice earlier, because the website changed (from mpegla.com to via-la.com).

19 pages with unexpired patents, 37 pages with expired ones.
BTW, in that list unexpired patents are:
  • US: 540
  • CN: 132
  • JP: 129
  • KR: 97
  • GB: 95
  • DE: 92
  • FR: 91
  • IT: 68
  • NL: 67
  • ES: 65
  • BR: 57
  • CA: 31
There is also an interesting analysis here: https://scratchpad.fandom.com/wiki/M...#H.264_patents : sometimes they differ in their expiration dates.

Last edited by oibaf; 31st October 2023 at 14:49.
oibaf is offline   Reply With Quote
Old 1st November 2023, 17:07   #60  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 275
It's been 3 months since August 1, 2023, so MPEG LA pushed a new list:
https://www.via-la.com/wp-content/up...tachment-1.pdf

----

Quote:
Originally Posted by oibaf View Post
BTW, in that list unexpired patents are:
  • US: 540
  • CN: 132
  • JP: 129
  • KR: 97
  • GB: 95
  • DE: 92
  • FR: 91
  • IT: 68
  • NL: 67
  • ES: 65
  • BR: 57
  • CA: 31
There is also an interesting analysis here: https://scratchpad.fandom.com/wiki/M...#H.264_patents : sometimes they differ in their expiration dates.
I may try and make a csv list or spreadsheet of the MPEG LA document sometime so I can feed it to Google Patents, but if you already have one, can you post it? Basically trying to come up with a "isn't royalty-free before" date for every country (only taking into account patents with a priority date from before March 1, 2005)

Last edited by kurkosdr; 1st November 2023 at 17:14.
kurkosdr is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:52.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.