Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd May 2023, 22:09   #1  |  Link
Spyros
Registered User
 
Join Date: Jun 2019
Posts: 16
More than half of the H.264 patents have expired

Just something I noticed on the official MPEG LA website.

The previous patent list (February 21, 2023) listed 32 pages with unexpired patents and 23 pages with expired ones.

The latest patent list (May 1, 2023) lists 24 pages with unexpired patents and 32 pages with expired ones.

Yes, I know, quick-and-dirty calculation. And that percentage doesn't change anything, all the patents (or almost all of them) must expire in order to use the codec freely. But that day is getting closer.
Spyros is offline   Reply With Quote
Old 6th August 2023, 14:59   #2  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 315
Keep in mind that MPEG LA adds the patents essential to implementing the backwards-compatible extensions SVC (Scalable Video Coding) and MVC (Multiview Video Coding) to the list of patents, so the essential patents required to decode AVC video ignoring those extensions will expire sooner.

The best I have found is this list, but it's only for US patents:
https://www.osnews.com/story/24954/u...3-mpeg-2-h264/
kurkosdr is offline   Reply With Quote
Old 6th August 2023, 21:30   #3  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,908
Nice. Given how widespread H.264 already is, it might make things even easier.
Perhaps H.264 is gonna stay with us for a little longer ehehehehehe
FranceBB is offline   Reply With Quote
Old 8th August 2023, 20:09   #4  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,771
Quote:
Originally Posted by FranceBB View Post
Nice. Given how widespread H.264 already is, it might make things even easier.
Perhaps H.264 is gonna stay with us for a little longer ehehehehehe
H.264 is already a particularly long-lived codec, and will be around for some years to come.
I can imagine a streaming service viably dropping H.264 entirely in maybe 2-3 years. >90% of player devices support HEVC today, so it's really older long-tail things that don't. And H.264 will remain reliably compatible far into the future.

H.264 was really a revolution and revelation, the first to truly break out of the H.261 model with lots of fundamentally new approaches to encoding. Multiple reference frames! 4x4 intra blocks! In-loop deblocking! Arithmetic entropy encoding!

Its timing with the birth of the ripping/warez/torrent community was also fortuitous, as that lead to x264. Past professional encoders largely refined reference encoders, focused on PSNR, and started with a similar basic set of test clips. x265 had a whole different target audience and developer goals as an open source project. There were thousands of people across the world throwing all kinds of wildly variable content at it, tuning and testing and sharing results like crazy. Subjective quality was all that matters, with PSNR and SSIM afterthoughts at best. Animation was a first-class content category (a BIG blind spot in classic codec development). Crazy ideas like MBTree were tried. Bad ones got discarded quickly, and the good ones refined.

And real-world performance mattered; groups were competing to get the best quality at the smallest file size published before everyone else. Optimization and multithreading were pushed farther than anyone had before.

x264 kept raising the bar for H.264 encoders, and encoders in general. Anyone who wanted to sell a H.264 encoder had to compete with the free x264, so lots of bitstreams and code were analyzed, and techniques ported over or at least caused inspiration.

All of that meant that H.264 kept getting better, longer, than almost any other codec (MPEG-2 as the arguable exception). H.264 wound up being more efficient for important use cases than the people who made the standard ever predicted.

HEVC got a big boost from x264 as well, as MultiCoreWare licensed the source code and started x265 as a mash-up of the H.265 reference encoder and x264, and then went from there.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th August 2023, 00:20   #5  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,908
Quote:
Originally Posted by benwaggoner View Post
I can imagine a streaming service viably dropping H.264 entirely in maybe 2-3 years. >90% of player devices support HEVC today
I don't really think they will, unless they're gonna use something like AV1 as a backwards-compatibility option.
The issue being that H.265 is still struggling in browsers compared to H.264, so much so that unless the user has a GPU with its own hardware decoder, it won't be able to play it (and this is definitely not guaranteed).
Although lots of people watch those contents on either a smartphone (if they're on the go) or with their TV (if they're at home) where none of this is a problem, there are still some folks watching contents on a PC via the browser (especially if it's a laptop and they're, dunno, on a business trip or whatever). As much as it would be nice to have H.265 replace H.264 once and for all, I don't really see that happening. The only feasible replacement for H.264 would (probably) be AV1 as it has a totally free software decoding.

The "big boys" in terms of streaming services can innovate and afford to have multiple different encodes of the same content in different codecs to achieve the best quality and that's fine.
If we look at smaller, non global, country-specific services, however, they all use the same fatal combination: H.264 + AAC as the only thing that they're sure works everywhere.

- BBC iPlayer -> H.264 + AAC
- ITVX -> H.264 + AAC
- All 4 -> H.264 + AAC

The list goes on, but essentially they're all using the H.264 + AAC combo displaying SDR contents only when you watch them on a browser. For instance, even the BBC, which offers HDR HLG BT2020 contents via the iPlayer, doesn't really offer them via browser but rather only via the installed app on the devices they know they can support.

Last edited by FranceBB; 9th August 2023 at 00:22.
FranceBB is offline   Reply With Quote
Old 9th August 2023, 00:56   #6  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
Google ships a software HEVC decoder in Chrome now (though without Widevine DRM support), so the browser argument against HEVC is winding down

Safari of course has full HEVC support. That leaves the small footprint of Firefox and others.
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Old 9th August 2023, 18:25   #7  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,771
Quote:
Originally Posted by Blue_MiSfit View Post
Google ships a software HEVC decoder in Chrome now (though without Widevine DRM support), so the browser argument against HEVC is winding down
Yeah, 18 months ago Chrome was the single biggest platform where HEVC couldn't be used. And it had AV1 support, so there was a better-than-H.264 alternative. Clearly that wasn't enough to justify broad adoption of AV1 for premium content that requires DRM. YouTube still remains the primary distributor of AV1 bitstreams.

Quote:
Safari of course has full HEVC support. That leaves the small footprint of Firefox and others.
Edge is Chromium derived, and I believe supports HEVC correctly (again, finally) as well. So that's the top three browsers by usage. FireFox has dropped below 6% on Windows/Mac now.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th August 2023, 20:44   #8  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,908
Quote:
Originally Posted by Blue_MiSfit View Post
Google ships a software HEVC decoder in Chrome now (though without Widevine DRM support), so the browser argument against HEVC is winding down
Ah! I didn't know that!
Nice, very nice. About time.
So now it just needs DRM and the game is on.


By the way, I was trying to play with: https://reference.dashif.org/dash.js...yer/index.html
I used the following stream: https://dash.akamaized.net/dash264/T.../MultiRate.mpd
But all I could see was dark:




Google Chrome 116.0.5845.42 (Official Build) beta (64-bit)
Revision d6ccfdce8c8e575e0a33c92686a29707d0dd4e4f-refs/branch-heads/5845@{#571}

Are you guys sure it supports software decoding and it's not just hardware decoding using the GPU decoder?
Have you actually got it working with software decoding?

Last edited by FranceBB; 9th August 2023 at 20:54.
FranceBB is offline   Reply With Quote
Old 9th August 2023, 11:43   #9  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Safari of course has full HEVC support.
So far as I know, Safari version is same as IOS version (at least on iPad), and so HEVC support relies on IOS version,
but which version has the full support ?

EDIT:
Dont think my iPad 4 supports (IOS v10.3.3),
iPad mini 4 not sure, (recent updated to IOS v15.7.7 due to security bug:- https://forum.doom9.org/showthread.p...79#post1989479 )
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 9th August 2023 at 11:50.
StainlessS is offline   Reply With Quote
Old 9th August 2023, 17:28   #10  |  Link
Ritsuka
Registered User
 
Join Date: Mar 2007
Posts: 95
The A8 chip should support HEVC, A9 surely does.
Ritsuka is offline   Reply With Quote
Old 9th August 2023, 19:49   #11  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
Indeed, I just tested hardware accelerated HEVC in Edge with multi-key PlayReady. Yay!

Now... if only Google would get this hooked up on Chrome.
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Old 12th August 2023, 13:06   #12  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 315
H.264 was the only case of the patent pools pulling their heads out of their posteriors and deciding to not charge "content fees" for free-to-view web content. All the other ISO/ITU video formats have been failures on the web mainly because of the "content fee", with websites opting for literally anything else than the ISO/ITU formats. The only exception is some subscription services using H.265, and even then some subscription services have opted for AV1 instead.

So, even if the problem of the decoder fees was solved for H.265 (which is a much bigger problem than it was for H.264 due to the stacking of multiple patent pool royalties), so that browsers can support H.265 reliably without dependence on the particular hardware configuration they run on, H.265 also has the problem of the "content fee".

The only viable formats for the web are VP8, H.264, VP9, and AV1. VP9 and AV1 have a non-zero possibility of a successful patent assertion, so the only 100% safe choice is VP8 and H.264, and since H.264 is better than VP8 and also universally supported by hardware acceleration, it's gonna be with us for a long time. Also, any attempt to replace it is not worth the effort since it's going to be royalty-free before any attempt to replace it can succeed.

Last edited by kurkosdr; 12th August 2023 at 17:01.
kurkosdr is offline   Reply With Quote
Old 15th August 2023, 01:58   #13  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,771
Quote:
Originally Posted by kurkosdr View Post
H.264 was the only case of the patent pools pulling their heads out of their posteriors and deciding to not charge "content fees" for free-to-view web content. All the other ISO/ITU video formats have been failures on the web mainly because of the "content fee", with websites opting for literally anything else than the ISO/ITU formats. The only exception is some subscription services using H.265, and even then some subscription services have opted for AV1 instead.
I'm not aware of any premium services are using AV1 for even 10% of their streams, or for UHD and HDR at all. I could be wrong.

Content fees haven't wound up being a problem in practice, but the ambiguity until things have clarified sure have (MPEG-4 pt 2 in particular).

Quote:
So, even if the problem of the decoder fees was solved for H.265 (which is a much bigger problem than it was for H.264 due to the stacking of multiple patent pool royalties), so that browsers can support H.265 reliably without dependence on the particular hardware configuration they run on, H.265 also has the problem of the "content fee".
I think the decoder fee has been solved for H.265. It is pretty well universal in mobile, TV, and attached devices. Web browsers were really the last big gap. Certainly HDR content can be done in just HEVC.

Quote:
The only viable formats for the web are VP8, H.264, VP9, and AV1. VP9 and AV1 have a non-zero possibility of a successful patent assertion, so the only 100% safe choice is VP8 and H.264, and since H.264 is better than VP8 and also universally supported by hardware acceleration, it's gonna be with us for a long time. Also, any attempt to replace it is not worth the effort since it's going to be royalty-free before any attempt to replace it can succeed.
"Viable" in what sense? UGC HEVC is now viable on the web with Safari + Chrome + Edge, with HW decoder support broadly available.

I concur that as of August 2021, if you can only encode to one codec, H.264 is the obvious choice. But for more complex efforts where multi-codec is viable, <20% of clients would get H.264 instead of something better.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 13th August 2023, 06:23   #14  |  Link
Ritsuka
Registered User
 
Join Date: Mar 2007
Posts: 95
Do you have a link to where those "content fees" are defined?
Ritsuka is offline   Reply With Quote
Old 13th August 2023, 09:58   #15  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 326
Quote:
Originally Posted by Ritsuka View Post
Do you have a link to where those "content fees" are defined?
AFIK it was removed in 2018 (except for physical media e.g. Bluray).

Last edited by excellentswordfight; 13th August 2023 at 10:01.
excellentswordfight is offline   Reply With Quote
Old 13th August 2023, 18:55   #16  |  Link
Snap
Member
 
Snap's Avatar
 
Join Date: Jan 2018
Posts: 17
Quote:
Originally Posted by excellentswordfight View Post
AFIK it was removed in 2018 (except for physical media e.g. Bluray).
Yeah. I saved the page/link of an article covering it at the time:

http://www.streamingmedia.com/Articl...ng-123828.aspx
Snap is offline   Reply With Quote
Old 14th August 2023, 04:38   #17  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 315
You are aware that HEVC Advance is only one of the several patent pools that you need to license patens from to implement HEVC, right? If one of the patent pools charges a "content fee" for streaming, then HEVC has a content fee for streaming

Last edited by kurkosdr; 14th August 2023 at 04:40.
kurkosdr is offline   Reply With Quote
Old 14th August 2023, 08:24   #18  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 326
Quote:
Originally Posted by kurkosdr View Post
You are aware that HEVC Advance is only one of the several patent pools that you need to license patens from to implement HEVC, right? If one of the patent pools charges a "content fee" for streaming, then HEVC has a content fee for streaming
Im fully aware, but non of the other holders do that, I work at a broadcast/streaming company and I havnt heard anyone talk about this fee for years, and I can find multiple industry sources stating that there are no fees;

from harmonic:

"we estimate that HEVC still has large growth potential, especially as streaming fees are waived from patent pools."

https://www.harmonicinc.com/insights...-and-streaming

From Tom Vaughan (ex VP for MulticoreWare)

"Again, keep in mind that no one is asking for patent license fees for content distribution (streaming, etc.), except for UHD-Blu-ray disc (a small per-disc fee to HEVC advance). Only hardware device manufacturers need to license HEVC patents, and they are dealing with that issue and they continue to support HEVC in every device they make that supports video. For video services, HEVC is free."

https://forum.doom9.org/showthread.p...84#post1864184

So what are your sources for this? Cause it seems like you know something we dont.

Last edited by excellentswordfight; 14th August 2023 at 12:19.
excellentswordfight is offline   Reply With Quote
Old 15th August 2023, 14:13   #19  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 315
Quote:
Originally Posted by benwaggoner View Post
I'm not aware of any premium services are using AV1 for even 10% of their streams, or for UHD and HDR at all.
Netflix streams AV1 if the device supports it

Quote:
Originally Posted by excellentswordfight View Post
So what are your sources for this? Cause it seems like you know something we dont.
I read somewhere that MPEG LA used to charge content fees, but that may be outdated. Anyway, unless there is a reliable source for every patent pool (that they have waived content fees), I don't believe it, there are just too many patent pools and entities getting their beak wet from H.265, and every one has to agree for H.265 to be free of content fees.

Quote:
Originally Posted by benwaggoner View Post
I think the decoder fee has been solved for H.265. It is pretty well universal in mobile, TV, and attached devices. Web browsers were really the last big gap. Certainly HDR content can be done in just HEVC.
It hasn't been solved for browsers, since the business model of the entire industry revolves around giving away the browser for $0. Just because a company like Google begrudingly pays the H.264 royalties for Chrome (and Cisco for Firefox) doesn't mean they will do it again for HEVC. Chrome "solves" the issue by using HEVC hardware acceleration if supported by the system (and not decoding HEVC if not), which means that as a service provider you will have to also encode in H.264 or VP8. Which is why if you can't encode and store multiple formats (on top of encoding and storing multiple resolutions that you have to do anyway), H.264 and VP8 are your only viable formats. But if you can encode in multiple-formats, then every format can be said to be viable, even something like VVC+Dolby Vision can be said to be viable if there is an H.264 or VP8 stream to fall back to.

Last edited by kurkosdr; 15th August 2023 at 18:33.
kurkosdr is offline   Reply With Quote
Old 16th August 2023, 18:17   #20  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,771
Quote:
Originally Posted by kurkosdr View Post
Netflix streams AV1 if the device supports it
For HDR as well? If so, I wasn't aware! Do you have any links?

Quote:
I read somewhere that MPEG LA used to charge content fees, but that may be outdated. Anyway, unless there is a reliable source for every patent pool (that they have waived content fees), I don't believe it, there are just too many patent pools and entities getting their beak wet from H.265, and every one has to agree for H.265 to be free of content fees.
MPEG-LA has had some commercial content fees before, but they've been reasonable. The bigger friction has been from ambiguity before the patent pool(s) have finalized and published details. This was more complex for HEVC which had three pools, not all of which published any license details and required an attorney and NDA to get the specifics.

Quote:
It hasn't been solved for browsers, since the business model of the entire industry revolves around giving away the browser for $0. Just because a company like Google begrudingly pays the H.264 royalties for Chrome (and Cisco for Firefox) doesn't mean they will do it again for HEVC. Chrome "solves" the issue by using HEVC hardware acceleration if supported by the system (and not decoding HEVC if not), which means that as a service provider you will have to also encode in H.264 or VP8. Which is why if you can't encode and store multiple formats (on top of encoding and storing multiple resolutions that you have to do anyway), H.264 and VP8 are your only viable formats. But if you can encode in multiple-formats, then every format can be said to be viable, even something like VVC+Dolby Vision can be said to be viable if there is an H.264 or VP8 stream to fall back to.
It has been "solved" in the sense that HEVC decode hardware is now ubiquitous in most markets, and a free browser can pass a HEVC bitstream to a system-level SW or HW decoder (just like with AV1). There will be some edge cases without a HEVC decoder, of course, so full ubiquity still requires H.264 as a fallback.

For premium content requiring hardware DRM, just HEVC is more viable because the systems without HEVC HW decoders rarely have sufficiently protected HW DRM. Same with HDR, as the 10-bit requirement rules out most H.264 decoders in systems without HEVC decoders.

I am using HEVC as an illustrative example here, as the same market dynamics apply to AV1, AV2, VVC, and on into the future. I doubt we'll see any major content platform adopt a single codec solution other than H.264 in the next decade.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.