Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > VP9 and AV1

Reply
 
Thread Tools Search this Thread Display Modes
Old 7th February 2019, 19:57   #1441  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,218
Quote:
Originally Posted by TD-Linux View Post
Aha, looks like 601 vs 709 matrix. Although JPEG is normally sRGB primaries, it uses what is basically a full-range 601 matrix. So if your sources are JPEG, a 601 matrix makes the most sense.
sRGB uses 709, which itself is the average of the 601 PAL (EBU 3213) and NTSC (SMPTE C) primaries. As an industry, we should probably stop talking about "601 primaries" since there are actually two different ones, unless we use it as shorthand for "the primaries used by the original SD video format."

709 was the compromise for HD to make it "international" - as the average of the two, if 601 gets treated as 709 or vise versa, that minimizes the worst-case error compares to 601 PAL <> 601 NTSC.

https://en.wikipedia.org/wiki/Rec._7...chromaticities

As a parochial American, I thought 709 was dumb when it came out, but I have since gained the wisdom to appreciate its simple brilliance.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 7th February 2019, 22:44   #1442  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 194
benwaggoner you just mentioned AV2 several times in the EVC thread on February 1st. Do you know where current work on AV2 is being committed to if it is public yet? The googlesource.com git site leaves something to be desired as far as usability and search is concerned.
soresu is offline   Reply With Quote
Old 8th February 2019, 21:45   #1443  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,218
Quote:
Originally Posted by soresu View Post
benwaggoner you just mentioned AV2 several times in the EVC thread on February 1st. Do you know where current work on AV2 is being committed to if it is public yet? The googlesource.com git site leaves something to be desired as far as usability and search is concerned.
I've heard from some people that they are doing some initial work on it, and the hope is that it will be a relatively quick turnaround.

One potential wrinkle to the VPx and AVx codecs is that they know in advance when essential patents are going to expire, so tools could be designed in advance and only deployed when IP is cleared up. So there could be stuff that was too early for AV1 that could be reused. That's just my own personal speculation, though. But that could speed some things up.

People looking at AV2 have also been a lot more optimistic about it than AV1, which didn't get enough attention to HW decoder design optimization, or getting tools to work together orthogonally. One comment I heard is that one tool might be sharpening a pixel while another is smoothing it.


2020 should be an interesting year in the codec space, with AV2, VVC, and EVC all potentially being far enough along to evaluate, and H.264, HEVC, and AV1 still competing for current deployments. After UHD, HDR, HFR, and object-based audio all launching in 2014-2016, it's been a little dull around new media technologies. So I'm pretty amped by all the exciting fun 2020-2022 is going to be for codecs! It'll be an interesting mix of technical, business, and legal factors, and I really don't have a guess yet about what the codec world will look like in five years*!

And audio stuff is heating up with xHE-AAC, AC-4 with Atmos, and MPEG-H all going mainstream.

* Well, I bet we'll still be using MP4 as a container format.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 8th February 2019, 22:47   #1444  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 194
Is there any word on Daala techniques like PVQ and Activity Masking, and also ANS going into AV2?

Though from the direction of VVC and Google's own encoding research priorities - I could see a more than healthy dose of machine learning put to use in AV2 aswell. ML seems to be affording some very significant complexity/efficiency gains in the area of Path Tracing, and I'm sure all of the involved AOM parties would cheer improvements in encoding complexity.
soresu is offline   Reply With Quote
Old 10th February 2019, 18:10   #1445  |  Link
IgorC
Registered User
 
Join Date: Apr 2004
Posts: 1,319
Quote:
Originally Posted by benwaggoner View Post
2020 should be an interesting year in the codec space, with AV2, VVC, and EVC all potentially being far enough along to evaluate, and H.264, HEVC, and AV1 still competing for current deployments. After UHD, HDR, HFR, and object-based audio all launching in 2014-2016, it's been a little dull around new media technologies. So I'm pretty amped by all the exciting fun 2020-2022 is going to be for codecs! It'll be an interesting mix of technical, business, and legal factors, and I really don't have a guess yet about what the codec world will look like in five years*!
This statement is beyond of a healthy optimism.
The market of video codecs is cooling down. There are few reasons for that. Royalty free formats start to gain share and some external factors like a big improvement of network bandwidth especially during last years.

Quote:
Originally Posted by benwaggoner View Post
And audio stuff is heating up with xHE-AAC, AC-4 with Atmos, and MPEG-H all going mainstream.

I don't know where You get this information from but this is not what happens with audio codecs lately.
xHE-AAC has nothing to do with mainstream. It's a low bitrate codec and companies adopt it only where bandwidth is very scarce. xHE-AAC/AC4 has no advantage over AAC (22 years old format) at 96-128+ kbps. Audio formats are mature at this point.

AC3 patents have expired in 2017 while LC-AAC's will be expired during 2019-2020. It will be imposible to force some company to use new codec when there are AC3 and LC-AAC with expired patents. Giant streaming platforms, Netflix and Youtube, use AAC and Opus. They don't plan to use any new audio codecs in near future.

Plus there is no one single developer team working on xHE-AAC, MPEG-H or AC4 audio codec. And xHE-AAC isn't actually a new format. It's a standard since 2012. Where its development? Adoption?

Last edited by IgorC; 10th February 2019 at 18:21.
IgorC is offline   Reply With Quote
Old 10th February 2019, 23:21   #1446  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,947
Ultra low bitrate is highly desirable for a few specific use cases for companies delivering video:

1) Countries with extremely poor (~2G, to maybe 3G at best) cellular connectivity. Delivering even good quality SD video is totally acceptable here. Total bit budget is often like 200 - 300 Kbps though, so you really do need to use the lowest bitrate audio you can possibly use. 96 Kbps for stereo AAC is not feasible. Opus is great here, but it doesn't have universal support, so more development into other formats is absolutely welcome.

2) Download / offline playback. Imagine you're at the airport about to board a flight. You forgot to download something to watch on your phone / tablet during the flight! You want to be able to download a movie or a couple episodes of a series as quickly, probably using over-crowded WiFi or cellular connectivity. See above.
Blue_MiSfit is offline   Reply With Quote
Old 11th February 2019, 03:14   #1447  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 194
IgorC,

I dont know about XHE-AAC, but I heard that UK FreeSAT chose AC-4 as the audio format for its next evolution. Link here.
Other platforms supporting it are shown, aswell as multiple hardware partners (Broadcom, Cadence, HiSilicon,
Mediatek, MStar Semiconductor,
Novatek and Realtek).
soresu is offline   Reply With Quote
Old 11th February 2019, 04:14   #1448  |  Link
IgorC
Registered User
 
Join Date: Apr 2004
Posts: 1,319
Quote:
Originally Posted by Blue_MiSfit View Post
1) Countries with extremely poor (~2G, to maybe 3G at best) cellular connectivity. Delivering even good quality SD video is totally acceptable here. Total bit budget is often like 200 - 300 Kbps though, so you really do need to use the lowest bitrate audio you can possibly use. 96 Kbps for stereo AAC is not feasible.
This is not true.
Look at the report https://opensignal.com/reports-data/...nal_201811.pdf

The modest mobile and/or fixed connections are about ~2-3 Mbps (in Algeria). Far from yours 200-300 kbps.

Generally people have a wrong idea that every county in Africa, Asia and Latin America (where I live actually) has very bad internet connection.
Here in Latin America I get 10+ Mbps on 4g/LTE+/4G+.
And Indians are mad about their "slow" 4G connection. It's "just" 6 Mpbs! https://www.indiatimes.com/technolog...ps-340086.html

https://ispspeedindex.netflix.com/country/india/

Do You still think "96 Kbps for stereo AAC is not feasible" and "India is so 2G", right?

Quote:
Originally Posted by Blue_MiSfit View Post
Ultra low bitrate is highly desirable for a few specific use cases for companies delivering video:
Yes, corner cases. Not mainstream as Ben claims.

Quote:
Originally Posted by Blue_MiSfit View Post
Opus is great here, but it doesn't have universal support, so more development into other formats is absolutely welcome.
Opus is used in Youtube, an endless number of VoIP and telephone clients including Cisco corporate solutions like Webex, Skype etc. And it is supported by large number of platflorms including Android and iOS https://caniuse.com/#search=opus

So are You sugesting to use something better like xHE-AAC which doesn't even has one single available encoder? Oh, nice. That will do.


Quote:
Originally Posted by Blue_MiSfit View Post
2) Download / offline playback.
What is wrong with current VP9, H.264, H.265, Opus and HE/AAC codecs?
xHE-AAC isn't any better than HE/AAC, Opus at 96 kbps, which is already low bitrate. https://www.ietf.org/lib/dt/document...N/file1298.doc



Quote:
Originally Posted by soresu View Post
IgorC,

I dont know about XHE-AAC, but I heard that UK FreeSAT chose AC-4 as the audio format for its next evolution. Link here.
Other platforms supporting it are shown, aswell as multiple hardware partners (Broadcom, Cadence, HiSilicon,
Mediatek, MStar Semiconductor,
Novatek and Realtek).
Great. Both xHE-AAC and AC4 have similar quality as they have the same/similar compression tools. So AC4 has an advantage but only on low bitrate as well. It makes sense to use it where BW is expensive like digital radio DRM but I won't expect it to see on internet platforms like Netflix, YouTube (Opus AAC), Spotify (AAC 128-256k, Vorbis 96/160/320l), Apple Music (AAC 256k), Tidal (96-256 kbps AAC and lossless FLAC) etc.

Last edited by IgorC; 11th February 2019 at 04:27.
IgorC is offline   Reply With Quote
Old 11th February 2019, 04:45   #1449  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 194
As someone who comes from a village in northern England, I can tell you that it only just got upgraded to VDSL from the 3 mbps ADSL 2 it had been at for 5-8 years.

Thankfully it is only 1.5-2 miles from the closest exchange, but many rural communities are much further out than that and still lack the FTTC/VDSL upgrades that have existed near the exchanges for over half a decade (therefore stuck with ultra low ADSL data rates). Expensive 4G mobile broadband data is sadly a bad option if you plan to consume any significant amount of video per month.

All this adds up to the fact that low/ultra low bitrate video is far from corner case, even in first world countries - mainly because rural areas being lower population density are treated like third world countries by BT/Open Reach.

It wouldnt surprise me to find out that many rural places in Europe and the US suffer from similarly slow uptake of landline fibre based broadband technologies.
soresu is offline   Reply With Quote
Old 11th February 2019, 07:30   #1450  |  Link
TomV
VP Eng, Kaleidescape
 
Join Date: Jan 2018
Location: Mt View, CA
Posts: 51
Quote:
Originally Posted by IgorC View Post
This is not true.
Look at the report https://opensignal.com/reports-data/...nal_201811.pdf
Igor, you're arguing with 2 technical professionals who are key members of their respective Tier 1 companies... Amazon and Disney. They have access to much better insights on end-user bandwidth and client device capabilities than you or I. These companies will license proprietary codecs like Dolby AC-4 or xe-AAC if and when that makes sense. Software audio decoding is certainly feasible on most devices, especially at very low bit rates (when audio is also likely mixed to one channel).

I'm glad you have decent bandwidth in Latin America. In many developing areas of the world, bandwidth is still scarce and expensive. And even if mobile networks have been upgraded, that end-customer that a video streaming service is trying to take care of may still have an older device.
TomV is offline   Reply With Quote
Old 11th February 2019, 07:44   #1451  |  Link
iwod
Registered User
 
Join Date: Apr 2002
Posts: 757
Quote:
Originally Posted by soresu View Post
As someone who comes from a village in northern England, I can tell you that it only just got upgraded to VDSL from the 3 mbps ADSL 2 it had been at for 5-8 years.

Thankfully it is only 1.5-2 miles from the closest exchange, but many rural communities are much further out than that and still lack the FTTC/VDSL upgrades that have existed near the exchanges for over half a decade (therefore stuck with ultra low ADSL data rates). Expensive 4G mobile broadband data is sadly a bad option if you plan to consume any significant amount of video per month.

All this adds up to the fact that low/ultra low bitrate video is far from corner case, even in first world countries - mainly because rural areas being lower population density are treated like third world countries by BT/Open Reach.

It wouldnt surprise me to find out that many rural places in Europe and the US suffer from similarly slow uptake of landline fibre based broadband technologies.
I know this is slightly off topic, but I can assure you, comparatively speaking BT isn't doing such a bad job at rural areas. They are actively investing into G.Fast and VDSL 35b. One of the earliest implementor of ADSL2+, ( That is up to 5000M from exchange ). The future is that once 5G matures, setting up Gigabits wireless network using Microwave as backbone should be way cheaper than layering out fibre. So I am optimistic in rural area's broadband.

But yes, ultra low bitrate ( Sub 1Mbps ) is still required in many places, especially if you are doing video which is hogging a lot of the capacity. There is a huge capacity difference between constantly hanging on to a 1Mbps Data stream than doing once in a while 6Mbps Speed test.

So hopefully as both Network Technologies improves and Compression improves, the long tail of world population can all enjoy online streaming video within the next decade. I just hope future codec focus more on sub 2-4Mbps bitrate,
iwod is offline   Reply With Quote
Old 11th February 2019, 12:20   #1452  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 1,054
Quote:
Originally Posted by iwod View Post
They are actively investing into G.Fast and VDSL 35b
BT are using VDSL2-17A Annex B not 35b.
hajj_3 is offline   Reply With Quote
Old 11th February 2019, 13:32   #1453  |  Link
IgorC
Registered User
 
Join Date: Apr 2004
Posts: 1,319
Quote:
Originally Posted by TomV View Post
Igor, you're arguing with 2 technical professionals who are key members of their respective Tier 1 companies... Amazon and Disney. .
That's really good. Then they should know that 200-300 kbps is a far from reality.
I have provided a study with real numbers of world bandwidth and I'm myself network specialist who has an information what's going on with ISPs and how xHE-AAC (ultra low bitrate audio format) will change very little if anything as it has already happened with MPEG Surround (standard since 2007). All those professionals were very positive how this MPEG Surround will save bandwidth to million people. Has it?

Last edited by IgorC; 11th February 2019 at 13:40.
IgorC is offline   Reply With Quote
Old 11th February 2019, 16:01   #1454  |  Link
Phanton_13
Registered User
 
Join Date: May 2002
Posts: 95
I don't think that 200-300 kbps is that far from reality, specially in a mobile phone as even through my phone speed is typilally +30mbps is not thar rare the occasion when it drops down to 1000-500 kbps, basicallly drops in signal quality or being in a overpopulate cell. On the other part more than bandwidth what it save is data usage and as most mobile connections are billed by data usage not by bandwidth this has economical advantage for the user.

I also admit that xhe-aac is mainly a letdown that have seen no real adoption beyond DRM and is understandable for more than one reason.
Phanton_13 is offline   Reply With Quote
Old 11th February 2019, 19:23   #1455  |  Link
TomV
VP Eng, Kaleidescape
 
Join Date: Jan 2018
Location: Mt View, CA
Posts: 51
Again, the people who run worldwide video streaming services know exactly how many customers are bandwidth limited, and they know which of the many Adaptive Bit Rate renditions (tiers) their customers are streaming. When customers can't sustain higher bit rates, the client player application requests the lowest bit rate rendition from the server. The video service provider has logs of all of this activity, so they know what % of streams are at the lowest bit rate tier, and where/when this happens. Some have tiers as low as 100 kbps (including about 10 kbps for mono audio). With HEVC, this tier is watchable, even if the picture size is reduced to 320x240. It's not possible at all with AVC. For those in rural parts of the world where the best they can get is a 2G mobile connection, they're thrilled to get super-low bit rate video, as long as they can tell what's happening, and they don't have to wait for buffering too often. It sure beats what they used to have, which was no video, or extremely long buffering times.
TomV is offline   Reply With Quote
Old 11th February 2019, 19:53   #1456  |  Link
IgorC
Registered User
 
Join Date: Apr 2004
Posts: 1,319
Quote:
Originally Posted by TomV View Post
Again, the people who run worldwide video streaming services know exactly how many customers are bandwidth limited,
So do network engineers know (in fact even better)

Again, where is xHE-AAC adoption? It's as old as HEVC. HEVC was adopted, xHE-AAC wasn't.
If ultra low bitrates are so important why nobody hurries to adopt this codec?

Name me just one relatively large broadcasting company who has adopted it.
Name me just one developer team (not Fraunhofer themselves) who actually developing xHE-AAC encoder in this moment.

Can You do it?
Because if You can't I don't see a reason to keep this dicussion.


Quote:
Originally Posted by TomV View Post
The video service provider has logs of all of this activity, so they know what % of streams are at the lowest bit rate tier, and where/when this happens. Some have tiers as low as 100 kbps... (including about 10 kbps for mono audio).
Are You sure this number isn't very small? 0.1%, 1-2% ?
Can You present any statistics?

Last edited by IgorC; 11th February 2019 at 19:56.
IgorC is offline   Reply With Quote
Old 11th February 2019, 20:43   #1457  |  Link
Nintendo Maniac 64
Registered User
 
Nintendo Maniac 64's Avatar
 
Join Date: Nov 2009
Location: Northeast Ohio
Posts: 446
So um, as someone on the outside, I've got to ask - outside of DRM and satisfying "not invented here" syndrome, what benefit does xHE-AAC provide over something like Opus?
__________________
____HTPC____  | __Desktop PC__
2.93GHz Xeon x3470 (4c/8t Nehalem) | 4.5GHz 1.24v dual-core Haswell G3258
Radeon HD5870  | Intel iGPU      
2x2GB+2x1GB DDR3-1333 | 4x4GB DDR3-1600       
Nintendo Maniac 64 is offline   Reply With Quote
Old 11th February 2019, 21:25   #1458  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,218
Quote:
Originally Posted by Nintendo Maniac 64 View Post
So um, as someone on the outside, I've got to ask - outside of DRM and satisfying "not invented here" syndrome, what benefit does xHE-AAC provide over something like Opus?
It supposedly offers somewhat better quality at very low bitrates. I've not seen a detailed double-blind listening test to validate that, though.

I think xHE-AAC is going to become more broadly supported on platforms, out of momentum. AAC licensees now get access to xHE-AAC for free so it's a trivial effort to roll in xHE-AAC support with platforms updates. And use of AAC in MPEG-4 streams is broadly understood and implemented. Opus does have a mapping, but I've not seem much use of it. MPEG-4 as a file format is more dominant than H.264 was; the Matroska based container formats aren't a significant player for commercially distributed content.

The biggest recent news is that Android Pie has xHE-AAC built in.

xHE-AAC also offers gapless switching between bitrates without having to do overlap decoding. Does Opus support that. I consider this a significant advantage of xHE-AAC over HE v1/v2 and LC, which can only do gapless switching within v1 or v2, but not between. xHE-AAC can switch from very low bitrates to very high quality, which wasn't feasible before.

And don't knock how critical DRM is for premium content. It's a Very Big Deal. And there are SoC reasons why mixing encrypted video and unencrypted audio can be problematic.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 11th February 2019, 21:48   #1459  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,947
^ DRM is absolutely positively mandatory - no two ways about it. You simply will not get the rights to distribute content if you don't have approved DRM implementations, and this is extremely specific e.g hardware implementations of PlayReady, Widevine with progressive restrictions to unlock HD or UHD content.

I don't love DRM, but it's just table stakes when you're delivering premium content. There's no way around it, so the best we can do is make it as unobtrusive as possible!

Regarding the 200 - 300 Kbps scenario - another use case would be rural customers with satellite or very poor cell service. I grew up in a very small town and many of my friends live outside the city limits where there simply is no broadband. Not even DSL, though you could maybe get an ISDN or T1 line if you're a masochist

You might get one bar of LTE (or two if you stand in exactly the right place), and you're sharing this one solitary tower with your neighbors, so during peak times you're lucky to get 1 Mbps, assuming you're not over your data cap, at which point you drop down to under 500 Kbps. Satellite can be fast, but generally is very over-sold in these areas, and cannot keep up with demand during peak times. It also has extreme data caps, and the throttled speed is extremely slow.

Anyway - hoping we can get back on topic - AOM codec discussion

Maybe viewing through the ultra low bitrate lens, has anyone done very low bitrate 2 pass VBR tests with AV1? I'd be interested to see how that might fit into the above scenario regarding downloading content for offline playback. This is a neat scenario because you get the bonus of being able to skip all the compromises one must make when encoding for adaptive bitrate delivery and can use very large buffers (vbv-maxrate + vbv-bufsize) and longer adaptive keyframe intervals.

Last edited by Blue_MiSfit; 11th February 2019 at 21:50.
Blue_MiSfit is offline   Reply With Quote
Old 11th February 2019, 22:29   #1460  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,218
Quote:
Originally Posted by Blue_MiSfit View Post
^ DRM is absolutely positively mandatory - no two ways about it. You simply will not get the rights to distribute content if you don't have approved DRM implementations, and this is extremely specific e.g hardware implementations of PlayReady, Widevine with progressive restrictions to unlock HD or UHD content.
Yeah. It is table stakes for anything that isn't piracy or user-generated content. If AV1 matters outside of social networks, it'll be because it gets 1st class DRM support, which requires HW decoders and other deep SoC integration. Weird little SoC DRM design decisions have kept a lot of amazing things from happening.

Quote:
Maybe viewing through the ultra low bitrate lens, has anyone done very low bitrate 2 pass VBR tests with AV1? I'd be interested to see how that might fit into the above scenario regarding downloading content for offline playback. This is a neat scenario because you get the bonus of being able to skip all the compromises one must make when encoding for adaptive bitrate delivery and can use very large buffers (vbv-maxrate + vbv-bufsize) and longer adaptive keyframe intervals.
It's not so much a long buffer window, but having a maxrate>>bitrate. Maxrate and bufsize can be kept at the profile @ level maximums while ABR can be way way lower. Doing maximum 10 sec Open GOP is pretty reasonable.

I worry that AV1's interframe CABAC dependencies will impair random access enough to make the practical maximum GOP duration a lot smaller. Some of that could probably be addressed via encoder tweaks, at the loss of a little efficiency.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:24.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.