EVC - Essential Video Coding / MPEG-5

[TomV]

Publicly available documents are posted here... https://mpeg.chiariglione.org/standa...oding-standard
The public Call for Proposals is pasted below...

1 Introduction
There is a constant demand for more efficient video coding technologies, however coding efficiency is not the only factor which determines the industry choice of video coding technology for products and services.

2 Background
Video coding technologies should address the needs of existing and emerging real-world use cases. Video coding technology should also be easy to adopt from both technological and business perspectives.
At MPEG meeting 122 in April 2018 some industry representatives identified the need for a “licensing-friendly” video codec that would facilitate the timely availability of clear and transparent Type 2 licensing terms. Discussions during the meeting resulted in draft Requirements for a New Video Coding Standard [1] which suggested a streamlined standard development process.

3 Objectives
The primary objective is to develop a new video coding standard that addresses combinations of technical and business requirements that are not adequately met by existing standards.
The new video coding standard should provide a video compression solution which combines:
• coding efficiency similar to that of HEVC
• complexity suitable for real time encoding and decoding
• timely availability of licensing terms
• the ability to address existing and emerging use cases, including
o offline encoding for streaming VOD
o live OTT streaming

4 Streamlined Development Process
It is anticipated that the following development process will achieve the above objectives:
• A brief statement of requirements, as contained in the present document
• A call for proposals
• A testing of the responses received
• Identification of functionalities, each of which provides a specific benefit in terms of efficiency, complexity or ease of implementation
• Definition of a test model, test sequences and test conditions
o The test model should consist of two tool sets: a base and an enhanced tool set
o The base tool set should be configured with tools that were made public more than 20 years ago or for which a Type 1 declaration is received
o There should be additional tools in the enhanced tool set, each of which shall provide a significant improvement in coding efficiency and be capable of being cleanly switched off on an individual basis
• Specifying a target threshold level of performance improvement for the addition of any new tool to the enhanced tool set (e.g. 3% improvement in compression efficiency)
• Systematically reviewing the contribution to performance of previously adopted tools and removing those whose removal results in performance loss that falls below a target removal threshold (e.g. 1% loss of compression efficiency)
• Giving preference to adopting a smaller number of high performance tools relative to a larger number of lower performance tools
• Giving preference to the adoption of only one proposal (from one or more organizations) per functionality if the conditions so allow

The standard should be written so that tools can be cleanly switched off wherever possible and practical. All other considerations being equal, preference may be given to tools that were made public more than 20 years ago.

Proponents shall be encouraged to commit to the timely publication of licensing terms (e.g. within 2 years of FDIS stage) either individually or as part of a patent pool.

5 Profiles and Levels
The standard shall define profiles and levels targeted at different application scenarios that are of interest to industry.
In addition, the standard shall facilitate the creation of defined subsets of the standard by external bodies, e.g. by making tools as switchable as possible. Examples of such external bodies may include ATSC, BDA, DVB, etc.

6 Requirements
6.1 Compression Performance
For 10 bit operation, the base tool set should have a compression performance that is at least as good as AVC High 10. A combination of base plus enhanced tool sets should have a compression performance that is similar to or better than that of HEVC Main 10.
6.2 Picture Formats
The new codec shall support rectangular picture formats that will include all commonly used picture formats, ranging at least from VGA to 8Kx4K. Picture formats of arbitrary size shall also be supported, within limits specified by Levels.
6.3 Colour Spaces and Colour Sampling
a) YCbCr colour spaces with 4:2:0 sampling, 10 bits per component shall be supported
b) High dynamic range and wide colour gamut shall be supported
c) YCbCr/RGB 4:4:4 and YCbCr 4:2:2 should be supported
d) Bit depths up to 16 bits per component should be supported
6.4 Frame Rates
Fixed and variable rational frame rates shall be supported, with upper limits specified by levels.
6.5 Source Video Content Characteristics
The standard shall support the encoding of the full variety of characteristics of video content encountered in the envisioned applications (to the maximum extent feasible). This includes (electronic and film) camera-captured scenes, text and graphics mixed into a camera-captured video source, rendered animation content, rendered computer graphics, etc.
6.6 Complexity
The complexity shall allow for feasible implementation of encoding and decoding within the constraints of the available technology at the expected time of usage.
A decoder conforming to a profile consisting of base plus enhanced tool sets should be no more than three times as complex as for HEVC Main 10.
Key hardware and software metrics may include memory bandwidth, maximum block sizes, decoder runtime, power consumption, etc.
6.7 Low Delay
Encode plus decode latency as low as one frame duration shall be supported.
6.8 Random Access and Trick Modes
The standard shall support random access to certain positions in time of a stored video stream, and allow fast channel switching in the case of multi-channel services.
Pause, fast forward, normal speed reverse, and fast reverse access to a stored video bitstream shall be supported.
6.9 Error Resilience
Video bitstream segmentation and packetization methods for the target networks shall be supported.
Proper balance of increase in complexity, loss in coding efficiency and benefits achieved by the error resilience measures at the coding layer should be achieved.
6.10 Buffer Models
Buffer models shall be specified for target applications.

7 Timeline
The tentative timeline is as follows:
• Finalize Requirements and Evaluation Guidelines documents in October 2018
• Submission of decoded sequences and bitstreams by 9 January 2019
• Evaluation of proposals in January 2019
• Working draft in January 2019
• CD in March 2019
• DIS in July 2019
• FDIS in January 2020

[TomV]

At the recent MPEG meeting in Marakesh, proposals were submitted for EVC. I'll share more details as soon as I'm able to.

[benwaggoner]

Yeah, I've recently heard of EVC as potentially becoming the preferred codec for 8K encoding. Sounds like we'd be able to do some head-to-head comparisons with HEVC pretty soon. If decode complexity is up to 3x more than HEVC, I'd want to see >25% efficiency improvements over HEVC for another codec to make sense from a technical perspective.

"Licensing friendly" is very interesting, but hard to predict the value of.

[Jamaika]

As a layman I have some questions? Is it new codec or container?
Is this different from the assumptions of the VVC codec?
What codecs can we import into mpeg5. Nothing is mentioned about audio codecs.

[TomV]

Originally Posted by Jamaika

As a layman I have some questions? Is it new codec or container?
Is this different from the assumptions of the VVC codec?
What codecs can we import into mpeg5. Nothing is mentioned about audio codecs.

EVC will be a new video coding standard, not a container. Don't be confused by the fact that MPEG-4 has multiple standards, including the MP4 container file format, and audio coding (AAC), as well as multiple video coding standards. This effort isn't meant to duplicate all that MPEG-4 produced. It is a separate effort from VVC.

From a recent blog post by the chairman of MPEG... https://www.linkedin.com/pulse/forty...-chiariglione/

At its 125th meeting MPEG has reviewed the responses to its Call for Proposals on a new video coding standard that sought proposals with a simplified coding structure and an accelerated development time of 12 months from working draft to FDIS. The new standard will be called MPEG-5 Essential Video Coding (EVC) and is expected to reach FDIS (Final Draft of International Standards) in January 2020.

The new video coding project will have a base layer/profile which is expected to be Option 1 and a second layer/profile that has already shown a performance ~25% better than HEVC. Licensing terms are expected to be published by patent holders within 2 years.

The baseline profile would be based on technologies that are no longer patented, and perhaps patents that will be licensed royalty free. It should have a complexity that is roughly on par with HEVC, or slightly higher, and an efficiency that is significantly better than AVC, but slightly less than HEVC. The main profile should have compression efficiency that is meaningfully better than HEVC (~ 30% when the standard is final), with a complexity that is maybe 3x HEVC.

Note the accelerated timeline. The goal is a final standard in one year.

[hajj_3]

It sounds like this codec is dead in the water before it has even begun.

1. There is no requirement for a single patent pool for the high profile version.
2. 2 years to publish patent licensing conditions is a long time. No-one is going to decide to adopt it during those 2 years due to the mess that HEVC licensing is.
3. Why would anyone want the free baseline profile that will offer roughly the same quality as HEVC when AV1 already matches that description but has already been ratified and will have hardware decoders way sooner? This would mean hardware chip vendors would have to use even more die space to add another codec decoder on their chips which many would not want to do.
4. Given that VVC will be ratified just months after this providing significantly more compression than this why would someone adopt the paid high profile version of this instead of paying a higher amount to licence the greatly superior VVC that will also have much greater hardware decoding support?

[TomV]

Originally Posted by hajj_3

It sounds like this codec is dead in the water before it has even begun.

Maybe you should wait to read the actual proposals that were submitted in the recent MPEG meeting.

Adopters are willing to bear a reasonable cost (patent royalties, and higher compute requirements) if there is a clear benefit in compression efficiency. While AV1 claims zero royalty cost, the compute cost vs HEVC is extreme, and there is little to no actual benefit in compression efficiency (subjective quality of the video, vs. HEVC). At least, not if you're using the right HEVC encoder.

[Beelzebubu]

Originally Posted by TomV

Maybe you should wait to read the actual proposals that were submitted in the recent MPEG meeting.

Adopters are willing to bear a reasonable cost (patent royalties, and higher compute requirements) if there is a clear benefit in compression efficiency. While AV1 claims zero royalty cost, the compute cost vs HEVC is extreme, and there is little to no actual benefit in compression efficiency (subjective quality of the video, vs. HEVC). At least, not if you're using the right HEVC encoder.

So you're allowed to choose the right HEVC encoder... I can only guess the same is true for AV1 then. Which AV1 encoder did you use for the subjective evaluations?

Don't mix up libaom, the PSNR-optimized reference encoder, and AV1, the standard - especially when using something else than PSNR as your output. People did this for libvpx also, and that unfairly puts the standard in a bad spotlight.

[iwod]

Originally Posted by hajj_3

It sounds like this codec is dead in the water before it has even begun.

1. There is no requirement for a single patent pool for the high profile version.
2. 2 years to publish patent licensing conditions is a long time. No-one is going to decide to adopt it during those 2 years due to the mess that HEVC licensing is.
3. Why would anyone want the free baseline profile that will offer roughly the same quality as HEVC when AV1 already matches that description but has already been ratified and will have hardware decoders way sooner? This would mean hardware chip vendors would have to use even more die space to add another codec decoder on their chips which many would not want to do.
4. Given that VVC will be ratified just months after this providing significantly more compression than this why would someone adopt the paid high profile version of this instead of paying a higher amount to licence the greatly superior VVC that will also have much greater hardware decoding support?

I am going to assume this codec will be similar to HEVC in technical terms, baseline decoding or even mainline decoding could reuse a lot of the current decoding block for HEVC. The additional die space requirement will likely be minimal. And likely a hybrid solution is possible with software update.

The current target of VVC isn't that much higher than what EVC is stating here. VVC is currently at 30+%, aiming to be 40%, and would like to achieve 50% if possible. While EVC is aiming to be 30% reduction.

I do agree though the patent pool within 2 years is way too long. And I am not understanding why they are doing this instead of working with VVC. I am guessing is because VVC are going with MCIF and aren't going through the MPEG group anymore?

[iwod]

Originally Posted by Beelzebubu

So you're allowed to choose the right HEVC encoder... I can only guess the same is true for AV1 then. Which AV1 encoder did you use for the subjective evaluations?

Which AV1 encoder are their to choose from?

[nevcairiel]

Originally Posted by TomV

Adopters are willing to bear a reasonable cost (patent royalties, and higher compute requirements) if there is a clear benefit in compression efficiency.

That still doesn't explain what the point of that baseline profile is. Worse and slower then HEVC? Who would ever use that? Anyone that matters already adopted HEVC or AV1, or even VP9, which sits in that performance space. Who is left?

For the alternate encoders, AV1 is very new, so they don't exist yet (publicly anyway). EVC on the other hand doesn't even exist yet on paper. So that comparison is still rather mood.
I'm sure EVE-AV1 will eventually exist and judging from EVE-VP9, it'll likely be a significant improvement over libaom.

Noone measures the performance of the MPEG reference encoders because everyone knows they are absolute rubbish. Yet libvpx or libaom are supposed to be the gold standard, and the entire codec is judged by their performance.

[Beelzebubu]

Originally Posted by iwod

Which AV1 encoder are their to choose from?

I couldn't find the paper slides, but an incomplete list is here.

[iwod]

Originally Posted by nevcairiel

That still doesn't explain what the point of that baseline profile is. Worse and slower then HEVC? Who would ever use that? Anyone that matters already adopted HEVC or AV1, or even VP9, which sits in that performance space. Who is left?

For the alternate encoders, AV1 is very new, so they don't exist yet (publicly anyway). EVC on the other hand doesn't even exist yet on paper. So that comparison is still rather mood.
I'm sure EVE-AV1 will eventually exist and judging from EVE-VP9, it'll likely be a significant improvement over libaom.

Noone measures the performance of the MPEG reference encoders because everyone knows they are absolute rubbish. Yet libvpx or libaom are supposed to be the gold standard, and the entire codec is judged by their performance.

Hypothetically speaking, EVC baseline reads to me as a Royalty free cut down version of HEVC that could replace the current standard which is AVC.

[utack]

Originally Posted by iwod

Hypothetically speaking, EVC baseline reads to me as a Royalty free cut down version of HEVC that could replace the current standard which is AVC.

But if you really want "roughly HEVC quality" and "royalty free" that is the market gap filled by VP9
And VP9 is deployed already, with every browser a lot of hardware supporting it out there

[TomV]

Originally Posted by Beelzebubu

So you're allowed to choose the right HEVC encoder... I can only guess the same is true for AV1 then. Which AV1 encoder did you use for the subjective evaluations?

Don't mix up libaom, the PSNR-optimized reference encoder, and AV1, the standard - especially when using something else than PSNR as your output. People did this for libvpx also, and that unfairly puts the standard in a bad spotlight.

If you don't know me, I make encoders for a living. I started x265 (with my team at MulticoreWare) 6 years ago, and led x265 until last year. For the past year I've been one of the leaders at Beamr, where I've worked with our much larger and more experienced team to insure that my new encoders (Beamr 4 and Beamr 5) are the best in the world.

Of course I recognize that libaom/aomenc is a PSNR optimized reference implementation of AV1. As I pointed out in a talk I gave a few months ago, aomenc is actually part reference encoder, part production encoder. When you study the encoding tools (algorithms) in AV1, you realize that there is a crazy amount of computational complexity in most tools for very small incremental encoding efficiency gains. The net result, as Tim Terriberry pointed out at about 33:07 in this presentation is that AV1 is more than 1000 (158 x a lot) times more complex than VP9. So, the opportunity for EVC is to provide a better combination of encoding efficiency, performance, and royalty cost/certainty than competing codecs. It seems to me that the baseline profile is targeting the same or better royalty cost/certainty as VP9, with a better combination of encoding efficiency and performance, while the main profile is targeting a higher coding efficiency (at a reasonable cost of compute), with much better royalty cost / certainty than HEVC. EVC main profile would then have significantly better encoding efficiency at a lower cost of compute than AV1, with similar patent license certainty, at a reasonable license cost. This is all brand new, so we'll have to see how the proposal is evaluated within MPEG, how well the standardization effort goes, and how many companies in the video domain choose to adopt and invest in EVC. Right now, we're at square 1. It's far too early to make any accurate predictions as to where EVC will be 1 year from now (when it hopes to be finalized), or beyond.

Keep in mind that standards that are developed within international standards bodies are preferred by many companies and other organizations over standards developed by individual companies or a consortium of companies, for a variety of reasons including the expected quality of the standard (completeness, documentation, interoperability, etc.).

[iwod]

Originally Posted by utack

But if you really want "roughly HEVC quality" and "royalty free" that is the market gap filled by VP9
And VP9 is deployed already, with every browser a lot of hardware supporting it out there

VP9 isn't really "standard", unlike AV1 which is well defined.
VP9 isn't supported by Apple, and likely never will be in Hardware. ( Because of Patents )
VP9 is pretty much a Google standard.
There are far more HEVC hardware decoder out there than VP9.

I reread the paper, and turns out the baseline is aiming at AVC HP10 quality or better. May be it is just better if we move on to AV2 or VVC.

[dapperdan]

Do they explain the strategy to avoid the problems that ruined their previous 4 or 5 attempts at royalty free standards (and/or royalty free subsets) some of which are detailed here: http://blog.chiariglione.org/forty-y...-and-counting/

But I believe there was at least two attempts to make a royalty free subsets of AVC not discussed there. What are they doing differently this time? Apart from removing the accidental veto power they gave to every member that sank the previous attempt. How confident are we they've not left similar poison pill provisions this time?

[TomV]

Originally Posted by iwod

VP9 isn't really "standard", unlike AV1 which is well defined.
VP9 isn't supported by Apple, and likely never will be in Hardware. ( Because of Patents )
VP9 is pretty much a Google standard.
There are far more HEVC hardware decoder out there than VP9.

I reread the paper, and turns out the baseline is aiming at AVC HP10 quality or better. May be it is just better if we move on to AV2 or VVC.

Is AV1 a final standard? It wasn't when they announced at NAB 2018. Now they have an Errata 1, and when you download the standard it gives you a warning that it will be labeled a draft. So it's unclear whether it's really final. I've heard a lot of feedback, even from AV1 supporters that the way the standard is written and the reference implementations are written leaves a lot to be desired, from a standards point of view. The AOM has a very different process from ISO/ITU, MPEG/VGEG where working groups meet, proposals are submitted, discussed and voted on, etc.

You should expect EVC baseline profile to be significantly more efficient than AVC high profile 10. Close to HEVC.

[TomV]

Originally Posted by dapperdan

Do they explain the strategy to avoid the problems that ruined their previous 4 or 5 attempts at royalty free standards (and/or royalty free subsets) some of which are detailed here: http://blog.chiariglione.org/forty-y...-and-counting/

But I believe there was at least two attempts to make a royalty free subsets of AVC not discussed there. What are they doing differently this time? Apart from removing the accidental veto power they gave to every member that sank the previous attempt. How confident are we they've not left similar poison pill provisions this time?

If I understand correctly, this not going to be a committee putting the standard together, as with AVC, HEVC, VVC. In fact, it's not a joint ISO MPEG/ITU VCEG standard (just ISO/MPEG). With EVC, companies, or small groups of companies submitted proposals, and only one proposal will be accepted. From that point forward, the winning proposal will be driven to the standard by the winning submitter. That company or group will be responsible for insuring the patents are licensable as well as for the technical standard.

[mandarinka]

Originally Posted by Beelzebubu

I couldn't find the paper slides, but an incomplete list is here.

Realistically speaking, everything "closed source" is likely only available to the Netflixes of this world, am I right? With the Intel's thing likely geared to datacenter/services use too?

I guess the only thing that will cater to the old group of hobbyist and small users like divx/xvid, x264 and x265 did, will possibly be Rav1e (which is a new project that have yet to prove it can produce a competetive encoder, it will likely take a lot of time).