Alliance for Open Media codecs

[Quikee]

can you add VP9 and HEVC in that too?

BPG uses x265 so there is already HEVC representation

[Clare]

can you add VP9 and HEVC in that too?

I've updated all the codecs and added VP9. I also fixed some erroneous numbers. VP9 is doing pretty good, but I don't think we will see anyone making it an image codec. The guys doing WebP can't change codec because it would modify their bitstream and all efforts to be integrated in browsers would go to waste.

BPG is based on HEVC, although it's an older revision, I've tried a more recent git revision from x265 and there's no real gain.

Nice blog btw.

[IgorC]

Clare,

Interesting comparison. Thank You.

AV1 gets better. Ringing has disappeared. There are less artifacts and more details.

[wiak]

did anyone post these?

https://image.prntscr.com/image/GWXVFSD0RkGnvy_BlPfnig.png

https://parisvideotech.com/wp-content/uploads/2017/07/AOM-AV1-Video-Tech-meet-up.pdf (English Slides)
https://www.youtube.com/watch?v=PVUIBbb-WcE (French
https://www.youtube.com/watch?v=t0oVlYcDAxM (French)

[Zebulon84]

In the videos, the AV1 developer says he estimated the availability of AV1 hardware chips around 18 month after codec freeze (so*mid 2019 if codec is realy frozen end of this year).

[easyfab]

@Zebulon84

18 months is for the very first products if I understand correctly.
And not all HW devices are planned to have it in the next generation, Apple now support HEVC, Qualcomm is not ( yet?) in the alliance...
if AV1 succeed against HEVC, It will take a long time for the adoption
Like Jean-Yves Avenard from Mozzila said : H264 will still be used for a very long time.

I think like for VP9 the first usage will be for youtube on desktop, perhaps Netflix.
but I hope the decoding will make progress, for now it's 3 times slower than VP9 and they target 2 times slower in the 6 months after bitstream freezing.
What CPU will be needed to decode 4K ?

[bstrobl]

IETF 99: https://www.youtube.com/watch?v=LJzvkOvPF30

[Clare]

The coding complexity is going through the roof. It would be nice they keep it reasonnable otherwise only big companies like Netflix or Google will be able to encode on their server. I'm running an encode and the eta is 15-20 days.

[wiak]

The coding complexity is going through the roof. It would be nice they keep it reasonnable otherwise only big companies like Netflix or Google will be able to encode on their server. I'm running an encode and the eta is 15-20 days.
well how many cores? it slow because they
1. not started optimzing for SIMD yet
2. no multi-threading yet

[mzso]

well how many cores? it slow because they
1. not started optimzing for SIMD yet
2. no multi-threading yet

Also there's a bunch of stuff they're experimenting with which they might take out or change.

[benwaggoner]

I've updated all the codecs and added VP9. I also fixed some erroneous numbers. VP9 is doing pretty good, but I don't think we will see anyone making it an image codec. The guys doing WebP can't change codec because it would modify their bitstream and all efforts to be integrated in browsers would go to waste.

BPG is based on HEVC, although it's an older revision, I've tried a more recent git revision from x265 and there's no real gain.

Nice blog btw.
HEIF is likely to leave BPG in the dust due to Apple's strong support. Same basic concept, but with a richer wrapper format.

[mzso]

HEIF is likely to leave BPG in the dust due to Apple's strong support. Same basic concept, but with a richer wrapper format.

Since it's not some fashion gunk Apple support means very little.

[nevcairiel]

New still image formats have just never taken off, and I doubt any will for quite a while. Most people are content with JPEG, especially due to it being supported by everything everywhere, and if you want quality you just go PNG.

[bstrobl]

New still image formats have just never taken off, and I doubt any will for quite a while. Most people are content with JPEG, especially due to it being supported by everything everywhere, and if you want quality you just go PNG.

I actually think HEIF is a well thought out container format. As long as there are no patent fees for the spec itself it would actually be neat if it replaced jpeg/png. The HEVC part can be thrown out for AV1 in the future and an additional lossless codec would make it pretty much the all-rounder image format.

The problem with a new image format is that if it can't do everything the previous ones do it won't succeed. HEIF seems extensible as well as containing pretty much every feature I can think of. WebP on the other hand was rushed and did not have too many benefits over jpeg/png (not to mention being stuck with VP8 as compression codec).

[Blue_MiSfit]

Yeah, Apple support for HEIF is a big deal, honestly. Lots of iOS apps will start using it internally to save bandwidth I'm sure - especially bandwidth hogs like facebook, instagram, snapchat, reddit, etc...

[benwaggoner]

Yeah, Apple support for HEIF is a big deal, honestly. Lots of iOS apps will start using it internally to save bandwidth I'm sure - especially bandwidth hogs like facebook, instagram, snapchat, reddit, etc...
Yeah, HEIF HEVC is a superior superset of JPEG, PNG, and Animated GIF. It spans all those use cases, and offers much improved encoding efficiency.

Also, Apple support means a LOT for an image format, given the overrepresentation of Macs in the creative and web design community.

HLS was a bad technology, and Apple's support made it extremely widely used. Apple's support of a really good format is going to be a big deal.

HEIF AV1 would be ballpark equivalent. However, the lack of HW decoders relative to HEVC could add friction for using AV1 instead of HEVC, particularly in mobile devices, Smart TVs, and other non-PC devices.

[huhn]

looking at VP9 hardware decoder which are standard on nvidia cards, on intel iGPUs and at lest TVs have them too for some time now. i have no clue about mobile devices. i can see AV1 hardware decoder in late 2018.

[benwaggoner]

looking at VP9 hardware decoder which are standard on nvidia cards, on intel iGPUs and at lest TVs have them too for some time now. i have no clue about mobile devices. i can see AV1 hardware decoder in late 2018.
VP9<>AV1 AV1 should be more competitive versus HEVC than VP9 is.

I found this interesting paper comparing HEVC HM and VP9 for intra-coded images, looking how different features impact efficiency. It's just PSNR based, but that is fair as HM and VP9 are both heavily tuned for optimizing PSNR.

http://www.m-hikari.com/ams/ams-2013/ams-137-140-2013/sharabaykoAMS137-140-2013.pdf

This looks interesting as well:
http://www.uta.edu/faculty/krrao/dip/Courses/EE5359/ShrutiSukumaranFinalReport_0508.pdf

I wouldn't want to extrapolate how AV1 would compare from these, of course.

[huhn]

my biggest problem with these papers is that they are pretty old and the encoder software made huge gains over the time up to this date.
only a new test can give a better idea how well they perform today.

ok this is about intra coding right now but i wouldn't be so sure anymore if VP9 can't beat HEVC.

[Beelzebubu]

VP9<>AV1 AV1 should be more competitive versus HEVC than VP9 is.

I found this interesting paper comparing HEVC HM and VP9 for intra-coded images, looking how different features impact efficiency. It's just PSNR based, but that is fair as HM and VP9 are both heavily tuned for optimizing PSNR.

http://www.m-hikari.com/ams/ams-2013/ams-137-140-2013/sharabaykoAMS137-140-2013.pdf

This looks interesting as well:
http://www.uta.edu/faculty/krrao/dip/Courses/EE5359/ShrutiSukumaranFinalReport_0508.pdf

I wouldn't want to extrapolate how AV1 would compare from these, of course.

If one paper claims 40-50% and another 10-15% BDRATE reduction, then you know - factually - at least one of them is bogus... A bigger issue is that neither of them offers actual parameters used, so it's impossible to reproduce.

That said, in my tests on intra-only cases, I have found that libvpx does indeed under-perform (in my tests by approximately 10%) versus HEVC encoders in any metric. I believe this makes sense, given per-symbol adaptivity in CABAC versus global fw/bw adaptivity in libvpx (which - for intra-only cases - accounts for 5-10%) and less intra prediction angles (which - according to 2nd paper - accounts for approximately 5%). The rectangular block sizes for intra, decomposed ADST/DCT combinations and ADST at larger transform sizes for VP9 gives a few % points back to VP9, but not enough to make up for this. Therefore, I'm willing to give the second paper the benefit of the doubt that it may be sensible (although results are still bigger than what I see in my tests). I think the first paper is bogus. Obviously these are time snapshots and since then, x265/libvpx would both have seen improvements.

Note again that this is intra-only. In inter coding, you will see AMVP (HEVC), filtered predictors (VP9) and entropy retention (VP9) making big differences. I believe entropy retention is by far the biggest one, and is the cause that as the keyframe interval increases, VP9 (libvpx) not only matches, but even overtakes HEVC in efficiency. This is obviously limited to VoD use-cases only, and cannot be used for RTC use-cases, so it also acts as an indication of particular cases where HEVC or VP9 would be more useful.

And I haven't talked about coefficient coding efficiency yet - the coefficient coding in HEVC is optimized for parallelization and hardware implementability, but the other side of the coin is that its compression performance is not so great. As a result, at higher bitrates, you tend to see HEVC suffering. I'm not sure anyone cares about this for internet VoD, though...

[benwaggoner]

If one paper claims 40-50% and another 10-15% BDRATE reduction, then you know - factually - at least one of them is bogus... A bigger issue is that neither of them offers actual parameters used, so it's impossible to reproduce.
I believe that the 10-15% is ballpark for IDR-only and 40-50% is ballpark for interframe encoded content. The problem with this kind of comparison is that it only compares what was specifically tests, and it is hard to generalize. And PSNR<>MOS!

That said, in my tests on intra-only cases, I have found that libvpx does indeed under-perform (in my tests by approximately 10%) versus HEVC encoders in any metric. I believe this makes sense, given per-symbol adaptivity in CABAC versus global fw/bw adaptivity in libvpx (which - for intra-only cases - accounts for 5-10%) and less intra prediction angles (which - according to 2nd paper - accounts for approximately 5%). The rectangular block sizes for intra, decomposed ADST/DCT combinations and ADST at larger transform sizes for VP9 gives a few % points back to VP9, but not enough to make up for this. Therefore, I'm willing to give the second paper the benefit of the doubt that it may be sensible (although results are still bigger than what I see in my tests). I think the first paper is bogus. Obviously these are time snapshots and since then, x265/libvpx would both have seen improvements.
I believe the first paper used HM, not x265, with both encoders using fixed QP to calibrate to the same equivalent file size. So, avoiding explicit psychovisual optimization, but including a fair amount of the implicit psychovisual optimization in any codec.

Note again that this is intra-only. In inter coding, you will see AMVP (HEVC), filtered predictors (VP9) and entropy retention (VP9) making big differences. I believe entropy retention is by far the biggest one, and is the cause that as the keyframe interval increases, VP9 (libvpx) not only matches, but even overtakes HEVC in efficiency. This is obviously limited to VoD use-cases only, and cannot be used for RTC use-cases, so it also acts as an indication of particular cases where HEVC or VP9 would be more useful.
Not just VOD - Live streaming can use B-frames and still get to competitive broadcast delay.

I haven't seen real-world cases where x265 doesn't have a bigger advantage over libvpx with interframe encoding. That is where psychovisual optimizations really start paying off, and encoder implementation>>bitstream syntax.

And I haven't talked about coefficient coding efficiency yet - the coefficient coding in HEVC is optimized for parallelization and hardware implementability, but the other side of the coin is that its compression performance is not so great. As a result, at higher bitrates, you tend to see HEVC suffering. I'm not sure anyone cares about this for internet VoD, though...
Ironically, parallelization is MORE important for internet VOD for VP9, since many more users are going to reply on SW decoders. VP9 is essentially single-threaded, without skipable b-frames and without Wavefront Parallel Processing; just horizontal slices, which are quite inefficient given the dominance of horizontal motion over vertical. VP9 is very much bound to peak single-thread performance when HW isn't available.

AV1 is more parallelizable than VP9 (loop filter can be its own thread, for example), although still less so than HEVC, where you basically get one potential decode thread per 64 pixels of height with WPP. And can parallelize P and B decoding if that isn't enough.

[TD-Linux]

AV1 is more parallelizable than VP9 (loop filter can be its own thread, for example), although still less so than HEVC, where you basically get one potential decode thread per 64 pixels of height with WPP. And can parallelize P and B decoding if that isn't enough.

There are some other wins - there are now fully parallelizable 2D tiles. In addition, entropy adapts on the fly rather than per frame, which makes parallel encoding and decoding easier. There are no counts to keep track of and the bitstream can be written on the fly. Frames can still start out with the probabilities from a previous one, so you need the encoder to write an appropriate reference structure to decode frames in parallel (alternately, you can decode the symbols ahead of reconstruction, as you could in VP9). There is no WPP.

[bstrobl]

Any word on whether PVQ will make it into the final codec?

[ls1dreams]

looking at VP9 hardware decoder which are standard on nvidia cards, on intel iGPUs and at lest TVs have them too for some time now. i have no clue about mobile devices. i can see AV1 hardware decoder in late 2018.

Late 2018 would be awesome but it would be a bit of a stretch, IMO. HEVC took around 3 years to get hardware decoding, and VP9 around 4.5 years.

There's a ton of people behind the AV1 alliance, but assuming that we'll get hardware decoding 1 year after the target code freeze seems optimistic to me.

Maybe we'll luck out though if hardware manufacturers are preparing in advance. The other thing that might help is the large push to low power / mobile chips which would want to get the battery life advantage.

I'm personally trying to hold out for AV1 hardware decoding to upgrade my laptop and tv media player, but it's been a long stretch. (I originally was going to wait for HEVC decoding but then quickly saw the AV1 train happening).

A better optimistic guess is probably mid to late 2019, but lets all cross our fingers that we get it sooner.

[huhn]

HEVC is from april 2013 and the 960 is from January 2015

[sneaker_ger]

First TVs had HEVC decoders no later than spring 2014. Maybe even end of 2013.

[Clare]

Late 2018 would be awesome but it would be a bit of a stretch, IMO. HEVC took around 3 years to get hardware decoding, and VP9 around 4.5 years.

There's a ton of people behind the AV1 alliance, but assuming that we'll get hardware decoding 1 year after the target code freeze seems optimistic to me.

Maybe we'll luck out though if hardware manufacturers are preparing in advance. The other thing that might help is the large push to low power / mobile chips which would want to get the battery life advantage.

I'm personally trying to hold out for AV1 hardware decoding to upgrade my laptop and tv media player, but it's been a long stretch. (I originally was going to wait for HEVC decoding but then quickly saw the AV1 train happening).

A better optimistic guess is probably mid to late 2019, but lets all cross our fingers that we get it sooner.

I'll be waiting for AV1 hardware decoding too, I currently have a Skylake, it does hevc decoding but only 8bit. Same for VP9.
Being under Linux, I think I'll have to wait even longer.

However I haven't seen Qualcomm being a supporter of AOM, I don't know what's it's gonna be like on the Android market.

[bstrobl]

http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/AV1-A-Status-Update-120214.aspx

Hopefully they can squeeze out a bit more than the targeted 20%. 25 would be nice and 30 really great.

[easyfab]

https://forum.doom9.org/showpost.php?p=1812356&postcount=259

from the pdf page 4 and 5 : target 40-50% and currently ( july 2017 ) 25-35%

So the 20% should be here but for what complexity ? Last time I try it was < 1fps for 720p

[Quikee]

20% over HEVC is the wish of NetFlix, others can have different goals in mind.

So the 20% should be here but for what complexity ? Last time I try it was < 1fps for 720p

That's not complexity - it's just the state of optimizations of the current implementation. Considering that the codec is in pre-bitstream freeze state, speed is not the focus, complexity of a coding tool however is.

[bstrobl]

There are nearly 100 experiments that are in the code base(some presumably don't work with each other). If the signalling for a large combination of those is practically free it would make sense to integrate as many as possible if they can be of use to an encoder later down the line, as long as the decoder doesn't become too complex.

What does confuse me is how Netflix and Google seem to be coming up with rather different percentages of improvement(20% vs nearly 35%?). Probably a combination of differing builds and metrics.

[IgorC]

As H.266 is already in development http://mpeg.chiariglione.org/standards/exploration/future-video-coding
it's very likely that development of a new format will be launched after AV1.

[StephanD]

Hello everybody,

I'm a media student of a university of applied sciences in germany and I have a lot of questions concerning the work of the new AV1-Codec of AOM. I know it is terrible when newbies are coming to a forum with a lot of real experts in it. So I won't ask my stupid questions here, but I'd like to know where I could ask them without spamming too much.

Have a nice day all togehter. :)

[huhn]

well this forum i guess.

but not sure if a real indeep AV1 codec profi is present/active on this forum.

so i recommend you to open a new thread in a matching sub forum or maybe even here and just post your question. so a person able to answer your question is present and willing to answer you may get your answer. if "not" you may not get it.

and like always :search: first.

[CruNcher]

https://groups.google.com/a/webmproject.org/forum/#!msg/codec-devel/idezdUoV1yY/ZfklwwhWDgAJ

could endup in a disaster growing up slowly now

[stax76]

staxrip's AV1 support is now completed, better early than late. :)

https://github.com/stax76/staxrip/blob/master/md/changelog.md

[nevcairiel]

Except it can't be complete as long as the codec isn't even finished. :p Should not encourage people to make files that eventually nothing will be able to play.

[CruNcher]

Yeah should be marked as "experimental not frozen" in the GUI on the other side Staxrip is not really a avg joe target GUI Methodology so rather unlikely that the target group of it isn't aware that AV1 isn't absolutely ready for Production.

[stax76]

@nev

Are you worried about future bug reports? :)

There is a warning I leave until it's finished.

MsgWarn("Please note that AV1 is unfinished!")

[CruNcher]

please add in bold "and shouldn't be used for any serious Production purposes yet, risk be none of your files created now with it will ever work in the future and only cause headaches, this is highly experimental"

[stax76]

@CruNcher

I changed it to this:

Please note that AV1 is unfinished/experimental!
The bitstream format is not yet frozen.

Should be clear enough.

[huhn]

not sure if the AVG user understands this as "it is very unlikely that my file will work in the future".

you can not assume that an AVG user knows what a bitstream format is.

[easyfab]

please add in bold "and shouldn't be used for any serious Production purposes yet, risk be none of your files created now with it will ever work in the future and only cause headaches, this is highly experimental"


For this people warning in not necessary, the current encoding speed is enough . :p

AV2 or AV3 will be out until they finish encoding there first movie.

It take me 5 min to encode foreman clip (300 frames 352*288).

[stax76]

OK, now it's this:

MsgWarn(
"Please note that AV1 is experimental!",
"The bitstream format is not yet frozen. It's very
likely that files created with the current encoder
become unreadable in the future.")

[CruNcher]

Hehe

AVG Joe ( oh i can read them why should i read them i read books or whatsapp messages but not files i create, nonsense ) :D

unplayable would be much better everyone understands that this wouldn't be nice.


Or did you ever saw (File->Read) ?

[stax76]

Unplayable, fine I changed it, can't get much more foolproof!

[wiak]

For this people warning in not necessary, the current encoding speed is enough . :p

AV2 or AV3 will be out until they finish encoding there first movie.

It take me 5 min to encode foreman clip (300 frames 352*288).
tell me i tried encoding 500fps 4k frames ;P took a few days
:stupid:

[easyfab]

From Timothy B. Terriberry's VDD17 presentation :

see : https://bambuser.com/v/6908002#t=2416s

Planning soft bitstream freeze by Oct. 31st


● Definition of “soft” is soft
– Some think it means “only critical bugfixes”
– Some think it means “we can tweak anything”
– Reality likely a pragmatic mix
● IPR analysis is not complete

From Netflix VDD17 prsentation :

Teaser : codec comparaison :

see : https://bambuser.com/v/6909221#t=2465s

[x265_Project]

From Timothy B. Terriberry's VDD17 presentation :
see : https://bambuser.com/v/6908002#t=2416s

Very interesting... but please use the latest build (v2.5) of x265 in your comparisons. v1.9 was released in January. Also be sure to use --tune PSNR for your PSNR comparisons, and --tune SSIM for your SSIM comparisons. Ideally, use VMAF for your objective comparison.

Can't wait for the new Netflix codec comparison.

[easyfab]

@x265_Project, yes agreed with you, x265 version is pretty old in this comparaison.

from Netflix teaser ( https://bambuser.com/v/6909221#t=2465s ) I don't see the versions but it use vmaf that should be better.

Full Netflix codec comparison should give more details.

HS : x265_Project, is this someone from your team ? https://bambuser.com/v/6909221#t=205