Google VP9 "Next Generation Open Video" information posted

[LigH]

In addition, as already stated: In YUV 4:2:2 (YUY2 / UYVY), dividing the bits of used data per frame by the number of pixels per frame, every pixel would have only an average bit amount (with hardly any "meaning per pixel") because the two chrominance components share the blue/yellow and the red/green difference from grey among two pixels. With 8 bit per component, the result would be an average 8 + 8:2 + 8:2 = 16 bit; but these 16 bit are not addressable per pixel, only per 2 pixel group. Even worse for YUV 4:2:0 (YV12 / i420), there are shared chroma difference values among a 2×2 pixel square to an average 8 + 8:4 + 8:4 = 12 bit per pixel. Such statistics have too little meaning if your scope is a single pixel, they are even misleading.

[Jamaika]

Some improvements VP10 (currently 12-20% bitrate reduction): (http://encode.ru/threads/2462-Google-VP10-video-codec)

•high bit-depth internal: 10/12 bits (also artificially to improve prediction quality: 2-4% gain for 8-bit content),
•partition structure: 8-way partition (4 in VP9), 128x128 unit size (64x64 in VP9),
•new intra prediction, “recursive 3-/4-tap filtering framework from the left and top edges”, more directional prediction modes,
•Inter-prediction – up to 6 reference frames (3 in VP9),
•InterIntra – “A compound prediction mode that combines an inter- and intra-predictor”,
•OWMC (overlapped wedge motion compensation) - “allows blocks to have oblique partitions”, using “wedge codebook”: for dividing a block into parts using separate predictors (inter/intra),
•OBMC (overlapped block motion compensation) – “blend multiple predictors using motion information from neighboring causal blocks”,
•Global motion: pan/tilt/zoom/shake, gaming,
•“Super transform: transforms that cover multiple prediction blocks, a combination of prediction and transform”,
•Extended transforms: 16 transforms ((DCT/ADST/FlipADST/DST)^2) for 4x4/8x8/16x16, explored 32x32 and 64x64 wavelet/DCT hybrids, transform skip mode, recursive transform units,
•Entropy coding – only Asymmetric Numeral Systems mentioned as much faster for decoding.

[zerowalker]

A bit confused about this stuff.

Google and others work on an Open Video Encoder thingy, but they still make the VP10, then there is Daala which is also aiming for the same thing?
Why are they allied if they are competing, am i missing something here?

[BadFrame]

Why are they allied if they are competing, am i missing something here?

The AOM (Alliance of Open Media) codec is based upon VP10 (which is still in heavy development as a separate branch), but they will use any useful technology which their members have access to, for example if you look at the AOM branch, there are Daala features being added:

https://chromium-review.googlesource.com/#/q/project:webm/aom

I don't know if VP10 will actually materialise as a separate stable codec release or if it will be decprecated as it is the base of the AOM codec (depends on how long it will take to develop I guess).

[mandarinka]

Some improvements VP10 (currently 12-20% bitrate reduction): (http://encode.ru/threads/2462-Google-VP10-video-codec)

•high bit-depth internal: 10/12 bits (also artificially to improve prediction quality: 2-4% gain for 8-bit content),
•partition structure: 8-way partition (4 in VP9), 128x128 unit size (64x64 in VP9),
•new intra prediction, “recursive 3-/4-tap filtering framework from the left and top edges”, more directional prediction modes,
•Inter-prediction – up to 6 reference frames (3 in VP9),
•InterIntra – “A compound prediction mode that combines an inter- and intra-predictor”,
•OWMC (overlapped wedge motion compensation) - “allows blocks to have oblique partitions”, using “wedge codebook”: for dividing a block into parts using separate predictors (inter/intra),
•OBMC (overlapped block motion compensation) – “blend multiple predictors using motion information from neighboring causal blocks”,
•Global motion: pan/tilt/zoom/shake, gaming,
•“Super transform: transforms that cover multiple prediction blocks, a combination of prediction and transform”,
•Extended transforms: 16 transforms ((DCT/ADST/FlipADST/DST)^2) for 4x4/8x8/16x16, explored 32x32 and 64x64 wavelet/DCT hybrids, transform skip mode, recursive transform units,
•Entropy coding – only Asymmetric Numeral Systems mentioned as much faster for decoding.

Note that some of those tools might be sacrificed later due to performance concerns ( :angry: ) or disappointing results. If you look at Daala, it dropped a large parts of originally planned features, with lapping being evaluated for possible removal too IIRC - they are not sure its complexity brings anything on top of simple loop filtering. Global Motion was famously a feature of Mpeg4 ASP, but the consensus was that it was useless, back then.

•Inter-prediction – up to 6 reference frames (3 in VP9)

Aww, I hoped they would be more ambitious, this is still just catching up to HEVC (8) and AVC (16).

Edit:
partition structure: 8-way partition (4 in VP9)
This seems to be just another name for rectangular partitions that AVC/HEVC already has, based on what they show in the video (atlhough it might not support partitions like 4x16 or 8x32?). I guess this is to get around patents, like that convoluted way to copy over b-frames in VP9.

Those oblique and wedge partitions are ambitions though, I wonder if that makes into the final codec (fingers crossed).

"Rotation/zoom motion"
\o/

"Intra-block motion" should also be useful.

Also, anyone who checked the video: is it just me or does the thing they call "OBMC" not really mean OBMC? Isn't that just combining/merging of predictors that conventional codecs do? The blocks on the picture in presentation don't look like being overlapped.

[nevcairiel]

Aww, I hoped they would be more ambitious, this is still just catching up to HEVC (8) and AVC (16).

The fact that HEVC moved down from 16 to 8 should already tell you something - its just not worth it to have such a high number of refs. :)

[mandarinka]

Generally perhaps, but don't you know all that matters in encoding is anime? :devil:

[mzso]

Note that some of those tools might be sacrificed later due to performance concerns ( :angry: ) or disappointing results. If you look at Daala, it dropped a large parts of originally planned features, with lapping being evaluated for possible removal too IIRC - they are not sure its complexity brings anything on top of simple loop filtering. Global Motion was famously a feature of Mpeg4 ASP, but the consensus was that it was useless, back then.

Well, if they remove the lapping transformation, then they removed the only notable thing about the codec. The thing the codec is known for.
They might as well just dump the project and start enhancing existing formats or contribute to VP10. (Unless they put something more radical in there)

[mandarinka]

They might as well just dump the project and start enhancing existing formats or contribute to VP10. (Unless they put something more radical in there)

It's almost what they do now, actually.
Frequency domain intra prediction is dead, some other thing too but I can't recall what it is. They originally thought lapping will spare them from the need to have loopfilter. But without loopfilter, they have terrible ringing because they don't have intra because lapping. I think Daala currently has at least two loopfilters (complex deringing one, then some smoothing(?) thing from Thor), so that beggs the question if the lapping was worth it, ever? Losing intra prediction at least to me sounds like a potential bomb.

The big remains from Daala specialities is their entropy coding and PVQ. Those are still used, but they are also experimenting with porting them to VP10. AOM might mean Daala won't actually ship.

[foxyshadis]

The fact that HEVC moved down from 16 to 8 should already tell you something - its just not worth it to have such a high number of refs. :)

HEVC certainly supports 16 refs, at least for B-frames; x265 just limits it to 8. If x265 ever gets long-term references, or even a reference structure beyond pure hierarchical, those extra refs might come in very handy, but for now, yeah, what's the point?

[nevcairiel]

HEVC certainly supports 16 refs, at least for B-frames; x265 just limits it to 8. If x265 ever gets long-term references, or even a reference structure beyond pure hierarchical, those extra refs might come in very handy, but for now, yeah, what's the point?

While the HEVC bitstream can have 16 references, every individual picture can only access 8 of them, at least in all defined profiles/tiers/levels. But you get a bit of extra flexibility by keeping more around for another frame, I suppose.

[dapperdan]

https://blogs.windows.com/msedgedev/2016/04/18/webm-vp9-and-opus-support-in-microsoft-edge/

VP9 support returns to Windows 10

[Motenai Yoda]

https://blogs.windows.com/msedgedev/2016/04/18/webm-vp9-and-opus-support-in-microsoft-edge/

VP9 support returns to Windows 10

I don't get why disabling vp9 even on desktop...

[Nintendo Maniac 64]

I don't get why disabling vp9 even on desktop...

Because even on the desktop, VP9 decoding is quite intensive - my 2.4GHz Core 2 Duo can only do 1080p 60fps VP9, and that's even in MPC-HC 64bit with D3D Fullscreen and LAVfilters 0.68 (this config gives the lowest CPU utilization possible AFAIK).

Now consider that you will get worse performance in a browser and that Skylake only has maybe twice the IPC of a Core 2 Duo.

Heck, even on my Pentium G3258 at its stock 3.2GHz, using the same lowest-CPU-usage-possible software configuration as the Core 2 Duo PC, it can't handle 3840x2160 VP9 @ 48fps (and not barely either; it also can't do 40fps); it could handle 3840x2160 VP9 @ 30fps though.

EDIT: With my Pentium G3258 @ 4.6GHz, again using the same MPC-HC configuration, I can do 3840x2160 VP9 @ 50fps, but 3840x2160 VP9 @ 60fps is too much.

[BadFrame]

The world’s best VP9 encoder: Eve

Promises to give 5-10% better compression ratio than the official libvpx, and also be 10-20% faster.

Compared to x264 it offers 15-20% better compresion rates, but is ~5x slower.

https://blogs.gnome.org/rbultje/2016/05/02/the-worlds-best-vp9-encoder-eve-2/

No word yet as of availability.

[hajj_3]

The world’s best VP9 encoder: Eve

Promises to give 5-10% better compression ratio than the official libvpx, and also be 10-20% faster.

Compared to x264 it offers 15-20% better compresion rates, but is ~5x slower.

Looks interesting, i shall keep an eye out on this thread for independent comparisons of Eve vs x264 vs x265.

[Selur]

No word yet as of availability.
from the blog entries:
rbultje says:
May 2, 2016 at 3:24 pm

@Braulio: multi-threading is planned, but not yet finished.

@krs: it is currently not available as opensource. We are hoping to test it with customers who are willing to pay for it, so we can build a business around it that can sustain its development and pay the rent etc.
so till the eve encoder is available for public use vp10 and it's successors might be available,..

[BadFrame]

from the blog entries:
so till the eve encoder is available for public use vp10 and it's successors might be available,..

Here's hoping they'll find the same business model as x264/x265 viable, which is a fully open source GPL version, with the option for commercial ventures to purchase a license which allow proprietary use.

[Beelzebubu]

Here's hoping they'll find the same business model as x264/x265 viable, which is a fully open source GPL version, with the option for commercial ventures to purchase a license which allow proprietary use.

I may be stupid, but I just don't see it working. The key issue is that GPL covers redistribution, not use, and therefore most streaming video companies (which is really the primary market where VP9 stands a solid chance of becoming a significant player) will have no incentive to pay at all. After all, they don't redistribute.

Or am I missing something?

[nevcairiel]

I may be stupid, but I just don't see it working. The key issue is that GPL covers redistribution, not use, and therefore most streaming video companies (which is really the primary market where VP9 stands a solid chance of becoming a significant player) will have no incentive to pay at all. After all, they don't redistribute.

Or am I missing something?

This is true, it really only concerns people wanting to ship it as part of their software.

[BadFrame]

I may be stupid, but I just don't see it working.
Or am I missing something?

Well, this is the exact model x264 and x265 is using to make money (free GPL distribution, selling license to use as proprietary, probably coupled with support), maybe the economics will be different now that we are rapidly moving to a streaming era.

I certainly hope the business model remains sustainable, because if not, great projects like x265 will suffer.

[Beelzebubu]

Well, this is the exact model x264 and x265 is using to make money (free GPL distribution, selling license to use as proprietary, probably coupled with support), maybe the economics will be different now that we are rapidly moving to a streaming era.

The one additional thing that doesn't help is the fact that there's a half-decent BSD-licensed VP9 encoder out there. So for VP9, you would really have 3 choices: BSD, GPL or commercial. For H264/HEVC, there's really only two: GPL or commercial.

Anyway, I currently don't see the market opportunity for releasing the encoder under GPL or something similar. I'm not against it, I just don't see it work.

[BadFrame]

The one additional thing that doesn't help is the fact that there's a half-decent BSD-licensed VP9 encoder out there...

I follow your reasoning, taking it further and looking at the actual deployment of VP9 in the wild, it seems to me that it's 99.9% Youtube, which would mean that the realistic target of this new encoder would be Google.

Makes me wonder if they are hoping to be bought by Google, Ronald Bultje who is part of the EVE development team also previously worked under contract for Google on VP9/VP8 IIRC.

Meanwhile Google's current codec development effort is likely being focused on the AOM nextgen codec and VP10 (unless it's assimilated into the OAM codec), I'm not sure they are very interested in spending resources on improving VP9 at this stage.

[wiak]

I follow your reasoning, taking it further and looking at the actual deployment of VP9 in the wild, it seems to me that it's 99.9% Youtube, which would mean that the realistic target of this new encoder would be Google.

Makes me wonder if they are hoping to be bought by Google, Ronald Bultje who is part of the EVE development team also previously worked under contract for Google on VP9/VP8 IIRC.

Meanwhile Google's current codec development effort is likely being focused on the AOM nextgen codec and VP10 (unless it's assimilated into the OAM codec), I'm not sure they are very interested in spending resources on improving VP9 at this stage.
AOM AV1 started on the VP10 codebase, so its safe to assume VP10 is at the end of the road

AOM codebase is a collection of vp10, daala and thor ideas/code

[Clare]

AOM AV1 started on the VP10 codebase, so its safe to assume VP10 is at the end of the road
I am not sure this is true.
On their GIT, there are two distinct codebase for VP10: master and nextgenv2. AOM AV1 seems based of the master codebase of VP10 (in my test , the nextgenv2 branch is noticeably slower (5x) than VP10 master and AV1, which are similar), but a lot of development is still happening daily in nextgenv2.
Maybe they will do like Daala, include bits of it in AV1, but keeping a VP10 branch for research purposes.

[benwaggoner]

AOM AV1 started on the VP10 codebase, so its safe to assume VP10 is at the end of the road
Well, the industry as a whole only switches codecs every decade or so, while VPx gets new versions a lot faster. AOM wants to have a much tighter spec, with profiles, levels, a lot of thought for how to enable efficient hardware implementations, and that kind of stuff.

It's plausible that there could be VP11/12/13 before an AV2.

Although I don't think that introducing a new bitstream spec every few years is a good idea. By the time any VPx was done and decoders started to become common, VPx++ was out. And there wasn't ever time for encoders to really mature targeting a given bitstream spec.

[leonccyiu]

I wanted to point out that Youtube is now encoding 8k videos in VP9 and their file sizes and bit-rates are monstrous unlike the h264 equivalents.

Here are some example videos
https://www.youtube.com/watch?v=QPdWJeybMo8
https://www.youtube.com/watch?v=ChOhcHD8fBA
https://www.youtube.com/watch?v=VXUdY5P0VJA
https://www.youtube.com/watch?v=sGZwH26IwA0
https://www.youtube.com/watch?v=_KoGjIDAFk8

In Chrome on my friends 5820k, it used on average about 70% of his GPU, it was a complete slideshow on an i7 6700. Even with FFvp9 I don't think it'll play smoothly on a quad core i7. It doesn't look like Pascal has 8k hardware acceleration for VP9 unlike HEVC, Kaby Lake might.

Nonetheless I find this exciting. I just wish I could watch this on the 5k iMac in the apple store but Chrome on Mac doesn't use VP9 in Youtube and the CPU's aren't powerful enough to play it back. I could try the Mac pro connected to a 32inch sharp 4k display if I can get vp9 to work in chrome on mac.

[mzso]

Rather sad though that even with this sort of bitrate you get really prominent banding and blocking artifacts. (really noticeable on the Norway video)

[Nintendo Maniac 64]

AFAIK all VP9 hardware decoders support up to 4k 120fps, and 4k @ 120fps has the same pixel count as 8k @ 30fps...

[nevcairiel]

AFAIK all VP9 hardware decoders support up to 4k 120fps, and 4k @ 120fps has the same pixel count as 8k @ 30fps...

One of those is just a metric of speed though, and the other requires more memory for buffers, reference frames and whatnot. Not quite so clear cut.

[mandarinka]

http://www.pcper.com/image/view/69408?return=node%2F65356

Nvidia lists 4K 120Hz HEVC or 8K 30Hz HEVC for its GP104 chip. For VP9 it only shows 4K 120Hz VP9, so it likely can't go to higher resolution even if framerate was low.

[hajj_3]

http://www.pcper.com/image/view/69408?return=node%2F65356

Nvidia lists 4K 120Hz HEVC or 8K 30Hz HEVC for its GP104 chip. For VP9 it only shows 4K 120Hz VP9, so it likely can't go to higher resolution even if framerate was low.

No-one is going to be using 8k vp9 video anyway. By the time 8k video is being used the new codec to succeed vp9 will be out and available as hardware decoders in intel, amd and nvidia chips.

[Blue_MiSfit]

8k is highly desirable for VR (360 video) and VP9 is also a good fit in this market since nobody wants to pay for HEVC licensing.

Of course, the lack of hardware decoders means that VP9 is best used on desktops, but that's fine because the premium VR experiences are tethered to desktop on Rift / Vive.

[Ely]

8k is highly desirable for VR (360 video) and VP9 is also a good fit in this market since nobody wants to pay for HEVC licensing.

Of course, the lack of hardware decoders means that VP9 is best used on desktops, but that's fine because the premium VR experiences are tethered to desktop on Rift / Vive.

Isn't google trying to pressure SoCs manufacturers to bundle a VP9 decoder too ? For example, the Tegra X1 has it, but it can't be the only one.

[Motenai Yoda]

@Ely there are but not for 8K, many not even 4K.

[mandarinka]

Most of newer ARM socs have VP9 decoder - but then they have HEVC too, sometimes even HEVC Main 10 (on PC CPUs/GPUs, HEVC hardware is more common than VP9 though). So regardless of the patent royalties nonsense, HEVC is already there.

[Kurtnoise]

Netflix is targeting VP9 (http://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=111296)...

[hajj_3]

Netflix is targeting VP9 (http://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=111296)...

Sounds to me that they are looking at replacing h265 with vp9 for 4k video to avoid patent licensing costs. Netflix only uses h265 for 4k resolutions so that would be a smart move even though it will use more bandwidth.

[nevcairiel]

Sounds to me that they are looking at replacing h265 with vp9 for 4k video to avoid patent licensing costs. Netflix only uses h265 for 4k resolutions so that would be a smart move even though it will use more bandwidth.

He also talks about efficiency improvements at the low end to replace H.264/AVC for mobile targets (if the device has VP9 capabilities).

[mzso]

Sounds to me that they are looking at replacing h265 with vp9 for 4k video to avoid patent licensing costs. Netflix only uses h265 for 4k resolutions so that would be a smart move even though it will use more bandwidth.

It'll only use more bandwidth if they encode the videos at higher bitrate. Which they might not. If memory serves they never quite used adequate bitrates to be perceptually lossless.

The Eve encoder people now have a new potential buyer, besides google. :)

[mandarinka]

It'll only use more bandwidth if they encode the videos at higher bitrate. Which they might not. If memory serves they never quite used adequate bitrates to be perceptually lossless.

The Eve encoder people now have a new potential buyer, besides google. :)

The article kind of sounds like they are going with libvpx :(

[Motenai Yoda]

At the high-end (4K, 10-bit) VP9 offers an alternative to HEVC.
Looks like very high-end as IIRC there isn't any consumer device/cpu/gpu capable of hw decoding for this kind of video.

[dapperdan]

I keep reading about 4K TVs that ship with Youtube and Netflix as a ready source of 4K content. I assumed they had hardware support for 4K VP9 for Youtube and HEVC for Netflix. And since they seem to always have both, Netflix could perhaps update their app to use VP9 instead.

[sneaker_ger]

I know Samsung TVs that have a VP9 decoder. (2160p30)

[Motenai Yoda]

I can't find anything about vp9 10bit (profile 1 2 as sneaker_ger wrote down here)

[sneaker_ger]

10 bit is profile 2 but no one uses it productively. No hardware support - I believe TVs and mobile SoCs included. Kaby Lake will be the first. YouTube has announced HDR but whether it will be VP9 10 bit or something else (AV1?) we have to wait and see.

[foxyshadis]

I very much doubt the world will see 10bit vp9 outside of test sequences, unless hardware support becomes universal before AV1 shows up. I'd love it, but at this point it'll probably be in the same situation 10bit AVC was.

[Nintendo Maniac 64]

it'll probably be in the same situation 10bit AVC was.
So 10bit VP9 will find a specific niche that will end up having all of the best versions in said niche be encoded in 10bit VP9?

[Jamaika]

I very much doubt the world will see 10bit vp9 outside of test sequences, unless hardware support becomes universal before AV1 shows up. I'd love it, but at this point it'll probably be in the same situation 10bit AVC was.
You say it is overrated, ex. TV LG G6.
http://www.flatpanelshd.com/article.php?subaction=showfull&id=1459247590
http://www.cnx-software.com/2016/05/19/xiaomi-mi-box-comes-to-the-us-with-android-tv-6-0-running-on-amlogic-s905x-processor/

[mandarinka]

14nm Radeons (Polaris) reportedly have VP9 decoding hardware (8bit only, likely) in addition to (10bit) HEVC.

The Stoney Ridge APU (E2-9010, A6-9210, A9-9410) according to some sources has VP9 decoding, according to some it does not (but maybe there could be a hybrid decoder available?). It does have 10-bit HEVC decoding though, that bit is confirmed.