Google VP9 "Next Generation Open Video" information posted

[benwaggoner]

I attended Google's "Next Generation Open Video" summit back in July. They've now posted links to the presentations from that.

• NGOV Product Requirements (http://downloads.webmproject.org/ngov2012/pdf/02-ngov-product-requirements.pdf)(M. Frost / J. Luther, 68kB PDF)
• WebM Product Updates
•WebM Update (http://downloads.webmproject.org/ngov2012/bankoski/vpnextsummit.html)(J. Bankoski, HTML5-format)
• WebM Adoption (http://downloads.webmproject.org/ngov2012/pdf/03-ngov-vp8-update.pdf)(J. Luther, 137kB PDF)• NGOV Project Update (http://downloads.webmproject.org/ngov2012/pdf/04-ngov-project-update.pdf)(P. Wilkins, 1.2MB PDF)
•Comparative Visualizations (http://downloads.webmproject.org/ngov2012/wilkins/vpnext-results.html)(P. Wilkins, HTML5-format)• WebM Contribution Process (http://downloads.webmproject.org/ngov2012/pdf/05-ngov-contribution-process.pdf)(J. Koleszar, 1.2MB 1.2MB PDF)
• NGOV in Hardware (http://downloads.webmproject.org/ngov2012/pdf/07-ngov-hardware.pdf)(A. Kuusela, 180kB PDF)
• Further Video Use Cases (http://downloads.webmproject.org/ngov2012/pdf/08-ngov-further-video-use-cases.pdf)(H. Finnan, 601kB PDF)
• WebP Still Image (http://downloads.webmproject.org/ngov2012/pdf/09-ngov-webp-still-image.pdf)(P. Massimino, 697kB PDF)

The two HTML5 links aren't working correctly for me.

There's plenty of interesting ideas in there. I do think that they need to clearly define Levels, which wasn't then on their roadmap.

[mandarinka]

In VP8 temporal layers use the Golden and Alt-Ref frames so that they cannot be used for boosting compression efficiency. An easy solution would be to add more alt-refs.

I like how they see that it needs b-frames and h.264-style multiple references, but seem to not be permitted to say that. :3

[hajj_3]

"Reduce video bitrate by 50% with image quality comparable to VP8 (SSIM, PSNR)."

This to me says they aren't looking to increase the video quality just reduce the bandwidth? 50% lower bandwidth than webm isn't as good as hevc is in quality terms.

Google should just joing forces with mozilla and xiph and help with Daala video codec development. It says that NGOV must work on core i5 processors that are out in Q3 2013, so presumably NGOV won't be out atleast until then.

[sneaker_ger]

This to me says they aren't looking to increase the video quality just reduce the bandwidth?

"Increasing video quality" and "reducing bandwidth" are the same thing.

[benwaggoner]

"Increasing video quality" and "reducing bandwidth" are the same thing.
And, relevant to veterans of VPx, they are also working to increase maximum quality of the NGOV codec to support visually lossless encoding, which wasn't always possible with earlier versions. So they should be able to support higher quality than was possible before.

I had the odd realization that I'd been working with the TrueMotion/VPx codecs and their developers for years longer than anyone currently working on the project! The last guy who was at Duck when I had my first business meeting with them left four years ago...

[iwod]

I stopped reading when they said VP8 was better then H264. Obviously they didn't use x264 encoder. And yes, if they should join force with the Daala codec development.

[SassBot]

Apparently when On2 was bought, Google made sure to bring along their marketeers as well. It is laughable that they still try to claim VP8 is superior to H.264. As was noted when VP8 first came out, they basically gimped the output to make VP8 look better and then threw in a bunch of fancy looking graphs to "prove" how great it was.

[benwaggoner]

I stopped reading when they said VP8 was better then H264. Obviously they didn't use x264 encoder. And yes, if they should join force with the Daala codec development.
I asked about that. This is actually a comparison with x264 and VP8 when encoding for YouTube. However, I believe they were comparing PSNR. x264 by default, and when used appropriately, rather pointedly doesn't tune for PSNR, but for perceptual quality.

The current VP8 encoders are very much tuned for PSNR; althogh I think they have implemented an SSIM tuning mode recently.

I wouldn't at all be surprised if when looking for the bitrate that gives the same PSNR between a typical x264 setting and a typical VP8 setting, VP8 would need a somewhat lower bitrate. However, I don't find that a particularly interesting comparison.

The Google people certainly spoke about the importance of tuning for perceptual quality, not just PSNR and SSIM. But at that point in NGOV codec work, it didn't sound like they'd done that much perceptual tuning yet. Which isn't unreasonable at an early stage of codec development; I doubt that much percuptual tuning has been done for HEVC implementations yet either.

[smok3]

some suggestions:
0 - make it easy for a user to actually use it properly (One-fits-all approach will not help here), "nvob -good and film file.mov -o file.gov" should be my default
0 - make it not suck like vp8

[SassBot]

I asked about that. This is actually a comparison with x264 and VP8 when encoding for YouTube.

So basically just another gimped comparison. The fact that they could get x264 output to look so poorly on Youtube seems to be quite a feat unto itself.

[benwaggoner]

So basically just another gimped comparison. The fact that they could get x264 output to look so poorly on Youtube seems to be quite a feat unto itself.
Oh, the people who make the x264 settings for YouTube know and care about what they're doing, and certainly aren't gimping anything. I think most YouTube quality issues are due to source issues. When I've tested with difficult content in pristine quality, they've produced reasonably good quality for those bitrates and encoding times.

Not what I could have hand-tuned on my own with a lot more CPU cycles, but their business is going to care a lot about picojoules per pixel!

[SassBot]

While they may not have intentionally done so, the output still looked worse, yes even with high quality source, than anything I was able to do locally. This wasn't using some command string where I turned every knob thinking I was doing something magical, but simply just using the --preset and --tune system. Google could also easily afford to use more CPU cycles than I can.

[Asmodian]

Ah but how would Google decide which --tune to use? Also while they could afford it each CPU cycle costs money which they could be keeping, and remember they are encoding really a lot of video.

[SassBot]

Ah but how would Google decide which --tune to use?

They don't need to. Just use the fast decode tune like I was.

Also while they could afford it each CPU cycle costs money which they could be keeping, and remember they are encoding really a lot of video.

Sure, but I was using the faster presets. If I was comparing their output versus using veryslow then sure that would be an absurd comparison. Considering the immense amount of capacity they have they could easily afford to use slower presets than I did especially with the clip length limits of 15 minutes and the fact that most clips are far shorter than that.

[vivan]

Looks like they messed up some graphs.
E.g. youtube doesn't use WebM for 480p. Also checked 10 random popular videos - for 360p resoultion youtube used 16-18% higher bitrate for WebM in all of them. And from 2 to hilarious 121% for 720p. Lol.

I think most YouTube quality issues are due to source issues.And what about vimeo? What magic are they using, so their encoding is so much better?

[Bloax]

Well too bad they absolutely insist on recoding every single x264 video so that it looks absolutely stunning.

Or well, You go compare the youtube version (http://www.youtube.com/watch?v=LMVp0dvR5Dc) to the Original video (http://www.mediafire.com/?zs147rg4uguz62d).

Of course I was very generous with bitrate on this one, but it's not like I can't get a fine looking video at half the size.
(Merely, it's because I actually wanted to try and avoid argh quality on Youtube.)

[oibaf]

There is an updated (2012-11-04) document about VP-Next: Overview of VP-Next - Objectives & Progress (http://www.ietf.org/proceedings/85/slides/slides-85-videocodec-4.pdf). This is the Google presentation for the first video-codec IETF working group (2012-11-05) (https://datatracker.ietf.org/meeting/85/materials.html#wg-videocodec).

Also, on the libvpx experimental git branch (http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=shortlog;h=refs/heads/experimental), VP-Next was definitively renamed to VP9.

The last slide of the PDF has current status:

[iwod]

I mean why compare a Company Backed Resources Heavy Tuned Encoder to a Video Standard Codec's Reference Encoder? I bet x264 would have beaten the HEVC reference encoder as well.

Heck, when H.264 first came out it didn't even touch xvid in terms of quality.

This whole thing is still very On2. Very unlike Google. Not that i like Google at all but i would have thought there will be less BS and more Engineering.

[oibaf]

There are two new branches on libvpx git: vp9-preview (http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=shortlog;h=refs/heads/vp9-preview) and pcs-2013 (http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=shortlog;h=refs/heads/pcs-2013) (this last one seems a showcase for Picture Coding Symposium 2013 (http://www.pcs2013.org/)?).

[Kurtnoise]

Wanna test VP9 compressor (http://www.4everload.com/index.php/download/0db3d3f18b7f/vpxenc-vp9_20121221.zip.html) ?

In addition, Chromium has added VP9 support few days ago (http://peter.sh/2012/12/vp9-and-opus-background-position-offset-and-ruby-positioning/), including Opus as Audio Codec...:)

[mandarinka]

Hmm, using Chromium sadly isn't ideal for decoding/viewing the encode... The author of the build didn't compile vpxdec too, by any chance?

[Kurtnoise]

Here is a new build (http://www.mediafire.com/?vyq5bqh2ct915iv) including the VP9 compressor & decompressor.

I'll try to add avisynth support today.

[mandarinka]

Thanks, I'll have a look. BTW, if anybody else is going to, you need to have Visual C++ Redistributable for Visual Studio 2012 installed (this (http://www.microsoft.com/en-us/download/details.aspx?id=30679))

Edit: Dunno why but it didn't want to decode VP9 encoded with --ivf /I assumed that is raw output/, but webm works.

[Leeloo Minaï]

Here is a new build (http://www.mediafire.com/?vyq5bqh2ct915iv) including the VP9 compressor & decompressor.

I'll try to add avisynth support today.

Thanks for this build, it is interesting to notice we can encode VP8 and VP9 videos with the same program.

[Sagittaire]

Here is a new build (http://www.mediafire.com/?vyq5bqh2ct915iv) including the VP9 compressor & decompressor.

I'll try to add avisynth support today.

well at this time VP9 in unable to obtain higher psnr than x264 ... and by far

[IgorC]

psnr, ssim and other for kids.

Can anybody post some screenshots for VP9 vs x264 or streams?

[Sagittaire]

psnr, ssim and other for kids.

Can anybody post some screenshots for VP9 vs x264 or streams?

Perhaps at 0.1 dB ... but not at 1.0 dB.

And to finish screenshots are more for kids than metric because screenshoot are unable to compare video for these reasons:
- Frame quality flicking: modern codec have really different frame type (Iframe, Pframe, Bframe, bFrame). With same image N you can compare Iframe for codec A and bframe for codec B.
- Rate Control: with the same codec you can configure really different RC or ratio between Frame type.
- Temporal artefact: Screenshots are unable to detect quality flick between frame (like metric for block flicking anyway)
- ... etc etc

With screenshot (and "good" screen choice) it's easy for me to make the demonstration than old DivX3 codec is better than x264. It will be hard to make that with metric ... ;-)

[mandarinka]

- Rate Control: with the same codec you can configure really different RC or ratio between Frame type.

Well, VP9 only has pframes still, afaik, so no problem.

I deleted my samples, sadly, but VP9 was easily beaten by x264 visually (I tested a DVD source at 3 megabits to see if it will still have quality issues with that generous bitrate).

The ratecontrol is probably faulty, select frames were even noticeably worse than VP8 with same settings.
Looking at the encoder parameters, I didn't find any mentions of adaptive quantization. A year or so back I heard that libvpx actually has some sort of AQ, but god knows how to enable it. (Any tips?)

--codec=vp9 --width=704 --height=480 --fps=24000/1001 --i420 --target-bitrate=3000 --auto-alt-ref --threads=1 --kf-max-dist=480 --passes=2 --end-usage=vbr --good --cpu-used=0 --min-q=0 --max-q=60 --drop-frame=0

--preset veryslow --pass 2 --bitrate 3000

[Sagittaire]

Well, VP9 only has pframes still, afaik, so no problem.

but x264 have I,P,B,b ... so screenshoot will always fails to make comparison.


I deleted my samples, sadly, but VP9 was easily beaten by x264 visually (I tested a DVD source at 3 megabits to see if it will still have quality issues with that generous bitrate).

Like say psnr or ssim ... ;-)


The ratecontrol is probably faulty, select frames were even noticeably worse than VP8 with same settings.

No problem with constant quantizer. Anyway the RC for VP9 seem not so bad (OPSNR higher in two pass mode than constant quantizer mode mean generaly good RC).

[benwaggoner]

Perhaps at 0.1 dB ... but not at 1.0 dB.
Also, On2 codecs have historically been extremely tuned for PSNR.

So if they were going to do well on one metric, that may well be it.

Of course, this is pretty early in the codec's development, so we shouldn't prejudge how good it will be in the end.

[mandarinka]

Constant quantizer is inefficient in x264, why would it be more efficient than 2-pass in VP8/9? Even the official breakdown of parameters for VP8 states that 2pass is better.

I said lack of bframes is no problem, because comparing VP8's pframe with x264's bframe is only going to hurt x264. When VP8/9 still looks bad there, the bias clearly wasn't enough to skew the result :D
BTW VP8/9 doesn'T seem to have more stable quality in the individual frames, despite no bframes. It has this crap-soso-okay-soso-crap-soso-crap frame cycling, so for all practical purposes, I don'T care if it has bframes under the hood or not.

In any case, the results were fairly conclusive over a series of frames. I didn't check just one pair, naturally.
I would love if it was possible to use some psychovisual model in the encoder, but its --tune parameter has only two options: PSNR and SSIM. So not much hope there.

[Sagittaire]

Finally after command line optimisation I obtain that:


|--------------|---------|---------|---------|---------|---------|
| Codec | PProc | Bitrate | Size | OPSNR | SSIM 2 |
|--------------|---------|---------|---------|---------|---------|
| DivX6 ASP | PP4 | 896 | 13493 | 42.78 | 77.99 |
| XviD ASP | PP4 | 896 | 13489 | 42.66 | 77.96 |
| LAVC ASP | PP4 | 897 | 13528 | 42.83 | 77.93 |
| Ateme AVC | PP0 | 896 | 13495 | 43.75 | 81.59 |
| x264 AVC | PP0 | 896 | 13509 | 44.30 | 82.44 |
| Elecard AVC | PP0 | 896 | 13481 | 43.71 | 81.13 |
| VP7 | PP2 | 897 | 13534 | 43.34 | 80.02 |
| VP8 | PP0 | 897 | 13549 | 43.27 | |
| VP9 | PP0 | 897 | 13510 | 43.98 | |
| VC1 | PP1 | 896 | 13510 | 43.06 | 78.75 |
| RV10 | HF2 | 896 | 13493 | 42.77 | 77.72 |
| DivX3 | PP4 | 896 | 13508 | 41.84 | 74.40 |
|--------------|---------|---------|---------|---------|---------|

VP9 is in "H264 codec group" for OPSNR, it's not so bad.


Constant quantizer is inefficient in x264

it's not true ... and by far. More quality level you have and more constant quantizer you must have. For exemple for Blu-Ray encoding, the best RC profil is near constant quantizer (better visual result are for high qpcomp and low P,B ratio). OPSNR for constant quantizer is near default RC for x264.

[mandarinka]

Well, (g/o/a)psnr can be wherever it wants to be. If the look of the video is godawful, it's meaningless. I don't want a codec to put numbers in a table, I need it to encode those obnoxious pictures :)

As for CQP, I'll just say that I doubt that a with the force of thousand suns and that's it o/. If nothing else, you definitely don't want all the macroblocks in a frame to have a common fixed QP.

[Sagittaire]

Well, (g/o/a)psnr can be wherever it wants to be. If the look of the video is godawful, it's meaningless. I don't want a codec to put numbers in a table, I need it to encode those obnoxious pictures :)


"I don't want a codec to put numbers in a table?" Anyway it's by definition the work of codec video. For codec pictures are just number in a table ... ;-)

As for CQP, I'll just say that I doubt that a with the force of thousand suns and that's it o/. If nothing else, you definitely don't want all the macroblocks in a frame to have a common fixed QP.

IMO AQ don't make miracle for HVS with H264. There are many other function more powerfull for psy in x264.

[iwod]

Now slightly offtopic, What ever happened to xvp8? May be xvp9 will come before even xvp8?

[Kurtnoise]

Now slightly offtopic, What ever happened to xvp8? May be xvp9 will come before even xvp8?
Still in development : https://github.com/DarkShikari/xvp8

Finally after command line optimisation
Could you share your command line please ?

[oibaf]

It looks libvpx 1.2.0 was tagged (http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=commitdiff;h=b9ce43029298182668d4dcb0e0814189e4a63c2a), altough still not announced on the web site. It also includes vp9, but it's not reported in the changelog, probably it still needs more work before being finalized and usable.

[easyfab]

could someone build avplay from lu-zero github : https://github.com/lu-zero/libav/tree/exp

Avconv is ok for me but i can't build avpaly (problem with my sdl lib)

[iwod]

Still in development : https://github.com/DarkShikari/xvp8


Could you share your command line please ?

But havn't been touched since august.

It was also interesting that DS is doing it. I thought someone else was.

[Sagittaire]

Could you share your command line please ?

yes ....

vpxenc hp.yuv -o outfile.webm --end-usage="vbr" --passes=2 --pass=1 --fpf="test.log" --target-bitrate=900 --min-q=1 --max-q=60 --best -v -t 4 -w 720 -h 304 --cpu-used=0 --fps=25000/1000 --lag-in-frames=25 --codec=vp9 --auto-alt-ref=1 --kf-max-dist=250 --kf-min-dist=0 --drop-frame=0 --static-thresh=0 --bias-pct=75 --minsection-pct=0 --maxsection-pct=800 --auto-alt-ref=1 --sharpness=0 --noise-sensitivity=0 --tune="psnr" --psnr

vpxenc hp.yuv -o outfile.webm --end-usage="vbr" --passes=2 --pass=2 --fpf="test.log" --target-bitrate=900 --min-q=1 --max-q=60 --best -v -t 4 -w 720 -h 304 --cpu-used=0 --fps=25000/1000 --lag-in-frames=25 --codec=vp9 --auto-alt-ref=1 --kf-max-dist=250 --kf-min-dist=0 --drop-frame=0 --static-thresh=0 --bias-pct=75 --minsection-pct=0 --maxsection-pct=800 --auto-alt-ref=1 --sharpness=0 --noise-sensitivity=0 --tune="psnr" --psnr --undershoot-pct=75 --overshoot-pct=125

[xooyoozoo]

Played around with a checkout from master branch on Sat, and the decoder fails with --tune=ssim and --best during scene changes. I then just did a build with their current code, which merged in some goodies from their experimental branch. Now --tune=ssim breaks the encoder instead with "segmentation fault 11" during the first 16 frames of pass 2. Maybe it's auto-alt-ref related, but either way, looks like they definitely aren't developing for SSIM... :confused:

With the older build, I did quick 2pass runs of a dark, slightly grainy source and no tunings on either encoders. VP9 had a lot higher PSNR (maybe it defaults to this tuning) and slightly higher SSIM than x264 placebo (forgot to match keyint though whoops), but x264 fared better visually, especially in scenes of grain against a flat background. VP9 also had this weird thing where grain suddenly reappeared for 1 frame and then goes away again, and the contrast is extreme enough to be noticeable in normal playback.

I think there's definitely still some qp and rc tunings to be done. In terms of objective metrics, however, things are at least very promising.

vpxenc -p 2 -t 1 --codec=vp9 --best --cpu-used=0 \
--target-bitrate=1500 --end-usage=vbr \
--auto-alt-ref=1 -v \
--minsection-pct=5 --maxsection-pct=800 \
--lag-in-frames=16 --kf-min-dist=0 --kf-max-dist=360 \
--token-parts=2 --static-thresh=0 --drop-frame=0 \
--min-q=0 --max-q=60 \
-o .webm .y4m

No hard numbers because I only wanted tune-ssim results and deleted everything after visual inspection. :o

[xooyoozoo]

Got around to actually testing the most recent build of libvpx, and it's quite something. Here's a chart of my results (http://i.imgur.com/BtuBM.png). If you're wondering about the abundance of metrics, it's because I was trying to see if anything could approximate x264's psy tweaks (nope).

There's no need to mince words: VP9 is impressive. Besides its apparent proficiency in objective metrics, it also *looks* good. I went through the 760-765 kbits clips and in terms of "pleasantness" I'd rank them as xs < xn < hevc =< vp9. The latter two are difficult to choose between because though HEVC makes the image look more 'clean', I think VP9 has some subtle psy tunings that gave more eye-pleasing bit allocations. Of course, this is only one sample, not representative, and so and so on, but we've certainly come a looonnng way since VP8's painfully underwhelming introduction.

Also, a confounding factor in the HEVC's test might be the changed intra period (normally 24 frames) and the 3 adaptive/psy QP switches I turned on. The changed intra period is to match with x264 and VP9, and I double checked the JCT-VC docs the adaptive-qp switches were based off of. The docs all showed increased obj+subj performance, but I guess you never can be sure.

Edit: Oh yea, VPX built on code from late Monday, HM from the weekend, and x264 from November.

Edit (2/2): should have done this little tidbit in the first place:
http://i.imgur.com/ThenIfE.png

[mandarinka]

I went back and made a sample encode with the decemberish build that Kurtnoise posted, since last time I deleted the samples/pictures.
This time I used a DVD source and encoded at 2500kbps. Basically, as you can see from the pictures, the problem is lack of texture/grain preservation, that leads to nasty banding/blocking in flat areas. At the same time, edges are kept dirty, which leaves the image sorta weird.
I didn't do a x264 sample, because with the utter lack of texture preservation in VP9, it would be meaningless to compare differences IMHO.

Here's a comparison with select pics, for illustration: http://check2pic.ru/compare/25338/

<BBB-work> we actually improved a lot since then, so if you want to play around, using a recent snapshot may be a good idea
I'm hoping that the final version of the encoder won't be so smooth-happy...

Settings:
vpxenc --codec=vp9 --width=704 --height=480 --fps=24000/1001 --i420 --target-bitrate=2500 --auto-alt-ref=1 --threads=1
--kf-max-dist=480 --passes=2 --end-usage=vbr --good --cpu-used=0 --min-q=0 --max-q=60 --drop-frame=0 --sharpness=0

[xooyoozoo]

I'm hoping that the final version of the encoder won't be so smooth-happy...

I did a test a while back and apparently tune=psnr is the default. As in, both encodes (no tunings and psnr tuned) produce exactly the same filesize/metrics, the doc website (http://www.webmproject.org/docs/encoder-parameters/) states something similarly, tune=ssim has been broken for several weeks (http://code.google.com/p/webm/issues/detail?id=518&colspec=ID%20Pri%20mstone%20ReleaseBlock%20Type%20Component%20Status%20Owner%20Summary), and oh, they don't even appear to be testing with ssim (http://code.google.com/p/webm/issues/detail?id=518#c7).

That said, at low bitrates where things get desperate, graceful quantization (a la x264) doesn't mean as much as simply having a larger mathematical buffer and getting as much into one bit as possible.

[Kurtnoise]

See post #49

[mandarinka]

Oh, nice! I'll go and have a look later then.

[Selur]

thanks Kurtnoise :)

[m3sh]

Thanks Kurtnoise, been meaning to test this against HEVC.

[Kurtnoise]

Any interests to play with lossless mode ?

new builds uploaded - added Lossless Mode to VP9 + WinXP compatibility + x86 (http://www.mediafire.com/?xyt4vxoy4ylv29d)/x64 (http://www.mediafire.com/?b68m9m855ia19m2) targets

[xooyoozoo]

I came across this discussion (https://groups.google.com/a/webmproject.org/forum/?fromgroups=#!topic/codec-devel/dsr56_2Nj3E) in vpx development google group. It produced this quote concerning GOP lengths: We used default for each (so 250 for x264, and 9999 for vpxenc).
We'll try to correct this for future comparisons. which was troubling because, while they didn't specifically mention HEVC, it implies that in most of the tests encodes shown in VP9 slides, HEVC would have had 10+ intra frames while VP9 got away with 1! So I then decided to do my own tests with a few clips while matching GOP lengths. I used HEVC's official bitstream files and results (ftp://ftp.kw.bbc.co.uk/hevc/hm-9.0-anchors/), and matched VP9's GOP length to that. As all of VPx's promotional material emphasizes that it's meant to be a "Web codec", it (hopefully) shouldn't have any problems with 1 intraframe a second (Youtube uses 2 a sec).

Additionally, I wanted to see how VP9 performs with its bleeding edge code. I used the latest Jan-31 commit (c94e55...) and switched on all experiments except for CSM (broken?), 6tap and abovesprefmv; the latter two appear to exist mostly for simplification. A couple of sanity checks showed almost equal PSNR versus vanilla VP9.

VP9 encode paramaters were copied from WebM guide for best quality (http://www.webmproject.org/docs/encoder-parameters/#2-pass-best-quality-vbr-encoding), but kf-max-dist was changed to match GOP and tune=psnr was added. The latter doesn't seem to do anything, but I might as well cover my bases. x264 tests were encoded with veryslow, psnr|ssim, matched keyint, open-gop, and crf.

Base spreadsheet and visual basic bd-rate/interpolation code copy-pasted from JCT-VC L0322 (http://phenix.int-evry.fr/jct/doc_end_user/current_document.php?id=7109).

http://i.imgur.com/F7CJEK7.png

(The data (http://i.imgur.com/jjj0AdS.png))

PSNR-HVS-M is included because I wanted something besides PSNR and single-scale SSIM. The downfalls of the first is obvious, and the second is getting deprecated as a perceptual metric. Some interesting tidbits pertaining to the second point: examples of relative ranking (http://i.imgur.com/1iNt6VE.png) of subjectively vs objectively measured distortion (paper (http://foulard.ece.cornell.edu/dmc27/vsnr/vsnr_a57.pdf)), SSIM's mediocre performance (http://i.imgur.com/GkvZHIU.png) on MPEG distortion (paper (https://etd.lib.metu.edu.tr/upload/12613733/index.pdf)), and statistical ranking of metrics (http://i.imgur.com/ZclTTNu.png) on certain video distortion types (first value of each cell is for compression results, paper (http://live.ece.utexas.edu/publications/2012/moorthy_vqpm2012.pdf)). I was going to include some other metrics (MSSSIM, NQM) but Matlab got the better of me, so next time. ;)

As for subjective results? Well, it's what you'd expect. It's difficult to swing a 40+ percent bitrate disadvantage in VP9's favor.

Edit: (2/6: MOAR PIX) Well, here are some comparison pics for the 3 lower tiers of bitrates. A separate 2pass veryslow no tunings x264 was ran just for this to match bitrates.
Kimono (http://i.minus.com/i0s0XpzE1uH7j.png) (1920x1080p24)
0540: HEVC (http://i.minus.com/ibvRAkKNC0Zwzm.png), VP9 (http://i.minus.com/ikMAcJfbJz6LI.png), x264 (http://i.minus.com/iYK2YZYTEiwbZ.png)
1050: HEVC (http://i.minus.com/iLA0yWRPgJ9aT.png), VP9 (http://i.minus.com/itHPoQOO0qYbo.png), x264 (http://i.minus.com/iml0gqU1WYs2M.png)
2150: HEVC (http://i.minus.com/ibh1E95jdTuz7f.png), VP9 (http://i.minus.com/i88E4YQiBDo4b.png), x264 (http://i.minus.com/if7V4ehbGcqWZ.png)
ParkScene (http://i.minus.com/iQQpAVByNF31G.png) (1920x1080p24)
0710: HEVC (http://i.minus.com/il688wxvxChO9.png), VP9 (http://i.minus.com/iM01lInBtPYHj.png), x264 (http://i.minus.com/igSYhPRTi7pad.png)
1520: HEVC (http://i.minus.com/iXhRzdWaJ25WT.png), VP9 (http://i.minus.com/i8PqfkIQL7t6q.png), x264 (http://i.minus.com/iSy68hwoj0UcX.png)
3275: HEVC (http://i.minus.com/ibdptHz962T8Nd.png), VP9 (http://i.minus.com/ibxR4PHnOcSXEb.png), x264 (http://i.minus.com/ibkzw4uG9lZI72.png)
BasketballDrill (http://i.minus.com/irtQ0Vdd1yX6T.png) (832x480p50)
0430: HEVC (http://i.minus.com/iSYw1KP28huuQ.png), VP9 (http://i.minus.com/ieAMQDdrGoXSW.png), x264 (http://i.minus.com/is1RfuWLrviPh.png)
0815: HEVC (http://i.minus.com/i5CCZ4AQCoTw1.png), VP9 (http://i.minus.com/i7M4F7CTXAH4f.png), x264 (http://i.minus.com/ioZNlsoKdDPBQ.png)
1660: HEVC (http://i.minus.com/iDa0a52zWgkr.png), VP9 (http://i.minus.com/igJH4xcoi1XdI.png), x264 (http://i.minus.com/iblikKTpcGbjJd.png)