Google VP9 "Next Generation Open Video" information posted

[mandarinka]

so I'm not sure the remaining suits from Nokia make VP8 any worse in this regard than H.264 given the Motorola suits.

That is sadly not true. Motorola was contractually obliged to license those patents on fair, reasonable and non-discriminatory terms to any implementer. (Their suit with MS was most likely a breach of that and the courts will beat them into submission.)

However, VP8 as a proprietary 1-vendor made codec is different. Nokia has no obligations at all with regards to 1) giving the license at all 2) asking ridiculous sums of money 3) withholding the license from select subjects (f.e. based on if they are competition to Nokia).

Basically, Nokias patent once found infringed are (can be used as) a show-stopper, Motorola's patents just mean paying a small royalty.
Naturally, there is a chance that Google will buy the needed license to those patents as it did with the MPEG-LA-related companies.

[dapperdan]

I don't believe it's been fully settled that you can't use those kind of patents to block imports. Even if it finally goes that way, that's been a long time that people have been able to throw crippling legal threats about that would be enough to sink most small players and had the big players worried enough to go running to the government for help.

There's also Microsoft vs Alcatel-lucent that's been rumbling on for years and with awards of up to 1.5 billion which was all about MPEG patents too. I believe part of the argument there is similar in that they were supposed to be signed up for MPEG-LA but thought they could do a bit better out of the deal by trolling instead.

Bottom line is, lots of patents + lots of money leads to lots of problems, and MPEG-LA isn't the panacea it's often made out to be

[mandarinka]

Well, with MPEG-LA, you have at least the promise of FRAND licensing by MPEG-LA players - and luckily, by other subjects too - because you enter into such an obligation when you license H.264 from them (and H.264 licensing is extremely wide-spread thanks to its ubiquity).

With VP*, the only one promising anything to you is Google. Google's license has similar grant-back term, so people who use VP8 can't sue you for using VP8. However, Google hasn't been able to catch as many as potentially threatening parties on that hook, compared to MPEG-LA.

I agree that MPEG-LA doesn't give you certainty either, but there is a real difference there.

[Bathrone]

Both VP8 and VC-1 wound up with lots of patent claims being asserted that weren't anticipated during standardization.

Ben you make some well founded points in your post, thanks for that

Im thankful for competition to HEVC and choice for us who want to use next gen codecs

[dapperdan]

Well, with MPEG-LA, you have at least the promise of FRAND licensing

...

With VP*, the only one promising anything to you is Google.

...

Google hasn't been able to catch as many as potentially threatening parties on that hook, compared to MPEG-LA.


Unfortunately there's no actual legal definition for FRAND (which is crazy when you think about it). They're currently beating out some kind of compromise in the courts, but that's where it's happening *in courts*, during high profile legal cases with no guarantee of how things will turn out and with all sorts of threats of import bans (which were ruled legal by at least some courts, so not a total hail mary play) based on MPEG-LA FRAND patents.

...

I posted a link to the list of the people who are promising you their patents for VP8 at no cost above:

CIF Licensing LLC

France Telecom

Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.

Fujitsu Limited

Koninklijke Philips Electronics N.V.

LG Electronics Inc.

Mitsubishi Electric Corporation

MPEG LA, LLC

NTT DOCOMO, INC.1

Panasonic Corporation

Samsung Electronics Co., Ltd.

Siemens Corporation2

...

VP8 has been a core part of Android since 2.3, and basically everyone uses android, or builds chips for android so the list of non-vp8 users is shrinking fast. (Android is of course big in phones and tablets, but it's expanding to laptops, desktops, TVs etc.)

...

The difference is subtle and complicated while it is often represented as black and white.

[xooyoozoo]

Did a quick 'spot-check' comparison of the latest 6/28 master commit. Long story short, nothing much has really changed, and I doubt much will change for months and years, as the 0.x-1% refinements take a while to add up.

This is versus an HEVC anchor clip: Kimono @ QP32. Matched bitrate and keyint as usual, and same settings as previous encodes but this time, preset was changed to Best (not sure if there's much of a difference...).

VP9 bitsream (http://dropproxy.com/f/303)

Side-by-side comparison vs HEVC. Encoded at CRF10 (http://dropproxy.com/f/304)

This is only one point of data, so I can't give an all encompassing bd-rate number, but I can say that in this single instance VP9 needs ~32% more bits for the same quality.

http://i.imgur.com/3iVMniw.png

[mandarinka]

The VP9 encoder shows quite serious issues after the scenechange, so it might be rate-control messing up the results a bit. /Still not an excuse, since they want to push this format fast for example on youtube. The videos will be screwed there for years I guess :) /

[pieter3d]

You can run the VP9 encoder in two-pass mode, probably that will fix the scene change problems

[BadFrame]

You can run the VP9 encoder in two-pass mode, probably that will fix the scene change problems

Yes, from what I gather from reading the mailing list, they (Google) run all their VP9 encodes as two-pass, also cq-level and end-use=cq seems to be silently ignored (defaulting to end-use=vbr), I guess they'll work on cq at a later stage in development.

[xooyoozoo]

You can run the VP9 encoder in two-pass mode, probably that will fix the scene change problems

I didn't realize the one-pass option is now enabled(?); it wasn't working for the longest time.

In any case, the sample above was with VP9's 2pass, as it'd be a lot more difficult to get matched bitrates otherwise.

vp9enc $input -o $output.webm --codec=vp9 --target-bitrate=$bitrate \
--kf-min-dist=0 --kf-max-dist=$kint -p 2 --best --end-usage=vbr --static-thresh=0 \
--min-q=0 --max-q=63 --auto-alt-ref=1 --lag-in-frames=25 --limit=$frms \
--minsection-pct=0 --maxsection-pct=2000 --bias-pct=50 \
--arnr-maxframes=7 --arnr-strength=5 --arnr-type=3

[schweinsz]

Did a quick 'spot-check' comparison of the latest 6/28 master commit. Long story short, nothing much has really changed, and I doubt much will change for months and years, as the 0.x-1% refinements take a while to add up.

This is versus an HEVC anchor clip: Kimono @ QP32. Matched bitrate and keyint as usual, and same settings as previous encodes but this time, preset was changed to Best (not sure if there's much of a difference...).

VP9 bitsream (http://dropproxy.com/f/303)

Side-by-side comparison vs HEVC. Encoded at CRF10 (http://dropproxy.com/f/304)

This is only one point of data, so I can't give an all encompassing bd-rate number, but I can say that in this single instance VP9 needs ~32% more bits for the same quality.

http://i.imgur.com/3iVMniw.png

Could you provide more results on more sequences such as the JCT-VC sequences, bqsquare, bqterrace, racehorses, basketballdrive, cactus.

[mzso]

Okay. So was vp9 finalized? Somebody talked about last minute additions a not long ago.

[hajj_3]

Okay. So was vp9 finalized? Somebody talked about last minute additions a not long ago.

It was finalised on the 11th june: https://groups.google.com/a/webmproject.org/forum/#!topic/webm-discuss/UzoX7owhwB0, here is the github for VP9: http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=summary

[dapperdan]

The latest WebM Project blog post:

http://blog.webmproject.org/2013/07/vp9-lands-in-chrome-dev-channel.html

It's not exactly long but the interesting bits are, their claims on how VP9 stacks up:

"VP9 development began eighteen months ago. In the short time since, according to our internal tests, we've produced a codec that shows video quality that is slightly better than HEVC (H.265) and is 50% better than VP8 and the best implementations of H.264 high profile."

and what the immediate project goals are now that the bitstream is final:

"We’ll now be working on optimizing libvpx for speed and performance, and working—with help from partners and the WebM community—to ensure that VP9 is positioned to integrate with the major encoding tools and consumer platforms, including mobile and embedded"

[xooyoozoo]

we've produced a codec that shows video quality that is slightly better than HEVC (H.265)

At this point, I would love to see some examples of situations and settings, however contrived, where this is true...

[dapperdan]

Well, they made very similar claims (VP9 1% behind HEVC) at Google IO so I'd guess most of the variables there hold true for this one as well (though they were less forthcoming about the HEVC details compared with the x264 ones, it was more of a passing remark at that time).

VP9/libvpx (obviously) vs the HEVC reference encoder. (I think they claimed they had access to other pre-release encoders but weren't allowed to publish benchmarks due to EULA restrictions)
Quality measured by PSNR.
Test set a broad selection of Youtube content.
They published their VP9 encoder settings on the mailing list and x264 as well, but I don't think they specified anything for HEVC.

Google are better than most at documenting/releasing their test setups for others to recreate so you might want to just ask them for more details on the areas that aren't fully specified.

[mandarinka]

It was probably some fringe case and the "lead" if true was a result of a difference in encoder decisions - since I think you could more or less think of VP9 as a weaker subset of compression tools available under HEVC...

[Keiyakusha]

"VP9 development began eighteen months ago. In the short time since, according to our internal tests, we've produced a codec that shows video quality that is slightly better than HEVC (H.265) and is 50% better than VP8 and the best implementations of H.264 high profile."
What I understand from this quote:
"best implementations of H.264 high profile" = x264
VP8 quality = x264 quality
VP9 quality = HEVC quality and 50% better than VP8/x264

So. I can believe that some HEVC implementation is 50% better than x264.
But in reality VP8 is like half the x264 quality...
Does that mean the truth behind their lies is that VP9 in fact no more than half of the HEVC quality?

[benwaggoner]

Quality measured by PSNR.
Test set a broad selection of Youtube content.
Unfortunately, PSNR is probably the worst available objective metric available. I'd like to see 3SSIM or MOVIE at least. I really don't understand why anyone focusing on a real-world codec would spend much time on PSNR.

I'd love to see some proper double-blind subjective tests comparing VP9 and other codecs. People watch a rapid succession of frames of decoded videos, not Rate/Distortion plots :)!

This is a quite interesting paper that highlights how useless PSNR can be, and how different codecs can show quite different correlations between subjective and objective measurements: https://ece.uwaterloo.ca/~z70wang/publications/vpqm13.pdf.

If Google is only tuning their VP9 implementation for PSNR, then their implementation may offer substantially lower subjective quality than the VP9 bitstream may be capable of.

[Rumbah]

If anyone is interested here is a Windows Cygwin compile of vpxenc/vpxdec to tinker with. I tried to get a MinGW compile to work but gave up after trying for some time.

x264.janhum.alfahosting.org/vpx05072013.7z

[Mangix]

If anyone is interested here is a Windows Cygwin compile of vpxenc/vpxdec to tinker with. I tried to get a MinGW compile to work but gave up after trying for some time.

x264.janhum.alfahosting.org/vpx05072013.7z

IIRC it's the same as making the cygwin compile but doing ./configure --cross-prefix=x86_64-w64-mingw32- --host=x86_64-pc-mingw32

MinGW has to be installed of course, which is possible through cygwin's installer.

[easyfab]

If anyone is interested here is a Windows Cygwin compile of vpxenc/vpxdec to tinker with. I tried to get a MinGW compile to work but gave up after trying for some time.

x264.janhum.alfahosting.org/vpx05072013.7z

Rumbah for MinGW compile you need a patch see :

http://forum.doom9.org/showpost.php?p=1633729&postcount=6

[zerowalker]

I have done some tests on VP9 VS VP8 on youtube, meaning there own encoded material.
And from that, i can say that VP8 looks better then VP9.

Now, i don´t know what bitrate VP9 uses, as i can´t download it.

If anyone knows a way to download the VP9 encoded videos from Youtube, please tell.

[MoSal]

I have done some tests on VP9 VS VP8 on youtube, meaning there own encoded material.
And from that, i can say that VP8 looks better then VP9.

Now, i don´t know what bitrate VP9 uses, as i can´t download it.

If anyone knows a way to download the VP9 encoded videos from Youtube, please tell.

What videos?

[zerowalker]

Youtube videos.
Some are encoded in VP9, they have a user and playlist that contains them.

[MoSal]

Youtube videos.
Some are encoded in VP9, they have a user and playlist that contains them.

You mean?
https://www.youtube.com/user/WebMVP9/videos

Those videos were not encoded with the final bitstream.

You can download VP9 versions by parsing the output of:
https://www.youtube.com/get_video_info?video_id=VIDEO_ID

This is what libquvi/cclive and others do to download from yt. It's just the VP9 formats in those experimental videos are not supported.

[zerowalker]

Yes.

I see, well is it that big of difference?

I tried to use that, i got a Get_Video_Info file, that was 32kb;S

[xooyoozoo]

Could you provide more results on more sequences such as the JCT-VC sequences, bqsquare, bqterrace, racehorses, basketballdrive, cactus.

I tried a couple more, made with latest July 4 '13 master.

Instantaneous MS-SSIM bitrate gap versus QP32 HEVC is +25% filesize for basketballdrive (bitstream (https://docs.google.com/file/d/0BzAA-H5x8NKTRzNjY3BIek80UE0/edit?usp=sharing), side-by-side (https://docs.google.com/file/d/0BzAA-H5x8NKTdXk0X1BvUTR0N2c/edit?usp=sharing) download) and +30% filesize for cactus (bitstream (https://docs.google.com/file/d/0BzAA-H5x8NKTMkVMTVpLWkxza1U/edit?usp=sharing), side-by-side (https://docs.google.com/file/d/0BzAA-H5x8NKTSld0Z0FlUmdpUGc/edit?usp=sharing) download).

Personally, I think I've lost interest in the next-gen "codec war" until we can find scenarios that seem nuanced at the very least.

(By the way, the bitstreams work perfectly on Chrome dev branch, and the side-by-sides had their frame rates reduced to 24.)

[zerowalker]

Could you record some 3d game lossless, and encode it in vp9 and hevc?
I don´t mean a whole session, but you know, some secs.

As it would be nice to see some, Unencoded material being encoded and compared.

Personally, I think I've lost interest in the next-gen "codec war" until we can find scenarios that seem nuanced at the very least.


I have to agree there. I was extremely excited with VP9, i have been waiting for it with high hopes.
But i can´t say i am impressed, but i still have a bit of hope:)

[MoSal]

Yes.

I see, well is it that big of difference?

I tried to use that, i got a Get_Video_Info file, that was 32kb;S

I see you need more hand-holding.

Assuming you replaced VIDEO_ID with the video id, you will get a file with url-encoded text. You need to escape/unquote the text (you might need to do that twice depending on the method you use for this).

Now, reading the (escaped) text should be easier, search for vp9, then copy the following url (what comes after url=) until and including the signature parameter. That's the video url.

[zerowalker]

Yes indeed.

Never done this, but i actually got it to work!
Much thanks to you:)!

[zerowalker]

Okay i tested myself, so here is a picture from the same source, which was lossless, and i encoded in vp9 1 pass -good -target 1000bitrate (but it was 300bitrate in the end for some reason)
So the x264 encode was 2 pass and 300bitrate, default slower preset.

The difference is significant, it´s night and day:)
And hopefully, i did things right.

EDIT: Forgot, the x264 i encoded is 10bit, so it has a bit of advantage there, especially as this is a dark clip.

[vivan]

And hopefully, i did things right.I bet defaults are still awful and presets are still broken. Try finding recommended settings, they were somewhere in this thread... They are way better than just --good or --best.

[BadFrame]

Okay i tested myself, so here is a picture from the same source, which was lossless, and i encoded in vp9 1 pass -good -target 1000bitrate (but it was 300bitrate in the end for some reason)
So the x264 encode was 2 pass and 300bitrate, default slower preset.

Why did you do 1 pass in vp9 (particularly when you did two pass in the x264 test)? As far as I know it's very dependant on two-pass for quality.

For my vp9 tests I used settings I picked up on the webm mailing list which I recall some dev said is what they are typically using:

vpxenc --best --cpu-used=0 --threads=0 --profile=0 --lag-in-frames=25 --min-q=0 --max-q=63 --end-usage=vbr --auto-alt-ref=1 --passes=2 --kf-max-dist=9999 --kf-min-dist=0 --drop-frame=0 --static-thresh=0 --bias-pct=50 --minsection-pct=0 --maxsection-pct=2000 --arnr-maxframes=7 --arnr-strength=5 --arnr-type=3 --sharpness=0 --undershoot-pct=100 --codec=vp9 --target-bitrate=BITRATE -o <output video> <input video>

[zerowalker]

I didn´t do a precise test, it was just more to check if it is noticable at all.

And with that encoding i did, x264 had alot of advantages. 2Pass and 10 Bit, and it still looked way worse than VP9.

Which means, x264 will look even worse with the same settings as vp9 right?


And if i did use 2 pass, it would have taken like 5 hours, for 2 secs, which isn´t that great;P
will have to wait till they improve it to atleast 1fps and not 0.005:)


EDIT: Also, is there a directshow filter/decoder for vp9?
As i need to watch in chrome, which is not that nice.

[benwaggoner]

Okay i tested myself, so here is a picture from the same source, which was lossless, and i encoded in vp9 1 pass -good -target 1000bitrate (but it was 300bitrate in the end for some reason)
So the x264 encode was 2 pass and 300bitrate, default slower preset.
Given the speed differences, I think --preset placebo would have been appropriate for an even faintly apples-apples comparison :).

--tune film may have also helped with that content.

Can you share the full x264 command line you used? The quality seems less than it should be for even that bitrate, unless there is a crazy lot of motion or something.

[zerowalker]

True, but i really doubt it would make a real difference.
Tune Film might though.

here: program --preset slower --pass 2 --bitrate 300 --stats ".stats" --output "output" "input"

It was a lot of movement i guess, it was a third person game, and i looked right and left about there (was scared from something and panicked;P).

I can give the original lossless clip if you want.


EDTI:

Added anyway: http://www.sendspace.com/file/embffb

VP9 file, encoded with the settings got from some posts before, though 1 pass, (don´t know how to get 2 pass working).
And also the lossless clip is there.

[BadFrame]

Which means, x264 will look even worse with the same settings as vp9 right?
I still think it's important to strive for as similar settings as possible when doing comparisons, which typically means going for the best each encoder has to offer.


And if i did use 2 pass, it would have taken like 5 hours, for 2 secs, which isn´t that great;P
The first pass in VP9 two pass encode is MUCH MUCH faster than the second one, they're not remotely comparable, first pass only gathers information on the file, it's the second pass where it encodes that gets super slow.


EDIT: Also, is there a directshow filter/decoder for vp9?
As i need to watch in chrome, which is not that nice.
I have no idea, I re-encoded the resulting webm file to a x264 file with --crf 0 (lossless) to check the end result.

[STaRGaZeR]

Okay i tested myself, so here is a picture from the same source, which was lossless, and i encoded in vp9 1 pass -good -target 1000bitrate (but it was 300bitrate in the end for some reason)
So the x264 encode was 2 pass and 300bitrate, default slower preset.

The difference is significant, it´s night and day:)
And hopefully, i did things right.

EDIT: Forgot, the x264 i encoded is 10bit, so it has a bit of advantage there, especially as this is a dark clip.

What in the world are you doing compressing 1920x1080 high detail game footage at 300kbps? This is the same kind of retarded testing Google did to prove VP9's superiority, if my memory serves me right.

[BadFrame]

I decided to re-do zerowalker's tests at higher bitrates, I also re-did the 300 bitrate test.

VP9 used the same settings I posted above (lifted from the webm mailing list) and is 2 pass.

For x264 I used --preset=placebo and --tune=film (as per benwaggoner's suggestion) and 2 pass.

I encoded at the following bitrates: 300, 1000 and 2000.

VP9 300 bits:
http://img24.imageshack.us/img24/9178/nklw.png
http://img811.imageshack.us/img811/593/exk1.png

VP9 1000 bits:
http://img441.imageshack.us/img441/2787/lhf.png
http://img19.imageshack.us/img19/3884/ysyz.png

VP9 2000 bits:
http://img28.imageshack.us/img28/406/akn8.png
http://img856.imageshack.us/img856/7550/x9i.png

x264 300 bits:
http://img841.imageshack.us/img841/4861/vi2z.png
http://img600.imageshack.us/img600/4782/qbqo.png

x264 1000 bits:
http://img7.imageshack.us/img7/9409/1n2k.png
http://img829.imageshack.us/img829/221/2v7.png

x264 2000 bits:
http://img585.imageshack.us/img585/6510/cghv.png
http://img580.imageshack.us/img580/3185/n6k.png

With this specific high definition content, and at these bitrates, VP9 certainly came out the clear winner.

I'll try uploading the resulting files to sendspace or something in a couple of minutes.

edit: here are the files http://www.sendspace.com/file/0quq36

[easyfab]

BadFrame,

It is possible to do the same for VP9 but with --good --cpu-used=4 instead of --best --cpu-used=0 . I will be normally faster ( a lot ?) and i'm interesting in the quality with that parameters vs x264.

[BadFrame]

BadFrame,

It is possible to do the same for VP9 but with --good --cpu-used=4 instead of --best --cpu-used=0 . I will be normally faster ( a lot ?) and i'm interesting in the quality with that parameters vs x264.
Hmm.. I don't quite see the point of re-doing the tests with 'good', it will only be a bit worse in quality, and as for comparisons between encoders I think the only 'fair' quality comparison is when you use the best options you know for each encoder. Unless I'm missing something here?

I'd certainly be interested in re-doing the tests if someone has better settings for either x264 or vp9 to try out.

As for --cpu-used=4, I've tried that and --threads=4 but to no avail, vpxenc only used one of my cores, of course I make sure to run 4 simultaneous encodings when doing tests so as to make use of my full cpu capacity, but I've never been able to have one encoding use more than 1 core.

Is there some configuration option necessary when compiling vpx?

[mindwin]

Share a source file, please.

Very interesting to compare with HEVC too.

[zerowalker]

BadFrame good comparasions there.

And i totally agree, i am impressed with how good VP9 looks, especially on low bitrate (under 1k).

But above 1k bitrate, it really polishes the picture compared to X264, which can barely show the picture correctly more or less.

[BadFrame]

Share a source file, please.

Very interesting to compare with HEVC too.

I used the source file posted by zerowalker right here in this thread just a few posts up (which is why I didn't include it with my test files) :

http://www.sendspace.com/file/embffb

And yes, I was thinking about doing tests using the hm reference encoder, but if you do it instead then that's just fine by me :D

[hajj_3]

Would be better to see how it handles 1080p bluray source rather than a video game.

[mindwin]

BadFrame
thx.
I am not familiar with reference HEVC encoder.
I want to try Strongene HEVC latest beta version (http://xhevc.com/en/downloads/downloadCenter.jsp).

[zerowalker]

Videogame is probably alot more complex with movement in terms of small details compared to Bluray.
At least in that source ( a dark scene with some effects from shooting etc).

And it´s very good to see how it performs in Dark Scenes, as that is where every encoder "Fails" to put it simple.

[BadFrame]

And i totally agree, i am impressed with how good VP9 looks, especially on low bitrate (under 1k).

But above 1k bitrate, it really polishes the picture compared to X264, which can barely show the picture correctly more or less.
Yes I was surprised at how good quality VP9 could muster out of such a low bitrate, I guess it has to do with it being developed with real time video in mind?

At 2000 bits the difference wasn't that profound though, in my opinon, although I think VP9 still clearly won.

I'd like to do some high definition animation tests on my Akira bluray, I just gotta figure out how to decrypt it.

[mindwin]

zerowalker
You may try this video sample (https://mega.co.nz/#!SEIhACSD!dRSlwkSofL-hfYPmZzN54zhaATdsJQXa6XxTb3be1Jo). (660mb)


Have dark, static, dynamic scenes, some film grain, and very high quality.

Format : AVC
Format profile : High@L4.0
Duration : 2mn 7s
Bit rate : 43.4 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Stream size : 660 MiB (99%)