Google VP9 "Next Generation Open Video" information posted

[JEEB]

Someone asked me to build vpxenc and vpxdec from the current VP9 experimental branch with just VP9, so here we go (http://x264.fushizen.eu/builds/vpx/vpx_enc_dec_experimental_g18e0742.7z).

Enjoy, and feel free to test.

Edit: Dear BBB said he'd like it if I'd publish the settings I compiled with as well ;) I later also learned that just --disable-vp8 would would have worked the same way for disabling VP8 functionality.
./configure --enable-vp9 --disable-vp8-encoder --disable-vp8-decoder

[hajj_3]

Frost also said that the VP9 bitstream will be frozen on 17 June, with Google's Chrome web browser and Chrome OS to implement VP9 by default.

Source: http://www.theinquirer.net/inquirer/news/2267629/webm-project-says-vp9-video-codec-is-nearing-completion



Mozilla was asked this today:

user: Google said VP9's bitstream will be frozen on 17th june, are you guys still making Daala?
mozilla ceo (brendan eich): Yes, we are. Daala aims to be more forward looking & different. VP9 given the schedule is more incremental.

http://www.youtube.com/watch?v=thtvQi6fEug&list=PLElsSrqQYMoHPY5k1l-X49OkW6Gz-9_zP&index=1

Ummm... wow, that is pretty terrible quality.

Grass is just one giant blob. Everything is VERY blocky, reminds me of the old xVid days. It is just really, well, terrible.

IDK, maybe the source video was this bad, but man, if they were trying to sell VP9 this video does them no favors.

Skip later into the video, the source was bad for the beginning it seems.

[dapperdan]

That just means that plugin doesn't support 4:4:4...
https://dl.dropboxusercontent.com/u/16254258/test/d9/DS.webp

No, it's not supported in lossy webp at all (yet). They're still thinking about how to add it:

https://groups.google.com/a/webmproject.org/forum/?fromgroups#!topic/webp-discuss/mCWm8IjAYJc


On the other hand he could have saved it as a lossless webp file.

[foxyshadis]

Mozilla was asked this today:

user: Google said VP9's bitstream will be frozen on 17th june, are you guys still making Daala?
mozilla ceo (brendan eich): Yes, we are. Daala aims to be more forward looking & different. VP9 given the schedule is more incremental.

This has serious whiffs of vaporware. Hopefully they're close to having a working prototype that they can add ideas to, instead of pie in the sky ideas and nothing to attach to them.

[foxyshadis]

No, it's not supported in lossy webp at all (yet). They're still thinking about how to add it:

https://groups.google.com/a/webmproject.org/forum/?fromgroups#!topic/webp-discuss/mCWm8IjAYJc


On the other hand he could have saved it as a lossless webp file.

I've noticed most WebP plugins only support lossy mode, not lossless.

[hajj_3]

This has serious whiffs of vaporware. Hopefully they're close to having a working prototype that they can add ideas to, instead of pie in the sky ideas and nothing to attach to them.

http://git.xiph.org/?p=daala.git;a=shortlog

Here is the VP9 google i/o talk:

http://www.youtube.com/watch?v=K6JshvblIcM

[mzso]

The had a VP9-focused session directly after the keynote:

https://developers.google.com/events/io/sessions/325741299

But it's not one of the sessions being broadcast/recorded.

I assume they'll post the slides though.

edit: a news story based on the presentation

http://news.cnet.com/8301-1023_3-57584706-93/google-urges-fast-adoption-of-vp9-video-compression/

Best bits:

They claim VP9 saves ~50% bandwith compared with H.264 and is ~1% worse than H.265

There's a VP9 test channel:

http://www.youtube.com/user/WebMVP9/videos

And apparently you can turn on VP9 in Chrome today to view them.

How do I play in vp9 mode? For me the video is played by flash so it can't be vp9.

[dapperdan]

How do I play in vp9 mode? For me the video is played by flash so it can't be vp9.

There's a few steps:

* have a recent enough Chrome (28+)
* visit chrome://flags/ and turn VP9 on
* restart chrome
* Join Youtube HTML5 demo here http://www.youtube.com/html5
* View items on those VP9 playlists

You can test what codec you're getting by right clicking and choosing show video info.

[mzso]

There's a few steps:

* have a recent enough Chrome (28+)
* visit chrome://flags/ and turn VP9 on
* restart chrome
* Join Youtube HTML5 demo here http://www.youtube.com/html5
* View items on those VP9 playlists

You can test what codec you're getting by right clicking and choosing show video info.

Huh, I have 27 with vp9 enabled, but I get vp8/vorbis. I guess I'll have to upgrade

[xooyoozoo]

Quick visual comparison using default Main HM 10 configs and "suggested" VP9 settings (http://goo.gl/fQ8BT) on latest experimental libvpx (5f3612c) compiled without any special switches.

Key interval at 32. 2pass VP9 bitrate matched against QP34 HEVC on a Sintel clip.

Here are the bitstreams and a convenient side-by-side cropping+recomposition in predictive lossless. (https://mega.co.nz/#F!yJMEUTCB!Imi4mxhpI3NVf1Z_QNPUFQ)

And this is an extra convenient, if slightly (edit: much) less accurate, version on Youtube (http://youtu.be/yNo9qVQ9Q84). I've learned my lesson and turned off comments. :)

I don't think there's anything new to see here, but it's good to touch base occasionally. The only think left to do is quantify bd-rate differences once the bitstream is finalized and perhaps revisit the codec once or twice a year to determine perceptual encoding improvements. At least the folks at MSU (http://www.compression.ru/video/codec_comparison/index_en.html) will be on top of the last bit.

There's also an in-progress encode using a Tears of Steel clip that I might edit in later if it finishes in a timely manner.

Here's what I know about VP9 coding tools [...]

Thank you for the quick rundown. Do you have a general sense of "inherent" computational complexity vs HEVC?

It occurred to me that VP9's speed advantage over the HM is perplexingly minor despite the former's assembly optimizations. On my system, normal VP9 is "only" 1.3x-1.5x as fast as reference HEVC, which is great for now, but the HM isn't a 'real' encoder. If VP9 is forced to go C-only (--disable-mmx/sse#), theoretically putting it on the same level as HEVC, it slows down by a magnitude.

[IgorC]

xooyoozoo,

Can I ask You what speed do you get with HEVC or VP9 to encode 720/1080p and what is your CPU?

Thank You.

[xooyoozoo]

The encoding machine has a quad-core Nehalem i7 @ 3.5GHz, and the encoders were compiled with gcc-4.8 -O3 -flto -funroll-loops.

I don't have exact numbers, but the clip above took ~3-3.5 hours for HEVC and ~2.5 hours for VP9. Both were single-threaded.

This is pure conjecture, but I'd predict that in the medium term, clever optimizations would increase speed by about a magnitude over reference C-code, and assembly optimizations would then speed things by another magnitude.

[GTPVHD]

http://www.youtube.com/watch?v=K6JshvblIcM

[Beelzebubu]

Bitstreams are still "data paritioned" like vp8, which is a pain for HW designs.

What does this mean? It sounds like you're saying that (like in vp8), modes/mvs and coefficients are in separate data partitions - that's not true in vp9 (it was removed about a year ago). They are interleaved per block. See here (http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=blob;f=vp9/encoder/vp9_bitstream.c;h=bcec13c4bedcafe87ad40e0026169bfe362a9207;hb=refs/heads/experimental#l834).

Support for tiles, like HEVC, for parallel decode. Mandated above 2k resolution.

Max. tile size is 4k, not 2k. See here (http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=blob;f=vp9/common/vp9_tile_common.c;h=ea26289b76ec1dff6589244c683fcc1cc575c210;hb=refs/heads/experimental#l14).

Other than that, nice rundown, thanks for writing it up.

[vivan]

http://www.youtube.com/watch?v=K6JshvblIcMOmg, he said that "We're using the very best [H.264] encoder with very best settings" and that vp9 is 50% better than that http://5.firepic.org/5/images/2013-05/19/osejf096mh6i.gif

[mzso]

Here are the bitstreams and a convenient side-by-side cropping+recomposition in predictive lossless. (https://mega.co.nz/#F!yJMEUTCB!Imi4mxhpI3NVf1Z_QNPUFQ)

Ugh. This Mega is messed up. It downloads the whole file into some mystical location and then then browser download window pops up to ask where to "download" (copy) the file. Any way to make it download where I want it from the start?

[nevcairiel]

Any way to make it download where I want it from the start?

No. thats how that weird site works.

[vivan]

What's the problem with that?They lie too much.

Can you explain this?
This is screenshot from their presentation (stated as 250 kbps encodes), cropped and downscaled to 720p: http://4.firepic.org/4/images/2013-05/19/6l3zi87aj0uy.png
This is screenshot of the same frame of encoded video (https://dl.dropboxusercontent.com/u/16254258/test/d9/enc.mp4): http://4.firepic.org/4/images/2013-05/19/s184t5fzelr7.png
(Source is 720p mp4 from http://www.youtube.com/watch?v=gF58loLxjRk, downscaled to 360p and encoded at 250 kbps bitrate (2-pass, veryslow preset). Screenshot is upscaled to 720p.)

Why does it look better than both right (supposed to be x264) and left (vp9) sides?

[vivan]

Why did you downscale it to 360p?Because encoding 720p at such bitrate is madness. Even on 240p youtube uses higher bitrate.

The video is available as 720p on YouTube. So, what led you to believe that they did not encode it at 720p for the comparison?I thought that they don't have autism. Looks like I was wrong.

Also, your source is the already compressed YouTube encoded file, whereas their source probably is the original source file that was uploaded to YouTube because them being Google they probably have access to the original uploaded file?Probably. But that makes their encode better, not mine. And that's why I can't compare PSNRs (using PSNR in 2013, lol) too.

[vivan]

Maybe it's madness to use such bitrates with x264, but maybe it's not with VP9?Okay, VP9 is better at producing blurry mess at insane low bitrates.
Sane person would just downscale an get better result with x264, insane would use vp9.

Why would anyone use a proprietary H.265 (HEVC) if there would be an open-source VP9 available being roughly as good (according to what they say in the video (http://www.youtube.com/watch?v=K6JshvblIcM)) :confused:?Proprietary? Open source? Do you even now what these words mean?
x264 is open source. vpxenc is open source.
H.264 and H.265 are open - anybody can get specification and write decoder/encoder.
VP8 is kinda open, since specification is mess. VP9 is closed, since no specification is avaivable.
Also read this (http://www.infoworld.com/d/open-source-software/googles-open-video-proposal-closes-door-software-freedom-218765).

[Beelzebubu]

Why does it look better than both right (supposed to be x264) and left (vp9) sides?

Youtube has re-compressed the I/O presentation, so naturally any image shown will have additional compression artifacts in addition to the ones already present in the encoding results shown on-screen in the room.

Source is 720p mp4 from http://www.youtube.com/watch?v=gF58loLxjRk

We didn't use the .mp4 you're referring to as source (that is actually a re-compressed version from Youtube) - we used the originally uploaded file as source for this test. Different inputs - different outputs, and thus different artifacts.

downscaled to 360p

All encodes were 720p, not 360p. Obviously, if you're using a quarter resolution at the same bitrate, it'll look 4x as good.

[Beelzebubu]

Because encoding 720p at such bitrate is madness. Even on 240p youtube uses higher bitrate.

Accomplishing amazing quality (nearly 40dB in PSNR, or equivalent gains in SSIM or whatever other metric you look at) at 250kbps is not acceptable just because you're used to it stuttering and buffering and clogging up your network?

That is the whole point of VP9! Yes, we want to half (or really: the less, the better) our bitrates. That's the whole point!

[vivan]

Youtube has re-compressed the I/O presentation, so naturally any image shown will have additional compression artifacts in addition to the ones already present in the encoding results shown on-screen in the room.That was 1080p stream. Of course youtube kills quality - but not that much.

All encodes were 720p, not 360p. Obviously, if you're using a quarter resolution at the same bitrate, it'll look 4x as good.It was upscaled back to 720p so comparison is fair.
Read How to cheat on video encoder comparisons (http://x264dev.multimedia.cx/archives/472), part about picking too low bitrate.

[mzso]

Okay, VP9 is better at producing blurry mess at insane low bitrates.
Sane person would just downscale an get better result with x264, insane would use vp9.

A sane person wouldn't complain about low bitrate when the purpose is to compare encoding artifacts.
The comparison wouldn't show much if they would choose a bitrate where both encoders are way past visual transparency.

[mandarinka]

A sane person wouldn't complain about low bitrate when the purpose is to compare encoding artifacts.
The comparison wouldn't show much if they would choose a bitrate where both encoders are way past visual transparency.

That's equally silly. But what would tell us most would be a test of both encoders at a bitrate where one already achieves acceptable quality but the second doesn't. Then you can tell which one is compressing better (with higher quality).

If you compare two streams that are completely ruined by two different sorts of artifacting, you have no idea how well the encoding scheme actually does under sane conditions (you see crap, but you don't know if that crap is necessary due to the low bitrate, or if the encoder decission and/or format just blows).

Maybe the codec that looks less bad will still have major quality issues at serious bitrates, while the second will get acceptable much faster and clearly outperform it? Or maybe vice versa...

[benwaggoner]

We didn't use the .mp4 you're referring to as source (that is actually a re-compressed version from Youtube) - we used the originally uploaded file as source for this test. Different inputs - different outputs, and thus different artifacts.
It would be great if you could make the actual files themselves available for the community to take a look at, including the command lines used to encode them.

That could help short-circuit a lot of the speculation on this thread.

All encodes were 720p, not 360p. Obviously, if you're using a quarter resolution at the same bitrate, it'll look 4x as good.
That is not accurate. Video codecs need fewer bits per pixels as the pixel count goes up, and more advanced codecs should have a stronger effect. Like HEVC, I'd think VP9 would do well in this regard with larger block sizes available.

Also, at "reasonable" bitrates adding more bits to the same pixels should help quality less than adding more bits and more pixels at the same time. Although that is a somewhat circular definition of reasonable :).

Alex Zambelli had a good discussion about this in a blog post about this I just saw. Check out the second half.
http://alexzambelli.com/blog/2013/01/28/h-265hevc-ratification-and-4k-video-streaming/

[dapperdan]

If you compare two streams that are completely ruined by two different sorts of artifacting, you have no idea how well the encoding scheme actually does under sane conditions (you see crap, but you don't know if that crap is necessary due to the low bitrate, or if the encoder decission and/or format just blows).

Maybe the codec that looks less bad will still have major quality issues at serious bitrates, while the second will get acceptable much faster and clearly outperform it? Or maybe vice versa...

They did show a PSNR graph right after the video:

http://www.youtube.com/watch?v=K6JshvblIcM&t=16m32s

[dapperdan]

Okay, VP9 is better at producing blurry mess at insane low bitrates.
Sane person would just downscale an get better result with x264, insane would use vp9.


Isn't one of VP9's features that it can do this automatically within a stream, for realtime streaming? Though if it's hanging together at the point that x264 falls apart then you'd need to go even lower before VP9 needs to downscale.

[pieter3d]

What does this mean? It sounds like you're saying that (like in vp8), modes/mvs and coefficients are in separate data partitions - that's not true in vp9 (it was removed about a year ago). They are interleaved per block. See here (http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=blob;f=vp9/encoder/vp9_bitstream.c;h=bcec13c4bedcafe87ad40e0026169bfe362a9207;hb=refs/heads/experimental#l834).



Max. tile size is 4k, not 2k. See here (http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=blob;f=vp9/common/vp9_tile_common.c;h=ea26289b76ec1dff6589244c683fcc1cc575c210;hb=refs/heads/experimental#l14).

Other than that, nice rundown, thanks for writing it up.

I meant that tiles are required to be used at pictures sizes above 2k (aka, anything bigger than 1080p). Is that not right?


Regarding the block-level interleaving, that is great news to hear.

[Beelzebubu]

It would be great if you could make the actual files themselves available for the community to take a look at, including the command lines used to encode them.

That could help short-circuit a lot of the speculation on this thread.

That's a great point, and we're happy to do so - see here (http://downloads.webmproject.org/rbultje/io2013/).

The given source files were converted to y4m/720p using "ffmpeg -i $file -an -y -vf scale=-1:720 -vframes 1000 -pix_fmt yuv420p in.y4m", then encoded with x264 using "--preset veryslow --profile high --keyint infinite --pass 1/2 --stats $statsfile --bitrate X -o out.264 in.y4m", and encoded with vpxenc using "--good --cpu-used=0 --threads=0 --profile=0 --lag-in-frames=25 --min-q=0 --max-q=63 --cq-level=20 --end-usage=0 --auto-alt-ref=1 --passes=2 --kf-max-dist=9999 --kf-min-dist=0 --drop-frame=0 --static-thresh=0 --bias-pct=50 --minsection-pct=0 --maxsection-pct=2000 --arnr-maxframes=7 --arnr-strength=5 --arnr-type=3 --sharpness=0 --undershoot-pct=100 --target-bitrate=X --codec=vp9 -o out.webm in.y4m".

[benwaggoner]

That's a great point, and we're happy to do so - see here (http://downloads.webmproject.org/rbultje/io2013/).

The given source files were converted to...
Thanks for the information. But those links are to the source files? These are already highly compressed with plenty of artifacts and weirdly lit to boot. MPEG-4pt2 ASP at 5.7 Mbps and H.264 Baseline at 8.4 Mbps? Using sources with lots of compression artifacts makes it very hard to analyze the output video quality in a meaningful way. Differences between codecs will make much less impact in the resulting quality than differences in source postprocessing. And metrics like PSNR are going to be even more useless than normal in estimating perceived quality after reencoding.

More representative samples using high-quality sources would make for a much more compelling and relevant comparison. Any source used for codec testing should have no visible artifacts.

There are a number of useful clips (many already in y4m format) at VQEG. Individually they are quite short, but multiple clips may be concatenated together: http://media.xiph.org/video/derf/

The Tears of Steel project also has nice sources (although the PNG sequence is Studio RGB and should be properly corrected to Y'16-240 range prior to compression): http://mango.blender.org/download/

Other blender.org productions are often used in codec testing, but their use of CGI leaves them rather easier to encode than most content.

Also, it would be helpful to provide the encoded files for comparison as well.

[Beelzebubu]

Thanks for the information. But those links are to the source files? These are already highly compressed with plenty of artifacts and weirdly lit to boot. MPEG-4pt2 ASP at 5.7 Mbps and H.264 Baseline at 8.4 Mbps? Using sources with lots of compression artifacts makes it very hard to analyze the output video quality in a meaningful way. Differences between codecs will make much less impact in the resulting quality than differences in source postprocessing. And metrics like PSNR are going to be even more useless than normal in estimating perceived quality after reencoding.

More representative samples using high-quality sources would make for a much more compelling and relevant comparison. Any source used for codec testing should have no visible artifacts.

We have certainly done tests against original sources also, and the results are basically the same - VP9 files are 50% smaller than x264 files of the same quality. This isn't hard to reproduce - you already pointed out yourself where to get samples, and the settings for vpxenc/x264 are identical.

However, our talk at I/O was in the context of Youtube, and Youtube uploads are typically already compressed. Therefore, the results shown in the I/O talk covered these already-compressed sources. Again - we found that this didn't change the results: VP9 files are 50% smaller than x264 files of the same quality.

[schweinsz]

We have certainly done tests against original sources also, and the results are basically the same - VP9 files are 50% smaller than x264 files of the same quality. This isn't hard to reproduce - you already pointed out yourself where to get samples, and the settings for vpxenc/x264 are identical.

However, our talk at I/O was in the context of Youtube, and Youtube uploads are typically already compressed. Therefore, the results shown in the I/O talk covered these already-compressed sources. Again - we found that this didn't change the results: VP9 files are 50% smaller than x264 files of the same quality.
Could you provide the detailed comparison results using all of the JCT-VC test sequences with the HEVC HM10.0 hierarchical B configuration?

[benwaggoner]

We have certainly done tests against original sources also, and the results are basically the same - VP9 files are 50% smaller than x264 files of the same quality. This isn't hard to reproduce - you already pointed out yourself where to get samples, and the settings for vpxenc/x264 are identical.
Sure, one of the goals of this kind of discussion is to allow independent verification of results.

However, it would be helpful in getting everyone on the same page if you could provide the source and encoded samples demonstrating your claim.

Also, a precise definition of "same quality" would be helpful. I would interpret you claim as saying "H.264 will require twice the bitrate to achieve the same subjective quality as a VP9 encode." Is that the sense that you mean it?

If you're measuring "quality" via PSNR or SSIM, your x264 should use --tune PSNR or --tune SSIM as appropriate.

However, our talk at I/O was in the context of Youtube, and Youtube uploads are typically already compressed. Therefore, the results shown in the I/O talk covered these already-compressed sources. Again - we found that this didn't change the results: VP9 files are 50% smaller than x264 files of the same quality.
Yes, providing the files you believe exhibits that behavior would be delightful.

[vivan]

For example park_joy_420_720p50.y4m:

x264 settings: --preset placebo --tune psnr --pass 1/2 --stats 1 --bitrate $BITRATE
vp9 settings: --codec=vp9 --passes=2 --good --cpu-used=0 --target-bitrate=$BITRATE --auto-alt-ref=1 --bias-pct=50 --minsection-pct=0 --maxsection-pct=2000 --lag-in-frames=25 --kf-min-dist=0 --kf-max-dist=250 --static-thresh=0 --min-q=0 --max-q=63 --arnr-maxframes=7 --arnr-strength=5 --arnr-type=3
PSNR measured by avisynth Compare.

http://5.firepic.org/5/images/2013-05/23/gyfba2x1ph9d.png
files (https://www.dropbox.com/sh/m2r0p83l4qj3b6r/rxatd1b-Bf)

@8000 kbps: vp9 (http://5.firepic.org/5/images/2013-05/23/tu9i27ran7t5.png), x264_psnr (http://5.firepic.org/5/images/2013-05/23/n6545v6og4x8.png), x264 (http://5.firepic.org/5/images/2013-05/23/ria4qkbvmynv.png)
(x264 is just --preset veryslow)

is not acceptable just because you're used to it stuttering and buffering and clogging up your network?I'm used to perfect video playback, thanks to Flash, h/w acceleration and 100 Mbit/s connection.

Is there any way to disable psnr tuning in VP9? Because this blurring looks awful.

[dapperdan]

A VP9 developer talked about this exact clip a couple of months ago:

https://groups.google.com/a/webmproject.org/d/msg/codec-devel/19wxAVyC_X8/BC_FSe4X_x8J

"Just to clarify, the --tune=ssim option in VP8 and VP9 does not currently result in adaptive quantization.

There is clearly scope in the bit-streams for such (e.g. using segmentation) but we have not yet addressed this in the encoder. I suspect that this clip is one that would benefit quite a bit in terms of psycho-visual quality (and metrics like ssim) from adaptive quantization, as do clips like park-run and park-joy (which are particularly good test cases for adaptive quant). For most clips (in a large test corpus such as YouTube), we have found that the benefit is much less significant, but there are a number of categories of content and use cases where it does seem to make a big difference, so we will certainly be addressing this."

The other clip mentioned (park-run) is also specifically called out in point 2d of Dark Shikari's "How to cheat at video codec comparisons", but I believe it applies to park joy too, though perhaps to a lesser degree.

http://x264dev.multimedia.cx/archives/472
" I’ve also used the standard test clip “parkrun” as a demonstration of adaptive quantization. But claiming that either is representative of most real content — and thus can be used as a general determinant of how good encoders are — is of course insane."

[Beelzebubu]

Is there any way to disable psnr tuning in VP9? Because this blurring looks awful.

What version (git hash)?

[vivan]

What version (git hash)?This one - http://forum.doom9.org/showthread.php?p=1628695#post1628695.
v1.2.0-2727-g18e0742

[tuqueque]

@vivan, you are using the --placebo preset in your x264 encoding... But just using the --good command in the VP9 encoding... How convenient!

use the --best command for VP9 if you want a fair comparison!

[vivan]

@vivan, you are using the --placebo preset in your x264 encoding... But just using the --good command in the VP9 encoding... How convenient!

use the --best command for VP9 if you want a fair comparison!1) I'm using proposed settings.
2) vpxenc is already slow as hell (30-40s for 1 frame, doing 8 encodes simultaneously).
3) Have you actually tested this? Even doc on website says thatSetting --good quality and --cpu-used=0 will give quality that is usually very close to and even sometimes better than that obtained with --best

[paradoxical]

@vivan, you are using the --placebo preset in your x264 encoding... But just using the --good command in the VP9 encoding... How convenient!

use the --best command for VP9 if you want a fair comparison!

The difference in compression efficiency between veryslow and placebo is not even remotely as big as you seem to think it is.

From Dark Shikari here (http://doom10.org/index.php?topic=141.msg1020#msg1020):

Based on your experience, what is the percentage of benefit gained (quality, size or others) that I get when I use "placebo" preset over "very slow" preset. Thank you in advance.

Probably less than 1%.


There's a reason it's called "placebo". It's the preset you use if you want your machine to burn energy as a space heater.

[phate89]

The difference in compression efficiency between veryslow and placebo is not even remotely as big as you seem to think it is.

From Dark Shikari here (http://doom10.org/index.php?topic=141.msg1020#msg1020):



There's a reason it's called "placebo". It's the preset you use if you want your machine to burn energy as a space heater.

another reason to use best instead of good..

[paradoxical]

another reason to use best instead of good..

Why? Is the difference somehow going to make VP9 magically better? The difference for x264 is negligible visually with a compression efficiency gain of less than 1% between the two. Also did you read the part that vivian quoted from Google's own docs?

Setting --good quality and --cpu-used=0 will give quality that is usually very close to and even sometimes better than that obtained with --best

[benwaggoner]

1) I'm using proposed settings.
2) vpxenc is already slow as hell (30-40s for 1 frame, doing 8 encodes simultaneously).
I see four basic comparisons that are useful for incorporating encode time in a codec comparison:

Use settings that provide similar encoding times, comparing the quality each provides.
Use settings that provide similar quality, comparing the encoding time required for each.
Use settings that provide the optimal quality for non-placebo encoding levels, comparing both quality and encoding time.
Use the settings that provide the utterly best quality irrespective of encoding time, to set a max bound

Encoding time can vary a lot with the same encoder on different hardware and sources, so this is rarely an area where specific claims can be made that aren't scenario-specific. But that's pretty much true of everything in codec testing :).

Given the relative maturity of H.264 and VP9 implementations, I don't think we should sweat big performance differences at this point. Making codecs faster is a lot more straightforward than making them provide better looking video! What I'm most curious at this point is to see the quality the best H.264 encoder and the best VP9 encoder with good challenging content with real-world scenarios.

The only prediction I'd make myself at this point is that each codec would be "best" for some scenarios. Codecs are generally very good at what they are designed to do, and what they are designed to do can be quite different.

[phate89]

Why? Is the difference somehow going to make VP9 magically better? The difference for x264 is negligible visually with a compression efficiency gain of less than 1% between the two. Also did you read the part that vivian quoted from Google's own docs?
If you want to compare the top quality for both you have to use both at comparable setting. It's almost sure it won't change anything but it's still not a fair test if you use the best possible in x264 because it doesn't change much with veryslow and you do the opposite with vp9 for the same exact reason

[paradoxical]

If you want to compare the top quality for both you have to use both at comparable setting. Probably it won't change anything but is still not a fair test if you use the best possible in x264 even if it doesn't change much with veryslow but you do the opposite with vp9 for the same exact reason

Placebo does not much more than make your encodes take way longer. Sure, veryslow would be more close of a comparison, but the difference is going to be so tiny as to make little difference especially visually.

[phate89]

Placebo does not much more than make your encodes take way longer. Sure, veryslow would be more close of a comparison, but the difference is going to be so tiny as to make little difference especially visually.
I know that but for the same principle (the maximum settings didn't give too much improvements with a lot slower encoding) you do for x264 a choice (slower but a little bit better) and for vp9 the opposite (faster but a little bit worse).
99% it won't change the result? Ok but it's still not a fair test.

[benwaggoner]

I know that but for the same principle (the maximum settings didn't give too much improvements with a lot slower encoding) you do for x264 a choice (slower but a little bit better) and for vp9 the opposite (faster but a little bit worse).
99% it won't change the result? Ok but it's still not a fair test.
What's fair or not fair depends on what's being tested. If the question is "what's the quality comparison between the best that VP9 and H.264 can do?" than speed doesn't matter. As a production codec, speed is a critical factor, but at this point I'm most interested in just figuring out what the current potential for both is.

I'm sure VP9 will get a lot faster and meaningfully better looking it if catches on. The lack of AQuant alone suggests significant headroom. And almost certainly there's more room for quality optimization in VP9 than x264 given the enormous amount of quality tuning has been applied to x264 over many years.

For a useful discussion today, I think we're fine looking at good representative sources encoded to scenario-appropriate bitrates using the best available encoders for each. If we see an apples-to-apples test demonstrating VP9 matching H.264 quality at half the bitrate, that would be huge. Heck, even matching quality at 80% bitrate would be quite significant at this point.

[paradoxical]

I know that but for the same principle (the maximum settings didn't give too much improvements with a lot slower encoding) you do for x264 a choice (slower but a little bit better) and for vp9 the opposite (faster but a little bit worse).

"good" is supposed to be pretty much the same and sometimes better than "best" as the quote I mentioned above says as well.

99% it won't change the result? Ok but it's still not a fair test.

It might not be "fair" but is going from "good" to "best" going to make much of any noticeable difference? Probably not. Which is what my point is. The person I responded to originally was making it out to seem that placebo was going to add some huge difference over veryslow when that just really isn't the case.

[benwaggoner]

"good" is supposed to be pretty much the same and sometimes better than "best" as the quote I mentioned above says as well.
These sorts of things are exactly why I'd like Google to provide encoded samples using the settings they select as optimal. As long as the process can be reproduced, we want to see the best VP9 can do.