Google VP9 "Next Generation Open Video" information posted

[Procrastinating]

Given that the GPU was only able to leverage motion prediction in X264(and in cases worse motion prediction at that), what areas of VP9/265 are able to better take advantage of GPGPU?

[Selur]

looks like they are only improving the decoder, not the encoder

[mzso]

Hmm. Fffmpeg for vp9 is garbage. Only encodes -b (cbr) everything else results in 200k bitrate.
I still can't a place for recent vpxenc builds with vp9. (Does anyone have a recent build?) Looks like people are pretty indifferent to this format.

[Selur]

uploaded vpx_v1.3.0-1742 to my google drive (https://drive.google.com/folderview?id=0B_WxUS1XGCPASUZibG5XZkRfeTg&usp=sharing)

[Monarc]

Hmm. Fffmpeg for vp9 is garbage. Only encodes -b (cbr) everything else results in 200k bitrate.

Try:

... -vcodec libvpx-vp9 -crf 10 -b:v 5M ...

-b = max Bitrate, if -crf is used.

[mzso]

Thanks guys.


Ouch. Can't default to using the same resolution as the input... :)

[mzso]

Is this wrong?

E:\Videó Felvételek>vpxenc --codec=vp9 --good --aq-mode=2 --kf-max-dist=30 --cq-level=20 -w 1920 -h 1200 -o vpx.mkv "input.mkv"
Pass 1/2 frame 82/83 12616B 1230b/f 36924b/s 21279 ms (3.85 fps)←[K
Pass 2/2 frame 34/9 794197B 1023652 ms 1.99 fpm [ETA 2:11: 47] ←[K 40FF

I just noticed that the ETA might not be 2 minutes 11 second, but two hours eleven minutes. Feels a bit exorbitant for a less than 10 second sample even on my dated computer.

[Selur]

vp9 is slow (lack of multi-threading), not knowing what cpu you have that might be right :)

[mzso]

vp9 is slow (lack of multi-threading), not knowing what cpu you have that might be right :)

Well, I encode that very same file with x264 in like two minutes. (I have an e6750 BTW) But cinse the couple frames that were encoded turned out to be total garbage. I'm assuming the input is not supported even though the encoder didn't complain.

[mzso]

Okay so I have to decode and pipe first. :)
Now I get an empty file. :)
ffmpeg -i "E:\Videó Felvételek\motor2.avi" -an -vcodec rawvideo -pix_fmt yuv420p -f rawvideo - | vpxenc --codec=vp9 --good --aq-mode=2 --kf-max-dist=30 --cq-level=20 --i420 -o -w 1920 -h 1200 vpx-teszt.mkv -

I get "av_interleaved_write_frame(): Invalid argument".

[Selur]

a. I normally feed vpxenc through a pipe, where I do the decoding&filtering.
b. yes, your system is probably too slow to fast encode vp9 faster atm. (hoping that there will be some optimizations in the future)

just for perspective on a i7-4770k using:
ffmpeg -y -v -10 -r 25000/1000 -analyzeduration 100M -probesize 100M -i "F:\TestClips&Co\MPEG-4 H.264\1080p25.ts" -an -sn -threads 8 -vsync 0 -r 25000/1000 -pix_fmt yuv420p -f yuv4mpegpipe - | vpxenc --codec=vp9 --passes=1 --pass=1 --end-usage=cq --cq-level=18 --target-bitrate=15000 --profile=0 --good --cpu-used=3 --undershoot-pct=0 --buf-sz=6 --buf-initial-sz=4 --buf-optimal-sz=5 --drop-frame=0 --resize-allowed=0 --kf-min-dist=0 --kf-max-dist=250 --auto-alt-ref=0 --noise-sensitivity=0 --sharpness=0 --static-thresh=0 --tile-columns=2 --tile-rows=1 --threads=16 --width=1920 --height=1080 -o "H:\Temp\test_23_38_28_8910_01.vp9" -
it takes 20:45.096min (1245 seconds) to encode 3178 frames -> ~2.55 fps (had no 1920x1200 source available so I went with 1080p)
side note: Encoding the same source with x265 and preset slow I get ~2.6fps, x264 with preset placebo runs at~3.57fps.

Could this get faster? Sure if more than one cpu core was in use during the encoding,... so still hoping for multi-threading support.
Main drawbacks, from my perspective with VP9 atm. are:
1. it's too slow to be useful (x265 encodes faster)
2. 2pass encoding only hits a target bitrate once a year

[mandarinka]

IIRC the author of the AQ patch thinks that the useless 2-pass rate-control is due to the use of adaptive quantization.

[Selur]

Main reason for using 2pass is to hit a specific average bit rate (end-size) and archive the 'best' possible quality under that restriction.
So if I have to disable adaptive quantization for 2pass encoding to work, I would say that sounds like either the adaptive quantization or the 2pass implementation is rubbish.

[Kurtnoise]

For your information, a new branch has been created few days ago in the source tree. It concerns High Bits Depth Feature...

So, I made some fresh windows compile in order to play with :

-> vpx tools for x86 plateform (https://www.mediafire.com/?qbrqt41412af72y)
-> vpx tools for x64 plateform (http://www.mediafire.com/download/t4itk2dgscuvwr1/vpx_hbdepth-x64_1.3.0-6956f43a9.zip)

[Selur]

hmm,... high bit depth sounds nice (probably it will slow things down even more)

[zerowalker]

Hope does VP9 Decoding fare for you compared to x265?

Cause x265 takes some serious CPU from me, haven't checked, but i think it's heavier than VP9, and i am guessing that x265 is better optimized (at least Encoding is, so probably Decoding as well).
And if so, that leaves hope for reduced workload in the future.

[Nintendo Maniac 64]

Honestly they were really similar; h.265 only averaged about 10% more CPU utilization than VP9 on my Brisbane (Core 2 Duo testing will have to wait).

However, there are some distinct differences. VP9 was more dynamic in its CPU usage with a range of like 40%; h.265 by comparison only had a range of maybe 25% CPU usage.

Most importantly though, VP9 spead the load between my two cores much more equally; h.265 on the other had had one core getting very close to being maxed out while the ovher core ran at below half usage.

[zerowalker]

Anyone technical, is there a reason why VP9 isn't multi-threaded yet?

I find it very hard to understand that a Codec made in the "Threads/Cores Era" not utilizing it at all.

[Selur]

Only reasons that pop into my mind (with Google behind it) are:
a. Google doesn't want to add multi-threading (since Google seems to be the only one really using VP9 it seems kind of in there interest to support multi-threading) or simply they don't want to share it and only added it in some non-public repository
b. There really is some enormous design flaw which prevents the encoder from using multi-threading, but reading https://groups.google.com/a/webmproject.org/forum/#!topic/codec-devel/IFFSy2xlKV0 it sounded like nobody got around to implement it. (and that conversation is from nearly one year ago)
c. Google can't find anyone who knows enough about VP9 who could spend the time to implement multi-threading (unlikely, but you never know)
d. Google gave up on VP9 and there is no man-power planned in to support it development wise any further

[mzso]

"d." is extremely unlikely. VP9 youtube videos just started to appear.

[zerowalker]

I supposed the closest is that Google don't care about Threading, as Youtube Videos are done in a way that whre threads doesn't matter.
It's faster to do 4 videos on a 4 core CPU, than 1 using 4 cores.

Which would mean Google themselves improves anything with Quality first (Bandwidth), then performance (not threading part).

Which would leave Threading to be done by others, as it's open source it can surely be the case.

I guess that's the theory of things at least.

I certainly hope it will be added, but i would say that improving the overall performance goes first,
If you get 3fps, 4 threads won't quadruple it as things are Never linear, you would probably get 6-9fps at best.
So better improve the 3fps first (Just a figure, as it's so extremely slow right now).

"d." is extremely unlikely. VP9 youtube videos just started to appear.

Right indeed, and google works on it everyday, or well it get's worked on by people at least, and i doubt many of them aren't connected to Google, as VP9 isn't really x265 famous.

[Kurtnoise]

a. Google doesn't want to add multi-threading (since Google seems to be the only one really using VP9 it seems kind of in there interest to support multi-threading) or simply they don't want to share it and only added it in some non-public repository
Since when ? I'd say that Google doesn't care coz they have plenty of datacenters all over the world...and can schedule stuff in parallel.

b. There really is some enormous design flaw which prevents the encoder from using multi-threading, but reading https://groups.google.com/a/webmproject.org/forum/#!topic/codec-devel/IFFSy2xlKV0 it sounded like nobody got around to implement it. (and that conversation is from nearly one year ago)
Well...multi-threading is available for VP8. So, we just need some time to get this also for VP9.

c. Google can't find anyone who knows enough about VP9 who could spend the time to implement multi-threading (unlikely, but you never know)
d. Google gave up on VP9 and there is no man-power planned in to support it development wise any further
I doubt that...there are plenty of commits every days (except the weekend) in their source tree.

[BadFrame]

Only reasons that pop into my mind (with Google behind it) are:
a. Google doesn't want to add multi-threading (since Google seems to be the only one really using VP9 it seems kind of in there interest to support multi-threading) or simply they don't want to share it and only added it in some non-public repository
b. There really is some enormous design flaw which prevents the encoder from using multi-threading, but reading https://groups.google.com/a/webmproject.org/forum/#!topic/codec-devel/IFFSy2xlKV0 it sounded like nobody got around to implement it. (and that conversation is from nearly one year ago)
c. Google can't find anyone who knows enough about VP9 who could spend the time to implement multi-threading (unlikely, but you never know)
d. Google gave up on VP9 and there is no man-power planned in to support it development wise any further

I can't recall if it was from the webm mailing list or IRC, but they've stated (VP9 devs) that the reason they haven't incorporated parallel encoding yet is because VP9 is still in very active development / tuning, and adding parallel encoding at this point would only complicate development and make it slower to make changes. They will add parallel encoding to VP9.

If you want to follow the (quite active) development of VP9 you can do so here:

http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=summary

[Monarc]

Only reasons that pop into my mind (with Google behind it) are:
a. Google doesn't want to add multi-threading (since Google seems to be the only one really using VP9 it seems kind of in there interest to support multi-threading) or simply they don't want to share it and only added it in some non-public repository
b. There really is some enormous design flaw which prevents the encoder from using multi-threading, but reading https://groups.google.com/a/webmproject.org/forum/#!topic/codec-devel/IFFSy2xlKV0 it sounded like nobody got around to implement it. (and that conversation is from nearly one year ago)
c. Google can't find anyone who knows enough about VP9 who could spend the time to implement multi-threading (unlikely, but you never know)
d. Google gave up on VP9 and there is no man-power planned in to support it development wise any further

e. hardware encoder?

[benwaggoner]

I can't recall if it was from the webm mailing list or IRC, but they've stated (VP9 devs) that the reason they haven't incorporated parallel encoding yet is because VP9 is still in very active development / tuning, and adding parallel encoding at this point would only complicate development and make it slower to make changes. They will add parallel encoding to VP9.
I suspect it's a lot harder to parallelize VP9 since the lack of B-frames makes frame-level parallelism somewhat harder, and the way that entropy coding and prediction are tied together makes intraframe parallelism harder. I'm sure there are ways to crack those nuts. After all, x254 can do frame level parallelism to some degree with Baseline profile which lacks B-frames.

I'd think getting parallel encoding going to some degree would be very helpful, though, because:

It proves it is possible, to address the "I'm not going to look at VP9 until I see a fast enough encoder" position. When you're use to 12-16 physical cores for encoding, a single-threaded encoder is maddeningly slow, and sometimes you want to get one file done fast, not a bunch of files done in parallel slowly.
It's allow the developers to iterate more quickly on testing particular changes. Having to wait 1 hour instead of 8 hours to see the results of a change is pretty massive.


If you want to follow the (quite active) development of VP9 you can do so here:

http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=summary
Yeah, that is a very impressive cadence. More than x264+x265 put together. Although it's hard to tell how material each change is; the notes in the full log aren't very informative. A lot of it sounds like code cleanup to make future work easier, and removing a lot of unused cruft. I always like it when a checkin indicates that it will have a speed or output quality impact, and at least a quick description of what that would be.

[BadFrame]

I suspect it's a lot harder to parallelize VP9 since the lack of B-frames makes frame-level parallelism somewhat harder, and the way that entropy coding and prediction are tied together makes intraframe parallelism harder. I'm sure there are ways to crack those nuts. After all, x254 can do frame level parallelism to some degree with Baseline profile which lacks B-frames.

I recall reading something about how golden frames and alt-ref frames would mitigate the lack of b-frames to a large extent, but that is a bit above my head.


I'd think getting parallel encoding going to some degree would be very helpful, though, because: ...

I fully agree with your reasons, and I personally find VP9 pretty much unusable for anything but small encoding tests until it offers multi-core encoding per clip.

I always like it when a checkin indicates that it will have a speed or output quality impact, and at least a quick description of what that would be.

Well I think that depends on if you look at the 'merge' or the original commit, for example, take the recent merge of 'Add static-threshold skipping in non-rd mode', it's not very descriptive, however if you look at the parent commit, it says the following:


Add static-threshold skipping in non-rd mode

Added a skipping test in non-rd inter-mode. After interpolation
prediction step, the residuals are tested to see if they will be
quantized to 0 based on modeling between spatial domain and
frequency domain.

Set static-thresh to 800 for >=720p and 300 for <720p, rtc set
tests showed
1. Speed 5, psnr: -0.514%; ssim: -1.748%;
speedup on related clips: 5% -11%
2. Speed 6, psbr: -0.628%; ssim: -1.637%;
speedup on related clips: 4% - 9%


Which is a lot more descriptive

[Nintendo Maniac 64]

Here's something interesting I just discovered today - on a CPU with SSSE3 support, VP9 requires less than half the CPU utilaztion that h.265 needs.

Since my HTPC hooked up to my HDTV (1080p) runs on a Core 2 Duo, this means that I can do 1080p VP9 with room to spare while I won't be able to do 1080p h.265 at all. Heck I could probably even do 1440p VP9 on my HTPC.

[zerowalker]

You are comparing them at similar bitrates and settings (as i am sure that has quite the impact on the decoding)?

If you are, or at least fairly close as they are quite different, then that is very impressive, very good indeed when you think of the aim of this codec where decoding needs to be highly optimize to be able to dominate..

[Nintendo Maniac 64]

Actually, I'm comparing the CPU utilization between my Brisbane and my Core 2 Duo. VP9 uses like half the CPU on my Core 2 Duo vs my Brisbane, but h.265 uses only a bit less (~10%) on my Core 2 Duo than my Brisbane.

Considering that VP9 already used about 10% CPU utilization on my Brisbane vs h.265, this results h.265 only breaking even with non-SSS3-optimized VP9.

Here's the tl;dr version:

Brisbane
VP9 ~60%
h.265 ~70%

Core 2 Duo
VP9 ~25%
h.265 ~60

While the VP9 and h.265 footage was not the same, I was sure to use the exact same VP9 clip and the same h.265 clip with both CPUs.

[xooyoozoo]

If your decoding stacked is built on libav/ffmpeg, and you need faster HEVC decode, you can always use the OpenHEVC fork (https://github.com/OpenHEVC/FFmpeg). Optimizations there eventually percolate outwards, but for whatever reasons, progress is intermittent and OpenHEVC is (still) 2x faster than ffmpeg-master.

On Sandy Bridge, both single and auto threaded software decode seem to be roughly 2/3 the speed of high profile AVC. I didn't compare directly against VP9, but the numbers here look competitive.

[Guest]

Audio is off topic in a video thread. Make a new thread about it and stop discussing it here.

[benwaggoner]

I recall reading something about how golden frames and alt-ref frames would mitigate the lack of b-frames to a large extent, but that is a bit above my head.
Yes, I think they could. But it'd require new encoder and decoder features to take advantage of this. One of VP9's big challenges is that it's having to compete as an ecosystem, not a bitstream, and there's a whole lot more companies competing and investing in H.264 and HEVC encoders and decoders than are for VP9. So one thing Google needs to demonstrate is that VP9 will actually have a competitive encoder implementation in the market. Possible in theory doesn't do anything for viewer experience.

I think the last marginally competitive VPx encoder was probably VP3, and even then only as a progressive download codec (lousy rate control) in the pre-standards market of 14 years ago. VP6 only mattered because On2 got really lucky with Flash, but even with the gift of that huge market they still struggled to make an encoder with good performance and rate control. VP6's main innovations were in post-processing, where they did a lot of postprocessing block removal and noise synthesis to mask loss of detail. Interesting stuff, but orthogonal to encoding.

Well I think that depends on if you look at the 'merge' or the original commit, for example, take the recent merge of 'Add static-threshold skipping in non-rd mode', it's not very descriptive, however if you look at the parent commit, it says the following:

Which is a lot more descriptive
Ah, thanks. That's a good feature to have. Although it is somewhat worrisome that basic stuff like that is still getting implemented (although "non-rd mode" doesn't sound like something that would get used that much). On the other hand it suggests that there could still be a lot of future improvement in VP9 visual quality.

From my perspective, I'm waiting to see the first platform for which VP9 provides the overall best customer experience inclusive of battery life, performance, playback smoothness, speed of random access, and of course quality at a given bitrate.

[BadFrame]

One of VP9's big challenges is that it's having to compete as an ecosystem, not a bitstream, and there's a whole lot more companies competing and investing in H.264 and HEVC encoders and decoders than are for VP9. So one thing Google needs to demonstrate is that VP9 will actually have a competitive encoder implementation in the market. Possible in theory doesn't do anything for viewer experience.

Well I don't think market share really matters to Google when it comes to VP9, it's not as if they are making money from it as they have released it under a perpetually royalty free licence.

As I see it (as in me speculating) Google wants their own codec which they can develop according to their needs (realtime is probably big here) as well as saving money long term on royalties, it's not as if their use of video on the web will decrease, then we have devices like Google Glass, and whatever else they've got cooking which will likely make use of video in some form.

This likely means that it's not important to them how well or not VP9 is received in the 'video market', meanwhile chipmakers targeting Android will most likely implement it in hardware as it doesn't cost them any royalties (hardware implementation is also royalty free) to do so while making their chips more 'attractive', particularly given Android's market dominance.

[foxyshadis]

OK, guys, I've split off the YouTube discussion that ate the thread (https://forum.doom9.org/showthread.php?t=170682).

[Kurtnoise]

For your information, a new branch has been created few days ago in the source tree. It concerns High Bits Depth Feature...

So, I made some fresh windows compile in order to play with :

-> vpx tools for x86 plateform (https://www.mediafire.com/?qbrqt41412af72y)
-> vpx tools for x64 plateform (http://www.mediafire.com/download/t4itk2dgscuvwr1/vpx_hbdepth-x64_1.3.0-6956f43a9.zip)
New builds uploaded concerning this branch :

http://kurtnoise.free.fr/vpx/vpx-x86_1.3.0-51790ab22.zip
http://kurtnoise.free.fr/vpx/vpx-x64_1.3.0-51790ab22.zip

>> The log (https://chromium.googlesource.com/webm/libvpx/+log/highbitdepth)

[hajj_3]

1080p vp9 youtube videos use about 20% cpu on my intel core i5 430m laptop processor, not too bad i guess. I wondered why my laptop was getting warm playing youtube videos. They seem to be using it for quite a lot of videos now.

[Nintendo Maniac 64]

Here are two short videos from Google I/O that are relevant to this thread:

Update on WebM/VP9: https://www.youtube.com/watch?v=xo_R40C7RTo

Demystifying encodes and decodes of WebM: https://www.youtube.com/watch?v=o-TAyIQBOuA

The first video is mainly about the functions and features of the codec itself along with upcoming developments; the second video is more about actually using it.

[zerowalker]

They say they have increased the speed about 40x if i remember right.
But it's still super slow, and no multi core support (Encoding wise).

And i am not into the way they say that it looks 50% better, it just doesn't, sad but true (At least when i tried it some time ago).

Other than those things, i am very happy that things are going forward, High Bit Depth, Hardware Decoding. This can really bring VP9 into the market place.

(Can't imagine the Realtime Encoding though, must be low resolution;P)

[Nintendo Maniac 64]

And i am not into the way they say that it looks 50% better, it just doesn't, sad but true.

Part of the issue is that general statements like "50% better" are really, well, general.

Different codecs respond differently at different bitrates, for example h.264 really starts to break down at the lower bitrates that YouTube focuses on. I mean, once you get into the high bitrates then even MPEG-2 will look quite good (as evident by OTA HDTV broadcasts).

[dapperdan]

http://youtu.be/GBAEG_RuqeE?t=33m28s

Another VP9 update given in the context of WebRTC. Looks like realtime encoding for video chat and screen casting is currently a focus for improvement.

Interesting stat that 60% of YouTube plays in Chrome are now VP9.

[kidjan]

Part of the issue is that general statements like "50% better" are really, well, general.

Different codecs respond differently at different bitrates, for example h.264 really starts to break down at the lower bitrates that YouTube focuses on. I mean, once you get into the high bitrates then even MPEG-2 will look quite good (as evident by OTA HDTV broadcasts).

Also, a lot of the comparisons I've seen are comparing VP9 to H.264/HEVC with main/high profile, which likely means B-frames. B-frames are a cool feature for playback and content encoding, but for low latency situations (e.g. live streaming) they're not an option. The more honest comparison would be VP9 to baseline H.264, with the caveat that H.264 could have better quality at the expense of live and/or low-latency situations. For example, this study (http://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf) is comparing VP9 to H.264 high profile with B-frames, which is totally legit for content encoding and completely bogus for live encoding. It also doesn't look at any performance below the ~1 mbps range, which is a particularly interesting range for live video streaming given most user's limited upload bandwidth.

For live streaming, my personal opinion is low-quality performance is a lot more important than high-quality performance to accommodate for adverse network conditions.

[sneaker_ger]

"Live streaming" is not really a low-latency application per se. It only needs low-latency if you require the other side to react fast, e.g. a video phone call where both sides interact or when you are remote controlling a robot.

[dapperdan]

For example, this study (http://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf) is comparing VP9 to H.264 high profile with B-frames, which is totally legit for content encoding and completely bogus for live encoding.

Isn't that the study that was consider ridiculously flawed? I remember it specifically because it claims that the VP9 encoder was released on the day they finalised the bitstream, which is a bit silly since it was equally available the day before, i.e. a public work in progress in git.

But someone else, with more knowledge, wrote it off due to one of the encoder settings for VP9 being silly if I remember correctly.

[kidjan]

Sorry, I should have been more clear: by live streaming I mean video conferencing, video calling, etc. I'm aware that "live streaming" isn't necessary low-latency.

[kidjan]

Isn't that the study that was consider ridiculously flawed? I remember it specifically because it claims that the VP9 encoder was released on the day they finalised the bitstream, which is a bit silly since it was equally available the day before, i.e. a public work in progress in git.

But someone else, with more knowledge, wrote it off due to one of the encoder settings for VP9 being silly if I remember correctly.

That's entirely possible; it's just the first thing I found when I went to look for comparisons this morning. Even if VP9 were mature at the time, it's silly in a few other regards: VP9 has no B-frames, no low-bitrate performance measurement, etc.

[sneaker_ger]

IIRC VP9 does have B frames, they just call it differently and employ a trick in an attempt to not break any patents.

[benwaggoner]

IIRC VP9 does have B frames, they just call it differently and employ a trick in an attempt to not break any patents.
Correct, although I don't know how heavily used that feature is in the current implementation.

As for "fair"comparisons, low latency live encoding is a different scenario than on-demand content. Those really should be separate tests.

[kidjan]

As for "fair"comparisons, low latency live encoding is a different scenario than on-demand content. Those really should be separate tests.

No arguments from me, it should be a separate test. And that's my point: comparing H.264 with B-frames to VP8 is a bit of a red-herring. I find that nearly all video encoding comparisons I see ignore this point, and as a "real time" guy (most of my work has a significant emphasis on latency), it's a little frustrating.

I also think in the video encoding world, people tend to see a codec that lacks a feature like bidirectional prediction as "inferior" to a codec that has such a feature, and I think that's a bit disingenuous when you take into consideration the real-world implications of that feature.

[x265_Project]

I think you'll be interested in this...
http://www.slideshare.net/touradj_ebrahimi/spie2014-hev-cvsvp9

[fumoffu]

I think you'll be interested in this...
http://www.slideshare.net/touradj_ebrahimi/spie2014-hev-cvsvp9

Yes, rather interesting, funny (sad?) how vp9 isn't better than AVC. HEVC (HM15) is doing pretty well but I wonder how well x265 would do considering it has problem with beating x264...