Log in

View Full Version : vbv-maxrate, vbv-bufsize, and live streaming


Sapiens
19th April 2013, 23:42
I'd like to ask for some clarification from the Doom9 forumgoers regarding how vbv-maxrate and vbv-bufsize behave, specifically in relation to live streaming on sites like Twitch.tv. If possible I'd also like to get a general idea of the path data takes from the time x264 receives it to when it shows up as playback on a viewer's screen. Here is what I'm working with so far in addition to the x264 settings page on MeWiki:

https://trac.handbrake.fr/wiki/VBVRateControl
http://forum.doom9.org/showthread.php?t=147460
https://www.xsplit.com/blog.php?post_id=305

The programs being used are primarily XSplit and OBS, if that makes a difference. I'll try to type out how this is currently working in my head; it will probably be exceedingly simplistic and mostly wrong. :cool:

I have been thinking of the x264 buffer as what viewers are watching. First x264 encodes video and fills the buffer at (up to) the rate defined by vbv-maxrate, then viewers preload the buffer and away we go. If the bitrate required to achieve the quality target/crf is less than the vbv-maxrate then the buffer is able to absorb bitrate spikes for brief amounts of time.

Example: vbv-maxrate 2000 Kbps and vbv-bufsize 4000 Kbps. Initially only 1500 Kbps is required to meet the quality target and the buffer is filled at 500 Kbps (I don't know how to properly describe this... 1500 Kbps is going out, leaving 500 Kbps to accumulate each second). At some point after the buffer is full, x264 decides it needs 4000 Kbps to meet the quality target for a particularly busy scene. 4000 Kbps exits the buffer during this scene, while at the same time 2000 Kbps enters the buffer, leaving it half filled. After 1.5 seconds the buffer is "empty" and the video bitrate becomes limited to 2000 Kbps.

According to this example it seems as though the person streaming won't need more than 2000 Kbps upstream, and viewers don't need more than 2000 Kbps downstream, even if the content itself exceeds these bitrates because of the buffer.

The alternate explanation presented to me seems to be kind of inverted, and takes into account more factors like the buffer of the video player, which I conflate with the x264 buffer pretty thoroughly. Here is a quick copy and paste:
So when x264 is deciding how much data it can allocate to a given frame it must look at how full the vbv-buffer is. If the buffer is nearly empty it will have nearly no limits in the size of that frame. (limits from the buffer there are still other limits from settings).

However if the frames you have been outputing are very large (larger than maxrate in size) and your buffer is nearly full (you may never overflow the buffer) then x264 will ensure you frames are small enough to fit within the buffer. (this will cause a reduction in quality if the frame is too complex)

This will affect your bandwith by only allowing you to stream at a rate > maxrate for a limited time (determined by the size of the buffer) before you will be forece to stream at or below maxrate.

You are still actually streaming at X kbps and your viewers will need to be able to download at X kbps (or have a large enough client buffer that they can get your data spike before they run out of buffer to watch)

The client buffer should be kept as full as possible. The x264 buffer should be kept as empty as possible to allow for large spikes if you need them.

But this seems to conflict with LoRd_MuldeR's explanation (http://forum.doom9.org/showthread.php?p=1292457#post1292457)? Or I'm just grossly misinterpreting it.

Okay this is already way too long for a first post, sorry for the wall of text. Did I just quote a perfect description? Have I gone terribly wrong somewhere else? Thanks in advance for any help! :thanks:

vivan
20th April 2013, 03:29
Well, it's not really inverted... It's the same thing but from different perspective (writing vs reading).

When you're writing (encoding) you have a buffer that you can fill with any speed, but the rate at which the buffer is emptied is constant. If buffer is empty - it's good, since you can fill it with big speed; when it's full - it's bad, since you have to fill it with low speed.
When you're reading (decoding) you have a buffer that fills with constant speed, but you can empty the buffer with any speed. If buffer is full - it's good, you can empty it with any speed; when it's empty, it's bad, since you can't get anything from it.

encoding (any speed) => [buffer] => transmitting (constant speed) => [buffer] => decoding (any speed)

LoRd_MuldeR is talking about left part, you're talking about right part.

Sapiens
20th April 2013, 04:24
That makes so much more sense now, thanks a ton. To follow up, if you use x264's variables it works like:

encoding (any speed) => [encode buffer (vbv-bufsize)] => transmitting (vbv-maxrate) => [decode buffer (?)] => decoding (any speed)

Is the decode buffer also defined by vbv-maxrate? (Edit: Oops, I meant to type vbv-bufsize here. Oh well.) Or does/can the playback software, Flash in this case, independently define the buffer size on the viewer's end of things? This sounds like it wouldn't work.

Also, with this diagram in mind it seems like the video's maximum bitrate levels would be limited to the size of the vbv-buffer, but that doesn't seem to square with Manao's explanation from the first link I posted. Specifically:

<06/05/07 12:23pm> Manao | each time you encode a frame that is XXX bits big, you add XXX bits to the VBV buffer
<06/05/07 12:23pm> Manao | each time you encode a frame, you remove VBV_buffer_bitrate * frame_duration bits
<06/05/07 12:23pm> Manao | the VBV must never fill completely
<06/05/07 12:25pm> Manao | so, let's say you have a VBV buffer with a bitrate of 1mbps, and which is 1 mbit big
<06/05/07 12:25pm> Manao | if it starts empty, during one second, if you have a bitrate of 2 mbps, you'll have added 2mbit, and removed 1mbit, so the VBV will be filled
<06/05/07 12:25pm> Manao | so, the max bitrate on 1 second is 2 mbps.
<06/05/07 12:25pm> Manao | on 2 seconds, you can, at most, how a bitrate of 1.5 mbps ( +3, -2 )

<06/05/07 12:42pm> Manao | dynaflash: 5000/5000 will allow 1 second spike of 10000 mbps
<06/05/07 12:42pm> Manao | 5000/2500 will allow 1 second spike of 7500 mbps
I get the feeling I'm misinterpreting this as well.

vivan
20th April 2013, 05:22
That makes so much more sense now, thanks a ton. To follow up, if you use x264's variables it works like:

encoding (any speed) => [encode buffer (vbv-bufsize)] => transmitting (vbv-maxrate) => [decode buffer (?)] => decoding (any speed)

Is the decode buffer also defined by vbv-maxrate?

Or does/can the playback software, Flash in this case, independently define the buffer size on the viewer's end of things? This sounds like it wouldn't work.Decode buffer is defined by decoder, encoder can't do anything about it. If decode buffer is same or bigger than vbv-bufsize everything is fine, but if it's smaller - then buffer underflow could occur. That's why you tweak encoder to match your decoder.
I think flash uses quite a large buffer (e.g. you can buffer several minutes of video), so you shouldn't worry about it. The thing you should worry about is moment when playback should start. I don't remember how it's called but it's like "delay playback until X KB downloaded". This X KB would be your buffer. And it should be not less than vbv-bufsize.

Also, with this diagram in mind it seems like the video's maximum bitrate levels would be limited to the size of the vbv-buffer, but that doesn't seem to square with Manao's explanation from the first link I posted.I don't really see any inconsistencies... Don't forget that they are talking about encoding. When you think about decoding you should revert everything. E.g encoding buffer is empty = decoding buffer is full.
Let's take that 5000/5000 example. In ideal situation your decoding buffer is full + it's filled with 5000 kbit's speed. If video have 10000 kbit size for 1 second (10000 kbit/s bitrate for 1 second) than after 1 second your decoding buffer would be empty. So it would be the same 10000 kbit/s spike for 1 second.

Sapiens
20th April 2013, 06:36
Let's take that 5000/5000 example. In ideal situation your decoding buffer is full + it's filled with 5000 kbit's speed. If video have 10000 kbit size for 1 second (10000 kbit/s bitrate for 1 second) than after 1 second your decoding buffer would be empty. So it would be the same 10000 kbit/s spike for 1 second.
Ah okay, for some reason I was having trouble getting my head around emptying the decoding buffer + immediately decoding what it was being filled while it emptied. Thanks again for your help, it's much appreciated. :)