PDA

View Full Version : The relationship between --vbv-bufsize and --vbv-maxrate


Comatose
1st June 2009, 11:29
So, after all this time, I'm not sure exactly what they mean, and how to come up with a proper bufsize.

My goal is to restrict peak bitrate to allow streaming. Does --vbv-maxrate do this? I used to think so, but someone once told me that's not exactly what it does, or that the actual bitrate also depends on the bufsize.

Does --vbv-maxrate 1500 restrict the peak bitrate to 1.5Mbps?
Is bufsize = the amount of bits that should be kept to assure playback?

I'm very confused, so much that I'm not even sure I'm asking the right questions. Halp :<

LoRd_MuldeR
1st June 2009, 14:02
Basically, it works like that: Bufsize specifies the size of the device's buffer. That buffer is filled with a constant data rate. That "input" rate equals the CD/DVD/BD read speed (or the Network bandwidth). Still the video is allowed to have a variable bitrate. So if the bitrate of the video increases over the input bitrate, then the buffer will slowly run out of data. Hence you can have only short bitrate spikes. If the bitrate of the video falls below the input bitrate, then the buffer will slowly run full. Hence you can have only short "low bitrate" periods. VBV takes care of all this! It makes sure that the buffer will never overflow nor underflow, as that would cause playback errors. Maxrate specifies the maximum bitrate that the video can ever have. I think Maxrate is independent from the current buffer filling level, but I may be wrong...

http://img33.imageshack.us/img33/4306/weg.gif

How do you com up with the correct buffer size? Well, the buffer size is defined by the device's hardware. You need to know the correct size of the individual device ;)

Comatose
1st June 2009, 14:43
So, --vbv-bufsize = [available bandwidth]. If I want to limit the bandwidth to allow streaming, the values of bufsize and maxrate should be the same, right?

The reason I think that maxrate has something to do with bufsize is that it seems that even if you specify maxrate, it's ignored unless you specify bufsize.

I still don't understand completely, mainly because you say that "VBV takes care of all this", but don't explain how. Still, for my purposes, this is enough (if I got the streaming bitrate limit idea correct), since for all other purposes, you know which settings you should use because they're laid out in a spec or something. But it would be interesting to know, if you have the patience to explain or can link to some resource :)

LoRd_MuldeR
1st June 2009, 15:30
So, --vbv-bufsize = [available bandwidth]. If I want to limit the bandwidth to allow streaming, the values of bufsize and maxrate should be the same, right?

Uhm, no.

For example: If you have a low bandwidth, but a big buffer, you can use a video bitrate that is even lower than the available bandwidth, so the buffer will accumulate more and more data. Then after that period of "low bitrate", your video can have a bitrate spike (for a short moment) and use a bitrate much higher than the bandwidth! That's possible because we have enough data in the buffer, right before the spike. Of course we still must make sure that the buffer will never run out of data (underflow) or exceed its capacity (overflow). And that's what VBV does! So the bigger the buffer, the better for VBV, because it can allow stronger video bitrate fluctuations (which results in better quality). Anyway, the maximum bitrate fluctuation (peak bitrate) is still limited by the Maxrate parameter...

I still don't understand completely, mainly because you say that "VBV takes care of all this", but don't explain how.

Very basically, VBV keeps track of the buffer filling level. If the buffer is in danger to run out of data, then VBV forces the encoder to lower the bitrate, even if that may cause visible artifacts. This also happens in the other direction: If the buffer is in danger to overflow, VBV forces the encoder to use an even higher bitrate. In reality VBV is more complex, of course...

Dark Shikari
1st June 2009, 15:35
So, --vbv-bufsize = [available bandwidth]. If I want to limit the bandwidth to allow streaming, the values of bufsize and maxrate should be the same, right? bufsize / maxrate = number of seconds of wall clock time the decoder has to buffer before it starts playing.

That's it. Plain and simple.

Comatose
1st June 2009, 15:50
But what LoRd_MuldeR said intrigued me. Through using the correct settings, could I make it so a user who has only 1.5Mbps of bandwidth can play a video which has large spikes (6-7.5Mbps from 1.5Mbps) (with low bitrate sections inbetween - anime) never has to wait for the video to buffer, as long as the user's bandwidth is constant?

Sharktooth
1st June 2009, 15:55
sure. you need a larger buffer that can contain the spikes. it must be big enough to store the highest average bps (1 second samples) of the whole encoding plus a safety margin.

Comatose
1st June 2009, 16:06
What exactly is the "highest average bps (1 second samples)"? Is that the highest peak?

Is there a way to know this, other than encoding without any VBV restrictions first? Do you know if DGAVCDec respects the bufsize of a stream when measuring the bitrate?
If the peak bitrate of an encode is 7717, is --vbv-bufsize 8000 --vbv-maxrate 8000 correct? Should I specify something else for maxrate? Assuming the available bandwidth is 1.5Mbps (web streaming), that buffer will take 5 and 1/3th seconds to fill, right?

Also, using those settings, during a 8Mbps spike, the video will only play for 1 second before it stops to buffer, right?

In other words, the buffer has to be able to store as much data to accommodate the longest spike?

LoRd_MuldeR
1st June 2009, 16:07
Through using the correct settings, could I make it so a user who has only 1.5Mbps of bandwidth can play a video which has large spikes (6-7.5Mbps from 1.5Mbps) (with low bitrate sections inbetween - anime) never has to wait for the video to buffer, as long as the user's bandwidth is constant?

Yes. If the buffer is only big enough and had a chance to be filled enough, you can have 6 Mbps spikes, even if the available bandwidth is only 1.5 Mbps. Of course the higher the bitrate spike, the shorter it can be, because the buffer will run out of data faster. And after each spike you need a period of "video bitrate is lower than bandwidth" for the buffer to recover, before the next spike can happen. Last but not least: The bigger the buffer, the longer it will take for the buffer to reach the initial fill level. So when using a very big buffer, then users with a "slow" connection will have to wait a long time until playback begins. Last but not least, for hardware devices you don't need to think about the buffer size much, because the hardware itself restricts the buffer size...

Do you know if DGAVCDec respects the bufsize of a stream when measuring the bitrate?

Nope, it doesn't. That's why Neuron2 made his VBV verification tool:

http://forum.doom9.org/showthread.php?t=144409

Comatose
1st June 2009, 16:15
Nope, it doesn't. That's why Neuron2 made his VBV verification tool:

http://forum.doom9.org/showthread.php?t=144409
derp. As a teenager, I have zero income :|

Maybe I'll get a job in the summer, but I was planning on using that time to see how I fare with one of the x264 SoC projects >_>

LoRd_MuldeR
1st June 2009, 16:22
derp. As a teenager, I have zero income :|

Maybe I'll get a job in the summer, but I was planning on using that time to see how I fare with one of the x264 SoC projects >_>

I think Elecard StreamEye has a similar functionality. It's a 2000$ tool, but you can have a free trial ;)

http://www.elecard.com/products/products-pc/consumer/streameye-tools/

Comatose
1st June 2009, 16:23
Thanks :P Did you notice I edited post #8? Am I totally off again? :E

G_M_C
1st June 2009, 16:39
So, after all this time, I'm not sure exactly what they mean, and how to come up with a proper bufsize.

My goal is to restrict peak bitrate to allow streaming. Does --vbv-maxrate do this? I used to think so, but someone once told me that's not exactly what it does, or that the actual bitrate also depends on the bufsize.

Does --vbv-maxrate 1500 restrict the peak bitrate to 1.5Mbps?
Is bufsize = the amount of bits that should be kept to assure playback?

I'm very confused, so much that I'm not even sure I'm asking the right questions. Halp :<

Want to know what values to use ?

I've found this site that helps me;
http://handbrake.dynaflashtech.net/cgi-bin/vbv_calculator.cgi

I use a buffer period of 1 second, cause i enc. mainly DIY Bluray's. Dont know the buffering period of streamed video though.

Comatose
1st June 2009, 16:47
That seems awesome, thanks.

With an ABR of 1.5Mbps, a 2s long peak, vbv-maxrate=4000 and vbv-bufsize=8000, it says it'll be able to withstand a 8Mbps peak for 2 seconds. But I don't understand why. My understanding is that maxrate limits the peaks, so the peak bitrate will never exceed 4Mbps with --vbv-maxrate 4000. What am I doing wrong?

edit: It seems the ABR value doesn't serve a purpose other than to check abr < maxrate. edit2: And the fps setting doesn't do anything either... how reliable is this tool? :|

Dark Shikari
1st June 2009, 16:52
That seems awesome, thanks.

With an ABR of 1.5Mbps, a 2s long peak, vbv-maxrate=4000 and vbv-bufsize=8000, it says it'll be able to withstand a 8Mbps peak for 2 seconds. But I don't understand why. My understanding is that maxrate limits the peaks, so the peak bitrate will never exceed 4Mbps with --vbv-maxrate 4000. What am I doing wrong?If the buffer is completely full and contains 8 megabits of data, 4 megabits of data per second is coming in, and 8 megabits of data per second is going out, the buffer will empty in 2 seconds.

Comatose
1st June 2009, 16:54
So the maxrate is not actually the peak bitrate - as in the highest bitrate that can exist in the stream, it's merely the max bitrate that can enter the buffer.

Your previous post makes sense to me now. Anyway, the buffer will fill completely before decoding is first started, right?

Dark Shikari
1st June 2009, 16:55
So the maxrate is not actually the peak bitrate - as in the highest bitrate the stream can have, it's merely the max bitrate that can enter the buffer. Right?Correct.

Comatose
1st June 2009, 16:58
Awesome, thanks. I finally understand VBV :D

edit: One last question... when x264 encodes with auto for bufsize and maxrate, can DGAVCDec, Bitrate Viewer or tools like them correctly measure the bitrate?
edit2: Actually, I can use the StreamEye trial to check the peak then compare to other tools.

LoRd_MuldeR
1st June 2009, 17:06
One last question... when x264 encodes with auto for bufsize and maxrate, can DGAVCDec, Bitrate Viewer or tools like them correctly measure the bitrate?

If you don't set the VBV parameters explicitly (I think you must set both), then x264 will not use VBV at all.

Comatose
1st June 2009, 17:08
Really? MeGUI says "(automatic)" on the buffer value, so I figured it does, but it decides on the optimal value. Kinda like B-frame mode where it'll pick between Spatial and Temporal when set to Auto.

Anyway, the goal was to find the peak bitrate so that I can re-encode with the optimal VBV settings. Strangely, neither Bitrate Viewer nor DGAVCDec report the same peak as StreamEye, though DGAVCDec is off by less than 10Kbps while Bitrate Viewer is off by more than 600Kbps.

LoRd_MuldeR
1st June 2009, 17:11
Really? MeGUI says "(automatic)" on the buffer value, so I figured it does, but it decides on the optimal value. Kinda like B-frame mode where it'll pick between Spatial and Temporal when set to Auto.

I don't think so. It's not like the encoder can automatically find the "optimal" VBV values. The VBV restrictions are "hardwired" in the individual playback device.

If you are targeting for a device that requires VBV, you must know the correct VBV values for that device and you must tell x264 these values!

And if you are not targeting for a VBV restricted device, then there is no reason to use VBV, because VBV can only hurt quality, never improve the quality...

Comatose
1st June 2009, 17:14
I guess finding the optimal VBV values makes no sense, though up until now I had no clue what VBV does in practice, only the purpose it serves.

I originally thought they were mandatory and that x264 would pick the minimal values for this encode, or something like that.

G_M_C
1st June 2009, 17:14
[...]
edit2: And the fps setting doesn't do anything either... how reliable is this tool? :|

fps doesnt have anything to do with it; Only amount of bits per second, and thats independend of fps ;)

LoRd_MuldeR
1st June 2009, 17:16
fps doesnt have anything to do with it; Only amount of bits per second, and thats independend of fps ;)

No, it does! Double the framerate and you also double the number of bits per second :p

Comatose
1st June 2009, 17:16
Durr. Yeah, only the encoder cares about the fps so it knows how to distribute the bits, but a tool like this wouldn't. Another great brain fart. Why even bother have a field for FPS there, though?

Another question for you guys: is it valid to consider the peak bitrate from a non-VBV restricted encode to set the VBV values for a restricted encode? Just making sure...

LoRd_MuldeR
1st June 2009, 17:23
Another question for you guys: is it valid to consider the peak bitrate from a non-VBV restricted encode to set the VBV values for a restricted encode? Just making sure...

The VBV parameters are "hardwired" in your target playback device. You either set the suitable VBV parameters for your individual target device or you don't use VBV at all.

(For network streaming the VBV parameters are also predefined, by the network bandwidth and by the buffer size of the client)

Comatose
1st June 2009, 17:24
I want to use VBV to avoid buffering (in the middle of playback) while streaming from the web (Flash player).

LoRd_MuldeR
1st June 2009, 17:27
I want to use VBV to avoid buffering (in the middle of playback) while streaming from the web (Flash player).

That's exactly what VBV does. It avoids buffer underflow in the middle of playback. Given you specify the correct VBV parameters!

Consequently for that application you will need to know two things: The minimum network bandwidth (-> vbv-maxrate) and the buffer size used by Flash Player (-> vbv-bufsize).

poisondeathray
1st June 2009, 17:27
I want to use VBV to avoid buffering (in the middle of playback) while streaming from the web (Flash player).

What do you mean by streaming? True streaming (rtsp), progressive download, or psuedostreaming?

poisondeathray
1st June 2009, 17:33
No, it does! Double the framerate and you also double the number of bits per second :p

Only if you assume you doubled the bitrate for the double framerate version.

LoRd_MuldeR
1st June 2009, 17:36
Only if you assume you doubled the bitrate for the double framerate version.

I was assuming you don't re-encode and simply double the frame-rate of your existing encode :p

Comatose
1st June 2009, 17:41
What do you mean by streaming? True streaming (rtsp), progressive download, or psuedostreaming?
Well, technically, progressive download, but through VBV, I hope to achieve the streaming trait of no buffering once playback started.

poisondeathray
1st June 2009, 18:01
Well, technically, progressive download, but through VBV, I hope to achieve the streaming trait of no buffering once playback started.

Then I'm not sure I see how useful VBV is in your case. The bottleneck is still the up/down connection. Even if you had "proper" VBV settings, it would pause once the buffer is exhausted if the connection is the limitation. It's just as easy to set a delay on server side for flash settings, and you wouldn't get the VBV quality issues. The "buffer" would be the progressively downloaded amount.

Comatose
1st June 2009, 18:12
I was thinking since it would initially fill completely - if I made it a bit bigger than the peak bitrate (and big enough to last the entire duration of the longest, largest peak), it would be fine. When the peak comes, it would empty but once the peak is over, it would have time to refill because most of the video is lower than average (anime).

Comatose
1st June 2009, 18:34
I'm totally confused now. I got the impression that --vbv-maxrate doesn't affect the local peaks at all, but it does! In other words, to have a smooth progressive download (video only) with a 1.5Mbps connection, ABR must be 1.5Mbps, vbv-maxrate = peak in unrestricted encode + 5%, vbv-buffer = vbv-maxrate*(length of longest peak)... or at least that's what I think. Time to test.

I'll encode and upload, then restrict my bandwidth to 1.5Mbps.

poisondeathray
1st June 2009, 18:36
That still won't help you if the end user connection (usually the case), or your server upload connection, is the bottleneck. You have no control over the end user connection, so the buffer might get exhausted despite your carefully selected settings.

If you just adjust the inital fill or delay from the flash vars (e.g. it's called "bufferlength" in jw flash player), it allows the end user to download an inital amount before playing (essentially a buffer), so downstream you less likely get pausing. It's analogous to using VBV in the progressive download scenario, but without the VBV issues.

I still don't see much benefit of using VBV in your case

Comatose
1st June 2009, 18:41
Hm, yeah, that seems like a good idea. My settings would allow no pauses if the downstream bandwidth stays at 1.5Mbps throughout. I don't think VBV would cause any issues in my case because the maxrate is going to be higher than the peak rate anyway - it was just to have the buffer.

Thanks, this is a much better alternative. It's also less time consuming, since I don't have to encode twice.

poisondeathray
1st June 2009, 18:53
I don't think VBV would cause any issues in my case because the maxrate is going to be higher than the peak rate anyway - it was just to have the buffer.


Actually the "issues" I was referring to were quality issues. If you do a quick search, many of the x264 problems discussed in this forum (e.g. blocky fades, random corruption, etc...) were eventually traced back or related to VBV. As Lord Mulder pointed out earlier, "VBV can only hurt quality, never improve the quality..."

Comatose
1st June 2009, 19:01
Yes, but wouldn't that only happen if the vbv-maxrate was lower than peak rates? Since in my case it's higher, it doesn't really do anything. "Logically" :P

poisondeathray
1st June 2009, 19:18
Yes, but wouldn't that only happen if the vbv-maxrate was lower than peak rates? Since in my case it's higher, it doesn't really do anything. "Logically" :P

Nope; there's 2 different issues here.

One refers to playback and buffering.

The second refers to image quality (like blocky fades and corrupted frames), not studdering playback. I'm talking regardless of the vbv-maxrate or peak rate. Do a quick search: many of the x264 problems have traced back to using VBV (despite different settings)