Setting tweaks for mobile delivery

[Gregleto]

Hi all,

I'm relatively new, but have spent many hours reading a wide range of topics on this forum. It’s been extremely helpful and I’ve been enjoying it very much. Thanks to everyone who contributes on a regular basis. However, I’ve only seen sporadic advice about encoding for multiple device types… especially mobile… so I’d like to pose a couple of questions.

1. Is it sometimes/never/always a bad idea to use CRF to encode for mobile devices (assuming a CRF that keeps the average bitrate below the target for a specific device)? I know there will sometimes be big peaks that go way over the average, but with progressive download I’m thinking those might not pose a big problem on 3G and better networks. Thoughts?

2. Are there a few setting tweaks or other tips related to mobile encoding that most of you would consider absolutely critical to ensure a good experience? For example, is it critical to use VBV for files targeted at mobile devices? Anything else we should pay particular attention to?

Any advice would be greatly appreciated.

Gregleto

[sneaker_ger]

Usage of --vbv-maxrate and --vbv-bufsize can be indeed be critical to ensure neither network bandwidth nor decoder limits are exceeded. Whether you combine these two with --crf or --bitrate does not matter. There is no "one size fits all" - everything depends on the network and device you are aiming at.

[benwaggoner]

1. Is it sometimes/never/always a bad idea to use CRF to encode for mobile devices (assuming a CRF that keeps the average bitrate below the target for a specific device)? I know there will sometimes be big peaks that go way over the average, but with progressive download I’m thinking those might not pose a big problem on 3G and better networks. Thoughts?
I think CRF with a --vbv-maxrate is a fine thing to use for mobile. The maxrate and bufsize are what controls the peaks, and you want those to be what the network can take. CRF just saves some bits during easy parts of the video, which is a good thing.

Even if you were doing RTSP-style streaming it is always okay to use fewer bits than the nominal bitrate. This reduces network congestion, customer metered bandwidth costs, and sometimes you get lucky to have a bitrate drop happen the same time as a bandwidth drop.

[Gregleto]

I think CRF with a --vbv-maxrate is a fine thing to use for mobile. The maxrate and bufsize are what controls the peaks, and you want those to be what the network can take.

Thanks Ben,

So what's a good way to set the maxrate and bufsize based on network speeds (for progressive download)? As you can see in the most recent speed tests (link to chart below), average download speeds for 3G are now above 1Mbit/sec (forget about Sprint), and in the case of AT&T, well above. 4G is pulling an average download speed nationally of well over 10Mbits/sec. So if we want to set a lowest common denominator of delivery over 3G at 1Mbit/second, how would you set the maxrate and bufsize?

http://www.pcmag.com/image_popup/0,1740,iid=381175,00.asp

Thanks again,

Gregleto

[IanB]

Given the "%500kbps streaming success" column for none of them is 100%, you probably need to be targeting a maxrate < 500k. Basically this means at some time during playing the test clips their VBV buffer got to be empty. i.e. you need to be conservative in what the long term maximum average bit rate can be.

In setting the bufsize you need to examine the client player base to see what the minimum and typical values are. Of course you would like to set quite a high value here to iron out the peaks and troughs in the actual network throughput, but if none of your clients players support this much buffering it won't help. i.e. you need to tell the encoder how much buffer it really has to work with. Many players configure their buffer is seconds, to convert to bytes multiply the seconds by maxrate, e.g. for 500k maxrate and 10 seconds buffering give 5000k of buffer.


For the encoder working with VBV, it makes the assumption that data will at least trickle into the buffer at maxrate, it may be faster but it will have at least that much to work with. It also assumes when the buffer is full, the data rate drops to zero and the player starts with a nearly full buffer. As the encoder goes about compressing each frame it calculates how much data the player will be removing from the buffer and how much new data will trickle into the buffer, it adjusts the level of compression to ensure the player will not empty the buffer. If the encoder writes big frames, the player will use big amount from the buffer. If the encoder writes small frames the player will use small amounts from the buffer and allow the trickle from the network to fill the buffer. In combination with CRF mode the encoder check the predicted amount in the buffer and if the compressed frame size is small enough to not risk emptying the buffer it uses the CRF frame size. If the buffer is about to run empty is compresses the frame to a smaller size (at lower quality).

[Gregleto]

Thanks so much for the valuable feedback. All of this leads me to wonder if trying to create a single file that can work well for both web and mobile playback can be a viable option. I've always been under the impression that as long as our overall bitrate stays within the the range of available network bandwidth, AND we use the profile that fits a wide range of devices (Baseline 3.0 in this case), the same file could perform well on both platforms. This may still be true, (I admit I'm still getting familiar with VBV) but I'm wondering what the impact of setting the Maxrate at 500k has on playback in a player running on a PC with broadband speeds?

The baseline profile is already a big compromise for dual platform (web/mobile) compatibility. If setting a relatively low maxrate affects web performance, that would be a double whammy.

[Blue_MiSfit]

This is why you use adaptive streaming :)

HLS is rapidly becoming ubiquitous for this purpose. 3 layers covers most SD use cases, and you can do a decent job with only 5 for SD + HD.

You're gimping yourself pretty heavily by only encoding using Baseline profile.

[Gregleto]

Thanks for your input!

HLS may be getting out of the scope of this forum (I don't know), so I'll try to be brief. There are two good reasons for not using HLS right now. One is compatibility. You'll find an excellent overview of that issue at this link...

http://www.longtailvideo.com/blog/31646/the-pain-of-live-streaming-on-android/

The second reason is related to storage. The main purpose (for us) to try and create a single file for both web and mobile is because two files takes up close to double the storage, which is costly with a library of 15k videos. HLS with even three layers would increase storage requirements about the same amount, and still leave us with the need to create at least one more file for android compatibility. This approach approximately triples our current storage requirement.

It looks like we may need to create separate files for web and mobile, once I'm confident we that VBV will make a difference in a file targeted to mobile, and we know how to set it appropriately. One thing that keeps me wondering is the fact that our current baseline file is playing back fine (no buffering) on both IOS and Android without using VBV at all (so long as we're getting decent 3G speeds), so there's something about VBV I'm not quite understanding yet.

Again, thanks for the input.

Gregleto

[IanB]

... One thing that keeps me wondering is the fact that our current baseline file is playing back fine (no buffering) on both IOS and Android without using VBV at all (so long as we're getting decent 3G speeds), so there's something about VBV I'm not quite understanding yet.
This means that the bit rate distribution of your default encode, with the default buffering in your target players is within the MAX RATE of "decent 3G speeds".

VBV calculations in the encoder are an extra safety net to constrain (by reducing quality) the peak bit rate in high demand parts of the video content.

By telling the encoder how much buffering is in the player and how much bandwidth you have available it can output a stream that is guaranteed not to exceed that constraint. Hence playback will not under-run the buffer and have to stop playback while the buffer fills up again.

Stated another way, for an N second buffer, the streams average bit rate over any N second period will not exceed the specified max rate. E.g. for a 10 second buffer and a 500k max rate, no 10 second fragment of the stream will be bigger than 5000k bits.

Consider this, a good 3G channel gives you 4mbits/sec, but with conflict and congestion you only get that 10% of the time, so long term you really only have a 400kbit/sec channel.

[Gregleto]

Thanks!

So as an example, let's assume a file with a maxrate of 500k and a buffer size of 5000k. Will those settings have any impact on playback over broadband... like on a PC with a fat 20MB/sec download connection? This gets to the question of whether or not it makes sense to try and create a single file for both broadband and mobile connections.

Thanks again,

Gregleto

[IanB]

With a file encoded for a maxrate of 500k and for a buffer size of 5000k, it will play fine over any pipe fatter than 500k, but any part of the stream where the 5000k/500k=10 second average bit rate would nominally have exceeded 500k then the quality will be reduced to the level available within 500k.

Whether this is significant depends on the material, if it naturally is easy to compress and you get your chosen quality at a nominal rate lower than 500k anyway then the vbv restrictions will not need to be invoked. However if the material is visually complex and many scenes frequently require more then 500k (10 second average) for the compression level you have chosen then the quality will be reduced in those scenes.

Remember we are talking about average bit rate for the length of the buffer, extreme example :- 50,000k for 0.1 seconds then 0k for 9.9 seconds is within our example vbv constraint so will not be molested.


Perhaps a more targeted approach might help. Test encode each video at your chosen CRF. Look at the size of the file, calculate the average bit rate. Decide if it is a suitable candidate for your chosen target pipe size. If no, then re-evaluate this clip, maybe reduce dimensions, or raise CRF or offer 2 versions. If yes, then re-encode at that CRF with appropriate VBV constraints. Check how much smaller the VBV encode's file is, this will give an indication how often the VBV constraints needed to be enforced. If the VBV file is only slightly smaller then the VBV constraints were only occasionally invoked, no body will probably ever notice. If the VBV file is seriously smaller then many scenes need excessive bit rate, re-evaluate this clip. Of course a good rate analysis tool can avoid all this futzing around.

[benwaggoner]

HLS may be getting out of the scope of this forum (I don't know), so I'll try to be brief. There are two good reasons for not using HLS right now. One is compatibility. You'll find an excellent overview of that issue at this link...

http://www.longtailvideo.com/blog/31646/the-pain-of-live-streaming-on-android/
Encoding for HLS is certainly in-scope for this forum.

Easy compatibility of multi-bitrate streaming in web clients is a pipe dream at this point. Honestly, a robust network video experience on Android really requires a custom app that incorporates an adaptive streaming library of some sort (HLS or Smooth Streaming, and probably DASH before long).

The only real way to deliver fixed quality across devices is simple progressive download. But that means that playback won't start in real-time for users who have a lower connection speed that the bitrate of your stream. That would be a single .mp4 file that would play everywhere.

The second reason is related to storage. The main purpose (for us) to try and create a single file for both web and mobile is because two files takes up close to double the storage, which is costly with a library of 15k videos. HLS with even three layers would increase storage requirements about the same amount, and still leave us with the need to create at least one more file for android compatibility. This approach approximately triples our current storage requirement.
What's the average duration of that content? What are you playing for storage? And what maximum bitrate are you targeting for the web stream? If you are targeting web delivery up to HD, the size of a single android stream is going to be tiny compared to start. And if you're averaging even a hit a day for each stream, your bandwidth costs could wind up higher than your storage costs anyway.

It looks like we may need to create separate files for web and mobile, once I'm confident we that VBV will make a difference in a file targeted to mobile, and we know how to set it appropriately. One thing that keeps me wondering is the fact that our current baseline file is playing back fine (no buffering) on both IOS and Android without using VBV at all (so long as we're getting decent 3G speeds), so there's something about VBV I'm not quite understanding yet.

If you don't specify a VBV, than it won't be constrained. Depending on content and other settings, the VBV might never be anything concerning, or it might wind up getting out of spec. I always recommended at least limiting VBV to the maximum supported by your Profile @ Level combo. And if you are doing progressive download, no more than 5x your peak or average bitrate would also be an appropriate limit.

But you've got much bigger architectural question to ask than VBV, it seems like. Can you describe your scenario and use case in more detail?

[Gregleto]

The only real way to deliver fixed quality across devices is simple progressive download. But that means that playback won't start in real-time for users who have a lower connection speed that the bitrate of your stream. That would be a single .mp4 file that would play everywhere.

We're not prepared to go in the direction of HLS (or DASH) at this time, so we continue to be all about progressive download. In the past we've been using Flash and delivering only one size (640x360). Now we're re-encoding to MP4 and adding a 720p version which will be selectable in the player. Our goal was to create a single 360p version in baseline profile so it could be used for both web and mobile playback, but that's beginning to look like too much of a compromise to make it work optimally for both platforms. That decision is what prompted this post to begin with. So far, with our re-encodes we haven't been setting the VBV, but it seems VBV might be more important for lower bandwidth mobile delivery than in broadband web use cases. That's my perception so far.

What's the average duration of that content? What are you playing for storage? And what maximum bitrate are you targeting for the web stream? If you are targeting web delivery up to HD, the size of a single android stream is going to be tiny compared to start. And if you're averaging even a hit a day for each stream, your bandwidth costs could wind up higher than your storage costs anyway.

We do over 250 million views per month across 15,000 videos. In the past we've only created one flv file per video, so our bandwidth costs always exceed our storage costs. That doesn't make storage trivial however. The average length of our videos are about 3-1/2 minutes (with a range across the whole library of 30 seconds to 7 minutes). Our storage on the CDN is $.10 per GB. We don't have a maximum bitrate now because we are re-encoding using CRF. Most of the content is CRF 26, but we have a couple of shows we do at CRF 24 for quality reasons. This makes the range of bitrates pretty wide. This gives a relatively simple video an overall bitrate of around 700 kbps (or lower), with especially complex videos clocking in at 1200 kbps or more. If I had to guess I'd predict an average of around 700-900. In the past with our Flash encodes we used three bitrates... 500k, 900k and 1300k. For the past year or two we've pretty much made 1300k the standard. The good news is that x264 is giving us equivalent or better quality at a 30-40% drop in file sizes. That's huge!

If you don't specify a VBV, than it won't be constrained. Depending on content and other settings, the VBV might never be anything concerning, or it might wind up getting out of spec. I always recommended at least limiting VBV to the maximum supported by your Profile @ Level combo. And if you are doing progressive download, no more than 5x your peak or average bitrate would also be an appropriate limit.

In the past we've encoded in Flash, and I don't recall that there is even an equivalent to VBV in Flash. If there is, we didn't set it. It's only now that we're converting to MP4s that it has come up as a concern. Our Vantage encoding system does not automatically set the VBV to meet profile requirements, so we need to set it manually if we want it. However, in my research I don't recall ever seeing anyone describe problems with playback related to the VBV being out of spec, though I'm far from experienced in this area. I've also never seen problems from it being within spec :), so unless we get information that's contrary, I think we're going to go ahead and set VBV at the max setting according to the profile/level specs. Of course, the max VBV settings for every profile are much higher than our average bitrates, so I don't think setting VBV to max rates is really going to help us much... especially on the mobile side.

But you've got much bigger architectural question to ask than VBV, it seems like. Can you describe your scenario and use case in more detail?

Yes we do! But I don't think there is a right or wrong answer because everything is a trade off between quality, performance, and cost. We're trying to arrive at acceptable quality and performance at the lowest possible cost. I think we can accomplish that. What we're struggling with is making sure we don't make some decision(s) that backfire on us and deliver poor performance in a significant number of cases... especially on the mobile side. We're not trying to service every legacy device with mobile, just iPhones and Android devices delivered over the past few years and running on 3G and 4G.

With regard to our scenario, I'v provided most of it above.
-15,000 videos with more added daily
-250m+ views per month (ad supported)
-Rapid growth (we could see a billion views in the next 90 days)
-Views are spread out over hundreds of different web properties through a proprietary syndication platform we built.
-All delivery progressive download via Limelight (CDN)
-Proprietary player with both Rate Limiting and Pseudo-streaming active (to control bandwidth costs and permit skipping ahead during download)
-Telestream Vantage system for automated encoding and delivery to CDN via multiple watch folders.
-Everything shot in HD and cut in FCP.

I'm sure we don't compare in size or scale to a company like Amazon, but we have a lot of the same issues related to scale and delivery to end users. So far this forum has been immensely helpful, and I intend to keep listening and hope to be able to help others as I gain more knowledge. I'd also love to find an organization or forum where the majority of participants are others in large scale video deployments. Since this is advertised as a forum for "...everyone interested in DVD conversion.", I'm afraid my posts may be wasting the time of many participants here. I'm not sure, but I don't want to wast anyone's time.

Thanks again for all the advice!

Gregleto

[benwaggoner]

Now we're re-encoding to MP4 and adding a 720p version which will be selectable in the player. Our goal was to create a single 360p version in baseline profile so it could be used for both web and mobile playback, but that's beginning to look like too much of a compromise to make it work optimally for both platforms. That decision is what prompted this post to begin with. So far, with our re-encodes we haven't been setting the VBV, but it seems VBV might be more important for lower bandwidth mobile delivery than in broadband web use cases. That's my perception so far.
Note most mobile devices can do at least Main 3.0 these days. If you don't need to support old phones, that could work well for both mobile and low-bitrate web.

VBV isn't that important from a networking perspective when doing progressive download. However by default x264 can make files with an illegally high VBV, so specifying a maximum ensures that you won't break decoders.

We do over 250 million views per month across 15,000 videos. In the past we've only created one flv file per video, so our bandwidth costs always exceed our storage costs. That doesn't make storage trivial however. The average length of our videos are about 3-1/2 minutes (with a range across the whole library of 30 seconds to 7 minutes). Our storage on the CDN is $.10 per GB. We don't have a maximum bitrate now because we are re-encoding using CRF. Most of the content is CRF 26, but we have a couple of shows we do at CRF 24 for quality reasons. This makes the range of bitrates pretty wide. This gives a relatively simple video an overall bitrate of around 700 kbps (or lower), with especially complex videos clocking in at 1200 kbps or more. If I had to guess I'd predict an average of around 700-900. In the past with our Flash encodes we used three bitrates... 500k, 900k and 1300k. For the past year or two we've pretty much made 1300k the standard. The good news is that x264 is giving us equivalent or better quality at a 30-40% drop in file sizes. That's huge!
Yeah, x264 is a lot better than h.263 or vp8 from flv. Plus you can use AAC to save some bits on audio. And all mobile devices can do HE AAC, so you can go down to 48 Kbps with pretty good quality.

For 720pencodes, I recommend High Level 3.1. Pretty much anything that can play HD can do it in High Profile.

In the past we've encoded in Flash, and I don't recall that there is even an equivalent to VBV in Flash. If there is, we didn't set it. It's only now that we're converting to MP4s that it has come up as a concern. Our Vantage encoding system does not automatically set the VBV to meet profile requirements, so we need to set it manually if we want it. However, in my research I don't recall ever seeing anyone describe problems with playback related to the VBV being out of spec, though I'm far from experienced in this area. I've also never seen problems from it being within spec :), so unless we get information that's contrary, I think we're going to go ahead and set VBV at the max setting according to the profile/level specs. Of course, the max VBV settings for every profile are much higher than our average bitrates, so I don't think setting VBV to max rates is really going to help us much... especially on the mobile side.
VP6 does have a syntactically weird way to do buffer size, but the details escape me.

Yes, use --vbv-bufsize and --vbv-maxrate. At your bitrates they're not likely to do that much, but can help out with some crazy hard content.


We're not trying to service every legacy device with mobile, just iPhones and Android devices delivered over the past few years and running on 3G and 4G.
You are probably fine to use Main, not Baseline, then.

Since this is advertised as a forum for "...everyone interested in DVD conversion.", I'm afraid my posts may be wasting the time of many participants here. I'm not sure, but I don't want to wast anyone's time.
Nah, this is straight on great stuff for the Video Encoding section. Doom9 has evolved a fair amount over the years :).

[Gregleto]

Note most mobile devices can do at least Main 3.0 these days. If you don't need to support old phones, that could work well for both mobile and low-bitrate web.

Of course it depends on what you call an "old" phone. The iPhone 3G needs baseline and it was still being sold in 2010. The bigger problem is Android support. There are android tablets and phones less than 2 years old that still require baseline. So, unless I'm mis-informed, main profile will leave a pretty significant number of devices unsupported.

VBV isn't that important from a networking perspective when doing progressive download. However by default x264 can make files with an illegally high VBV, so specifying a maximum ensures that you won't break decoders.

We're going to set the VBV according to profile just to be safe. I assume you mean that it's "possible" to make illegally high VBV setting because it doesn't automatically enforce them? This raises a question, however. Can an illegally high VBV "break" a decoder if the actual bitrate doesn't exceed the limits? Finally, can you clarify what you mean by "from a networking perspective"?

Thanks,

Gregleto

[Asmodian]

By definition an illegally high VBV does exceed the limits, that is why it is illegally high.

[IanB]

We're going to set the VBV according to profile just to be safe. I assume you mean that it's "possible" to make illegally high VBV setting because it doesn't automatically enforce them? This raises a question, however. Can an illegally high VBV "break" a decoder if the actual bitrate doesn't exceed the limits?
"Breaking" a decoder generally means exceeding the maximum instantaneous "bitrate" that it can handle. The result is that the decoded frame arrives late causing a "break" in the playback.

In CRF mode x264 just throws encoding constructs (bits) at a frame to get a specified quality. So for insanely complex frames this can result in so much data that the decoder cannot decode it in time. With profile/level driven VBV constraints you ensure this does not happen i.e. encode at quality CRF 26 BUT make sure it is also less then the profile/level requirements.


When you started this thread you gave the impression you were talking about streaming delivery over 3G, now you say you do progressive delivery, so peaks and troughs in instantaneous bitrate do not matter from a network pipe perspective, you just care about the total file size and that the file is playable on the devices of choice.
Most of the content is CRF 26, but we have a couple of shows we do at CRF 24 for quality reasons. This makes the range of bitrates pretty wide. This gives a relatively simple video an overall bitrate of around 700 kbps (or lower), with especially complex videos clocking in at 1200 kbps or more. If I had to guess I'd predict an average of around 700-900. In the past with our Flash encodes we used three bitrates... 500k, 900k and 1300k. For the past year or two we've pretty much made 1300k the standard. The good news is that x264 is giving us equivalent or better quality at a 30-40% drop in file sizes. That's huge!
If you want your cake and most of the time getting to eat it as well, you could pick a mostly achievable maxrate like 1500k with a buffer sized for say 15 to 30 seconds (22500k to 45000k). Now given you say 1300k and 30-40% drop in size, this is 1300*0.7=910k average, for which a 1500k maxrate is actually quite generous.

[Asmodian]

-Proprietary player with both Rate Limiting and Pseudo-streaming active (to control bandwidth costs and permit skipping ahead during download)

This makes me think setting bandwidth appropriate VBV values would help both bandwidth costs and allow stall free playback when pseudo-streaming, you might even be able to lower CRF a small amount. Is there a difference in user experience between pseudo-streaming and streaming when average bitrate is close to the bandwidth available?

I know I hate it when I am watching a video online and it stalls a second or two into a high motion scene. It seems to me that you would want to keep the data required for 10 seconds of video close to the amount of data available in 10 seconds. I prefer lower quality high-motion scenes (VBV kicking in) to stalling and rebuffering part way through them.

[Gregleto]

Is there a difference in user experience between pseudo-streaming and streaming when average bitrate is close to the bandwidth available?

There's not a big difference in user experience, but there is a difference from a business perspective. Pseudo-streaming happens over http, whereas real time streaming like RTSP requires a streaming server and is blocked more frequently by corporate firewalls. Pseudostreaming also helps us save bandwidth costs.

Pseudso-streaming simply enables fast seeking to not-yet-downloaded portions of the video when delivering via http. It mimics real streaming by using a little trickery on the http server. We haven't tried it yet on mobile, but we use it extensively on web delivery. It's a bit of overkill for short content like ours, but when paired with rate limiting it provides the best of both worlds from a cost saving perspective.

Rate limiting keeps the file downloading just ahead of the playback, and stops the download if the viewer pauses the video. This prevents the expensive situation where on broadband connections the file fully downloads to the browser cache 15-20 seconds into the video playback. If the viewer bails out of the video at that point, we've already spent the bandwidth for the full video.

So with the combination of pseudo-streaming and rate limiting we can effectively mimic the real time streaming experience and get its benefits without some of the downside of real time streaming.

[Gregleto]

When you started this thread you gave the impression you were talking about streaming delivery over 3G, now you say you do progressive delivery, so peaks and troughs in instantaneous bitrate do not matter from a network pipe perspective, you just care about the total file size and that the file is playable on the devices of choice.

This intrigues me. You say that with progressive delivery we "... just care about the total file size and that the file is playable on the devices of choice." Are you saying that VBV is not really relevant or important to delivery via progressive download?

[benwaggoner]

This intrigues me. You say that with progressive delivery we "... just care about the total file size and that the file is playable on the devices of choice." Are you saying that VBV is not really relevant or important to delivery via progressive download?
Yes, since you're going to get a fair amount of buffering anyway, and there's no promise or requirement for real-time delivery of content.

Setting it to the spec max is a good idea, but it probably won't have much practical effect.

Also, with short content like this, the downsides of progressive download are much smaller.

[Asmodian]

I think I am missing something as to the difference between pseudo-streaming and streaming as it applies to VBV but I think VBV would have the same advantages for both applications, at least in some situations. The local buffer can be larger for pseudo-streaming so if you have extra bandwidth you can use a large enough buffer to avoid stalls even during a high motion scene but what if you do not have extra bandwidth?

If the average bitrate is close to available bandwidth without VBV set to the maximum bandwidth available you will get video stalls part way through even with pseudo-streaming won't you? At 3G speeds you would need a very low average bitrate or a long buffering time to allow the buffer to fill enough to not get stalls during a short high bitrate segment. My personal experience with such services makes me think long buffering times or random stalls can cause users to wander off. With VBV set to something reasonably conservative couldn't you reduce the amount of video buffered (saving bandwidth when a user stops watching early) and still not get stalls? Of course high motion scenes could have a noticeable quality drop, there is always a trade off.

While there may be no promise or requirement for real-time delivery as an end user it is better when delivered in (almost) real-time. :)

[Gregleto]

I think I am missing something as to the difference between pseudo-streaming and streaming as it applies to VBV but I think VBV would have the same advantages for both applications, at least in some situations.

I regret bringing up pseudo-streaming since we only use it now for Web delivery and I don't believe it is relevant to discussions about VBV. It is simply the player communicating to the server to restart playback at another point in the video. So I think it's best to keep it out of the discussion and bring the discussion back to the effect of VBV on video playback over 3G networks... real life network speeds of 400k-1000k.

Having said that, it appears that Ben is saying that VBV is meaningless in a progressive download environment, and Asmodian is contradicting that. (gentlemen, correct me if I've mis-stated your position).


While there may be no promise or requirement for real-time delivery as an end user it is better when delivered in (almost) real-time. :)

I absolutely agree. And that's all we're trying to accomplish to the best of our ability.

Thanks!

Gregleto

[Asmodian]

You have my position correct but I am more asking if there really is no advantage to setting VBV to limit bitrate spikes much above the available bandwidth. benwaggoner is obviously much more experienced on the server side of streaming video.

Unless you download a lot of the video before starting playback (a long buffering time at 400k) there would be an advantage to controlling bit rate spikes.

With an average bitrate of 350 kbps and available bandwidth of 400 kbps. Trying an example: the first 10 seconds are at the average, the next 10 seconds at 600kbps, you try to buffer 20 seconds (7000 kb), and wait 5 seconds before starting playback.

(10 * 400) + (5 * 400) = 6000 kb
6000 - (350 * 10) = 2500 kb
2500 + (10 * 400) = 6500 kb
6500 - (10 * 600) = 500 kb left in buffer 20 seconds into playback.

If you go over that 600 kbps by much you will stall playback somewhere between 10 and 20 seconds into the video.

[IanB]

This intrigues me. You say that with progressive delivery we "... just care about the total file size and that the file is playable on the devices of choice." Are you saying that VBV is not really relevant or important to delivery via progressive download?
Well, yes and no.

Yes in that with a too slow link you cannot live play the stream no matter what, so you are stuck with the old fallback model "download the file, make a cup of coffee while it comes down, watch the file". Bandwidth distribution is irrelevant here, playback is ultimately from a very fast local copy.

A definite No! in the marginal speed link case and Asmodian's example cuts right to the core of the issue. This is what VBV protects against. If we extend this example such that the second scene is 12 seconds instead of 10 at 600kbps the last lines become :-
...
2500 + (12 * 400) = 7300 kb but buffer is only 7000 kb
7000 - (12 * 600) = Oops! -200 kb missing from the buffer 22 seconds into playback. We get a stall!

With VBV active some of the second scenes rate would be constrained to 583.3 kbps to ensure no buffer under-run.

Now this example is grossly simplified, real VBV is calculated every frame, has some look ahead and has accurate modelling for I, P and B frame dependencies.

Do some tests, compare unconstrained CRF with profile/level maxrate constrained and with say a 1500kbps maxrate constrained and with say a 910kbps (the actual average bitrate) maxrate constrained.

[benwaggoner]

Unless you download a lot of the video before starting playback (a long buffering time at 400k) there would be an advantage to controlling bit rate spikes.
The typical progressive download algorithm is to start playback when the time remaining in the download is less than the duration of the file. Thus buffering is proportional to clip duration, hence progressive download is a much better fit for short form content.

With a 150 second clip, a 500 Kbps clip playing on a 400 Kbps connection would only result in 38 seconds of buffering.

Having a max-rate and vbv-bufsize keeps the clip from messing things up by being way too forward-loaded if the complexity at the start is higher than at the end.

[Asmodian]

I would call that a long buffering time but I see how VBV is less important using that method. As you point out though, it is still possible to get a stall when you could avoid it. You are in a much better position to judge weather the quality fluctuation is worth it to avoid the rare stalls.

The user will probably have lots of stalls caused by changing bandwidth rather than video bitrates. :p

If we extend this example such that the second scene is 12 seconds instead of 10 at 600kbps the last lines become :-
...
2500 + (12 * 400) = 7300 kb but buffer is only 7000 kb
7000 - (12 * 600) = Oops! -200 kb missing from the buffer 22 seconds into playback. We get a stall!

Actually, I don't think we can clip at the buffer because it is playing out at 600kbps while coming in at 400kbps. If you extend it to 13 seconds we underflow by 100kb so the point stands. ;)

[Gregleto]

All of these specific examples, with their associated math are excellent in helping to understand VBV better, which base on my experience is of the things that gives users (especially newbies) more trouble understanding than most other settings. So thanks for this input!

However, these examples use a constant network connection speed (ie. 400k) which is not how things work in the real world. Once you've made a connection on a 3G (or other) network, your available connection speed can, and usually does vary wildly even over periods as short as a few seconds. You might see an average connection speed of 900k over some period of time, but that is likely to have been made up of short bursts of 2000k, combined with short periods of 200k. It seems to me that all the theoretical calculations related to VBV go out the window once you get a file out into the wild... or do they?

This is also what makes it so hard to test on a real network. No matter what you do you always have one constantly changing variable... network speed.

Thanks,

Gregleto

[Asmodian]

I think that is exactly why you should weight benwaggoner's comments more heavily than mine, he has real world experience and I just like understanding things. :)

...but I still think a reasonable VBV would be better than not, as a single user I have had a lot of experience with a high complexity scene stalling different varieties of streaming video. Of course I have had even more experience with "internet weather" causing stalls. :p

[Gregleto]

One thing I forgot to mention is that most users we've surveyed would prefer a lower quality image to frequent buffering. A little bit of degradation in the image is far preferable to having the video stop playing even momentarily. Maybe that argues for just settling on the lowest possible size and average bitrate that will still ensure a watchable image.

On the other hand, as a video professional that kid of lowest common denominator approach makes me cringe, and it also provides a poor quality experience for all those 4G users who have plenty of bandwidth... and even 3G users with a really good connection. Unfortunately, we don't have any good way to determine the user's connection speed prior to sending the file.

I know HLS solve a lot of this, but it's only really practical for IOS, which cuts out a big chunk of the market.

[benwaggoner]

I think that is exactly why you should weight benwaggoner's comments more heavily than mine, he has real world experience and I just like understanding things. :)

...but I still think a reasonable VBV would be better than not, as a single user I have had a lot of experience with a high complexity scene stalling different varieties of streaming video.
Well, I agree with you entirely, so your understanding is aligned with the real world :).

[benwaggoner]

One thing I forgot to mention is that most users we've surveyed would prefer a lower quality image to frequent buffering. A little bit of degradation in the image is far preferable to having the video stop playing even momentarily. Maybe that argues for just settling on the lowest possible size and average bitrate that will still ensure a watchable image.

On the other hand, as a video professional that kid of lowest common denominator approach makes me cringe, and it also provides a poor quality experience for all those 4G users who have plenty of bandwidth... and even 3G users with a really good connection. Unfortunately, we don't have any good way to determine the user's connection speed prior to sending the file.

I know HLS solve a lot of this, but it's only really practical for IOS, which cuts out a big chunk of the market.
If you are making a custom app, you could license HLS or Smooth Streaming heuristics components to incorporate. That's what all the premium video apps do today.

It's the sad truth that premium video experiences simply aren't feasible in a web browser yet. Especially if you are using any kind of DRM.