VBVChecker 1.7

[neuron2]

I've made a GUI VBV Checker for AVC elementary streams (no stats file from the encoder required!). It requires my "neuron2 supporter" license (see last paragraph below). Existing licenses for DGAVCDecNV and DGVC1DecNV will work (one supporter license will enable all my licensed tools). Your feedback will be appreciated.

http://neuron2.net/dgdecnv/VBVChecker17.zip

Following is the help file:

H.264 VBVChecker (version 1.7)
-------------------------------

When streaming or muxing to transport for broadcast, the video stream must obey an appropriate VBV model
to avoid exceeding the limits of the channel and/or the decoding device.

VBVChecker accepts an AVC (H.264) elementary stream and a set of VBV model parameters, and checks whether
the stream obeys the VBV model.

VBVChecker does not require a stats file. It automatically determines the frame sizes in the input
elementary stream (even when access unit delimiters [AUDs] are not present).

Your encoder will generally require special settings to generate VBV-compliant streams. If you run
VBVChecker on a stream not encoded for VBV compliancy you may find that a very large VBV buffer is
required, and/or that the channel bitrate must be specified as very large to obtain compliance, due to
the large variance of encoded bitrate in such streams (bitrate spikes).

For example, for the x264 encoder, VBV compliancy is enabled with these parameters:

--bitrate B --vbv-maxrate R --vbv-bufsize S

VBVChecker is also able to display the profile/level and HRD information from the stream (if present).
All HRD info encountered prior to the first picture is displayed.

-------------------------------

The available options are:

Buffer Size: This defines the size of the VBV buffer in bits. Generally, it is determined by hardware
in the decoder. For example, STMicroelectronics specifies a 50000000-bit buffer for HD decoding with
its reference software. You should determine this from the specifications of your decoder. Do not just
make it arbitrarily large! You have to know what your decoder can support.

If this is set to 0, then VBVChecker will search for a Buffer Size and Initial Fill
that can satisfy the specified Bit Rate setting. The Initial Fill is assumed to be 90%
of the Buffer Size. Note that when using this search mode, the Bit Rate must be specified,
i.e, you can't search for both Buffer Size and Bit Rate simultaneously. Finally, note that the
buffer size will be made at least as large as the largest frame encountered.

Initial Fill: This is the fullness in bits when decoding begins. Again, the decoder generally
determines this, but if it is unknown, a good estimate is 0.9 * Buffer Size.

Bit Rate: This is the bitrate that the channel can sustain.

If this is set to 0, then VBVChecker will
search for a Bit Rate that can satisfy the specified Buffer Size and Initial Fill settings. Note that
when using this search mode, the Buffer Size and Initial Fill must be specified, i.e, you can't search
for both Buffer Size and Bit Rate simultaneously.

Frame Rate: This is the frame rate of the video. If you enter 0, it will be determined automatically
if possible.

Strict Overflow Check: Strict adherence to the VBV model requires the encoder to stuff bits when the
encode is not generating enough bits to prevent VBV overflow. Generally, this is not done as it is a
waste to send stuffing bits over the channel and decoders are smart enough to deal with this without
any problems. If your encoder includes stuffing and you want to check for overflow, then check this
check box.

-------------------------------

If an overflow or underflow is encountered, an error will be reported via a message box. If the
check runs to completion without an error message box, then your stream satisfies the specified
VBV model.

Typically, this would be used to ensure that a stream does not exceed the constraints of the
decoder and channel, or to determine the bitrate for muxing to transport for broadcast. But it can
also be used to check whether an encoder is properly implementing its VBV processing.

-------------------------------

This program requires a license that can be obtained here:

http://neuron2.net/dgdecnv/dgdecnv.html

Scroll down and press the donate button. The program uses the same license file as
DGDecNV. If you have already licensed those programs, just place your license file
in the same directory as VBVChecker.

-------------------------------

Copyright (C) 2009 Donald A. Graft, All Rights Reserved

[neuron2]

If you specify 0 as the bitrate, then VBVChecker will find a bitrate that works with the specified buffer. See the revised link above.

[Dark Shikari]

If you specify 0 as the bitrate, then VBVChecker will find a bitrate that works with the specified buffer. See the revised link above.Could this work the other way, where if bufsize is zero and the max bitrate is specified, it can find the min bufsize that would work with that stream?

This wouldn't work for the CBR model, but it should work for VBR.

[neuron2]

Sure, I can add that. I thought it wasn't really needed because the buffer size is determined by the decoding hardware. But I'll make that for you.

BTW, the idea of searching was borrowed from the script you gave me, although I implement it differently. Implementing the VBV check itself is well-known. My value added is the ability to get the frame sizes directly from the ES, which makes it a good tool for *validating* encoder VBV handling, and of course making a GUI out of it.

Dark, I'll give you a free license if you don't have one. This tool doesn't need or use an NV card. Just PM me with the machine ID shown at the bottom left of the dialog.

In fact, all mods and developers can have a free license.

[Esurnir]

Sure, I can add that. I thought it wasn't really needed because the buffer size is determined by the decoding hardware. But I'll make that for you.

I read from some random guy working in an obscure company that to compensate some drawback of x264 vbv methods since their maximum bandwidth (DVB-S satelite link) was as flexible as a swiss clock they used a big amount of buffer size to be able to cope with some spike of bandwidth, buffer space that is not that difficult to find since they work with software based solution (what's next, using free software that help your business model?). So maybe those people working with software solution or user of software players out there might be more interested into measuring "how much buffer do they need on average" instead of measuring "what is the minimum bandwidth do I need with X hardware player".

[Dark Shikari]

I read from some random guy working in an obscure company that to compensate some drawback of x264 vbv methods since their maximum bandwidth (DVB-S satelite link) was as flexible as a swiss clockThat may have been me, as I work for an obscure company that uses satellite transport for DVB... :p

And that was a totally separate issue; there is an extra few megabytes of unused bandwidth on top of all the DVB streams so that if one encoder has a VBV violation, the extra space in the stream allows it to avoid actually dropping out.

This was moreso necessary before the original VBV changes by Gabriel and I, as back then x264 would very commonly underflow (now it basically never does, at least going by the logs generated by the encoder processes we've run overnight).

[Esurnir]

That may have been me, as I work for an obscure company that uses satellite transport for DVB... :p

And that was a totally separate issue; there is an extra few megabytes of unused bandwidth on top of all the DVB streams so that if one encoder has a VBV violation, the extra space in the stream allows it to avoid actually dropping out.

Thanks for the clarification :D.

P.S. I appologise to the swiss clock makers.

[neuron2]

Here's the version with buffer size search. It assumes 90% initial fill in this mode.

http://neuron2.net/dgavcdecnv/VBVChecker12.zip

Dark (hereafter to be called "random guy"), what transport multiplexer do you use?

BTW, my encode with stuffing finished oversized by 5Mbps (I asked for 10Mbps) and it encountered VBV underflows reported by x264 on the second pass. So I assume the stuffing patch is hosed. My re-encode without stuffing will be done in a few hours. I will report on it when it finishes. I did look at the start of it and saw some I-frame flicker in a really hard part (other parts were fine). I may have to set ipratio < 1.0 to see if it helps. I'm anal about it because the original VC1 shows no flicker. My next tool: I-frame flicker analyzer. :)

[Dark Shikari]

Dark, what transport multiplexer do you use?Avail uses a custom in-house muxer.

[neuron2]

Avail uses a custom in-house muxer. Can I have a copy of it? :)

I use Manzanita Muxer (purchased by STMicro). It costs $5000 but works very well. I just take the dongle home when needed.

[neuron2]

Would you like to see the capability to display NAL HRD parameters if present? Do any real streams include them? The idea would be to check whether the declared parameters are honored.

Interestingly, the Elecard Buffer Analyzer costs $350 and refuses to open streams without NAL HRD info.

[Dark Shikari]

Would you like to see the capability to display NAL HRD parameters if present? Do any real streams include them? The idea would be to check whether the declared parameters are honored.

Interestingly, the Elecard Buffer Analyzer costs $350 and refuses to open streams without NAL HRD info.This would be quite useful--could you make it do VCL HRD as well? This could help with regard to testing a potential "real" HRD patch for x264 to actually be committed, since NAL HRD technically requires knowledge of the transport layer.

This program is shaping up to be a nice replacement for what are really quite crappy (and rather overpriced) proprietary programs.

[neuron2]

This would be quite useful--could you make it do VCL HRD as well? I don't see why not. I'm fully parsing the AVC syntax already to get frame sizes.

This could help with regard to testing a potential "real" HRD patch for x264 to actually be committed, since NAL HRD technically requires knowledge of the transport layer. Can you provide some samples, please?

This program is shaping up to be a nice replacement for what are really quite crappy (and rather overpriced) proprietary programs. Thank you. Coming from you, it means a lot to me.

[Dark Shikari]

Can you provide some samples, please?You mean samples of streams with correct NAL HRD information? I would assume any Blu-ray disc could do the job.

[neuron2]

You mean samples of streams with correct NAL HRD information? I would assume any Blu-ray disc could do the job. Oh, torrents aren't good enough?

Just kidding guys!

I'll have to see if any of my BluRay disks have AVC video.

[Esurnir]

Oh, torrents aren't good enough?

Just kidding guys!

I'll have to see if any of my BluRay disks have AVC video.

torrents nearly never use --nal-hrd, for the simple reason that the maximum they would care about is dxva compatibility. The maximum concern for most is the time it'll take to encode.

[Gabriel_Bouvigne]

BTW, my encode with stuffing finished oversized by 5Mbps (I asked for 10Mbps) and it encountered VBV underflows reported by x264 on the second pass. So I assume the stuffing patch is hosed.
The "stuffing" (ie filler) patch has only been developed for single pass mode.
I understand the use case of single pass (live) CBR, but I'm having a hard time figuring out a use case for 2 passes CBR (as opposed to VBR).

[neuron2]

I know what CBR and VBR mean but I don't see what you mean by distinguishing between them for x264 doing a VBV encode. There's no option to set CBR versus VBR. Bitrate can always vary within the limits of the VBV. There is no "real CBR". So what are you referring to?

In any case, without 2 passes, I get poor results as documented in my other thread.

[Gabriel_Bouvigne]

What I meant is that because filler is only for cbr, and because I did not had a 2 passes CBR use case in mind, I only implemented filler for single pass mode (for which I had a real use case).

[Trahald]

This would be quite useful--could you make it do VCL HRD as well? This could help with regard to testing a potential "real" HRD patch for x264 to actually be committed, since NAL HRD technically requires knowledge of the transport layer.Yay! The current 'patch' needs to die already.

and thanks neuron for developing this great tool.

[neuron2]

@Gabriel

Sure, I understand that part. What I don't understand is what you mean by CBR versus VBR in the context of x264 VBV mode.

@all

I added a bar to show buffer fullness during checking. See the link in the first post.

[Gabriel_Bouvigne]

Just that my use cases for 2 passes are all targeted toward VBR, and VBR excludes filler, so that's why I did not implemented filler for 2 passes but only for single pass mode.

From a technical POV, there is indeed no difference between CBR and VBR for x264, as long as you are using VBV. CBR is indeed (VBV + max rate equal to target rate + filler). Remove any of those, and the stream becomes VBR.
Sorry if I've once again lost someone.

[neuron2]

Ah, OK. You just mean CBR = VBV compliance + filler = strict VBV compliance (no overflow). Yes, I agree that there is no apparent use case for it. But then why was the patch developed? Why would we need filler even for live single pass?

[Gabriel_Bouvigne]

It was developed for a dumb transport layer which was dropping signal if there is not enough data.
Often, if your encoding is not live, you can afford to buffer a lot of data on the receiver side (computer, set top), so the sender can push data faster than "real time" and the transport layer is never in the case of no data to transfer.

[neuron2]

A proper transport muxer adds fill packets to maintain a constant transport rate.

[nautilus7]

I'll have to see if any of my BluRay disks have AVC video.Did you find the samples you were looking for? I had sent you some for adding HD audio support to your tools: http://forum.doom9.org/showthread.php?p=1224125#post1224125

[neuron2]

Already got the HRD info dump working, but thanks anyway. I'll post the binary when I get home in a few hours.

[neuron2]

Version 1.4 adds the HRD info dump. Use the link in the first post.

[neuron2]

Version 1.5 adds profile/level display. Use the link in the first post.

[laserfan]

neuron2 or others here would probably know the answer: if HRD INFO is not present in an x264 encoding, is the only way to "add it" to re-encode with "--nal-hrd" option? I have only looked at a couple of .264s, one a BD original and another an x264, and both show the same thing, as if perhaps the below simply reflects a common characteristic of High profile, Level 4.1 encodings (as both are):

NAL HRD INFO:

cpb_cnt_minus1 = 0
bit_rate_scale = 3
cpb_size_scale = 3

bit_rate_value_minus1[0] = 78124 [40000000 bits/s]
cpb_size_value_minus1[0] = 234374 [30000000 bits]
cbr_flag[0] = 0

initial_cpb_removal_delay_length_minus1 = 23
cpb_removal_delay_length_minus1 = 23
dpb_output_delay_length_minus1 = 23
time_offset_length = 0

---END HRD INFO---

So I'm wondering if maybe this is only "header" info and can be added somehow to a 264, after-the-fact?

[neuron2]

Yes, it can be added. The SPS's must be rewritten. It's not a trivial thing but not too difficult.

[laserfan]

Thanks neuron2, I guess there is not *currently* an off-the-shelf utility to do this then...

I had read elsewhere here that "having HRD info" may be a factor in playability of homemade BD discs on Sony stb players. So I was wondering, if I ever got a Sony player, whether the BDs I'd made would work, as they're the only way I save TV shows I've recorded OTA. I haven't been religious about using --nal-hrd to this point. :o

[neuron2]

If you get the player and your HRD-less streams don't play, I'll hack something up to add HRD and we can test whether it has any effect.

[laserfan]

If you get the player and your HRD-less streams don't play, I'll hack something up to add HRD and we can test whether it has any effect.Thanks for the offer!!! Today you get a Gold Star, sir! :)

I just did an encoding with --vbv-maxrate 17500 and --vbv-bufsize 14500, so for grins took a look w/VBVChecker and I get a different result from my above post:
---HRD INFO---

NAL HRD INFO:

cpb_cnt_minus1 = 0
bit_rate_scale = 1
cpb_size_scale = 1

bit_rate_value_minus1[0] = 136718 [17500032 bits/s]
cpb_size_value_minus1[0] = 453124 [14500000 bits]
cbr_flag[0] = 0

initial_cpb_removal_delay_length_minus1 = 31
cpb_removal_delay_length_minus1 = 21
dpb_output_delay_length_minus1 = 21
time_offset_length = 24

---END HRD INFO---
Two of the numbers look familiar, and the rest are an "I dunno" for me, but at least now I know that HRD INFO is different for different encodings (duh).

[Selur]

@neuron2: Does the vbvchecker require a Nvidia graphic cart?

[Kurtnoise]

This tool doesn't need or use an NV card.

In fact, all mods and developers can have a free license.
pretty clear...:)

[Selur]

Thanks, totally overlooked that. :)

Cu Selur

[rack04]

I am having issues with a recent 1280x720 blu-ray encode playing on my standalone player. While troubleshooting I decided to check the NAL HRD info on two files. Here is the information obtained using VBVChecker 1.5:

Battlestar Galactica_S1D1

---HRD INFO---

NAL HRD INFO:

cpb_cnt_minus1 = 0
bit_rate_scale = 3
cpb_size_scale = 3

bit_rate_value_minus1[0] = 46874 [24000000 bits/s]
cpb_size_value_minus1[0] = 234374 [30000000 bits]
cbr_flag[0] = 0

initial_cpb_removal_delay_length_minus1 = 16
cpb_removal_delay_length_minus1 = 5
dpb_output_delay_length_minus1 = 3
time_offset_length = 24

---END HRD INFO---

Battlestar Galactica_S1D2

---HRD INFO---

NAL HRD INFO:

cpb_cnt_minus1 = 0
bit_rate_scale = 3
cpb_size_scale = 3

bit_rate_value_minus1[0] = 46874 [24000000 bits/s]
cpb_size_value_minus1[0] = 234374 [30000000 bits]
cbr_flag[0] = 0

initial_cpb_removal_delay_length_minus1 = 16
cpb_removal_delay_length_minus1 = 5
dpb_output_delay_length_minus1 = 4
time_offset_length = 24

---END HRD INFO---

I authored both using tsMuxeR and burned to a DVD+R. The Battlestar Galactica_S1D1 disc plays flawlessly on my PC using MPC-HC and standalone Panasonic BD-35. The Battlestar Galactica_S1D2 disc plays flawlessly on my PC but stalls on my standalone Panasonic BD-35. The only difference in the encodes is:

Battlestar Galactica_S1D1 was encoded with ref=5 using patched x264_hrd_pulldown.15_interlace.diff

Battlestar Galactica_S1D2 was encoded with ref=8 using patched x264_hrd_pulldown.15_interlace.fix.1198.diff

The only reason I posted in this thread is because when I analyzed the NAL HRD information I noticed that the playable file had dpb_output_delay_length_minus1 = 3 and the non playable file had dpb_output_delay_length_minus1 = 4. Could this in any way lead to playback problems or should I look more into the playback issues regarding ref=8? Thanks.

[neuron2]

Morre likely to be the refs.

[neuron2]

Fixed a bug that caused the profile and level boxes not to be written for some kinds of file open. Browse worked but drag-n-drop and direct typing in the edit box failed.

Also removed a line in the help file regarding 2-pass VBV that is no longer valid.

http://neuron2.net/dgavcdecnv/VBVChecker16.zip

[Biggiesized]

Have you thought of adding a preset drop down menu to check compliance for things like Blu-ray streams (primary and secondary)?

[Riki]

so I got a license and Im just trying to check the vbv on files encoded with sx264 CRF.

When I drag a video in I get a bunch of errors like you can see in the attachment, which I ignore and when I hit check the program crashes everytime.

Perhaps this is a problem with vista 64? I tried using xp I have in virtual PC but it seems to have a different hardware ID.

[neuron2]

It has to be an elementary stream. Demux it if needed.

What is the file? Can you post a sample of it? Use DGSplit to cut some from the beginning if it is too large.

[Riki]

oh right, yeah if I demux it it will work.

Well I just wanted to check the vbv average and peak settings on my files that will be used on a video streaming server. The numbers its giving me doesnt make any sense, I guess this was meant for hardware or something?

[neuron2]

The numbers make good sense, you just apparently don't know how to interpret them.

I just wanted to check the vbv average and peak settings on my files What do you mean by that, specifically? "vbv average and peak settings" has no clear accepted meaning in the context of a VBV model, so you have to tell me exactly what you are talking about.

If you are doing streaming, then you must know the transmission bitrate of your channel. So you enter that as bitrate. Then if you know the buffer size of your decoder you enter that too. Then the program will tell you if your buffer will ever undeflow with your file. If you do not know the buffer size, enter it as 0 and then the program will tell you the size of buffer that you need so as not to underflow.

[Riki]

Oh ok, that makes more sense. There is still a lot I need to learn.

What I do is use sx264 to encode and set the VBV settings to 200kbps maxbitrate, and 2000 kb buffer size. However Im using CRF so Im not sure if its overriding it for quality reasons or not, since you have to set a quality number too.

Then I use a program like bitrate viewer (http://www.softpedia.com/get/Multimedia/Video/Other-VIDEO-Tools/Bitrate-Viewer-winhoros.shtml) to check it. The problem is that program will say and average bitrate of 90kbps and a Peak of 1033 kbps, I was assuming x264 would set the peaks to 200 kbps so I dont know if the program is inaccurate or what.

So then I purchased vbvchecker hoping to verify my files are being encoded right but I cant seem to translate it I guess.

The goal for me is to have an encoded file with a dependable vbv so my streaming doesnt just start stuttering in the middle of playback because of a high peak (this concern may not even be warranted). What Im beginning to understand is with streaming filesize doesnt matter, its the bitrate, so how can I check my bitrate to make sure its what its supposed to be throughout the entire video.

Thanks for your time.

[Dark Shikari]

Oh ok, that makes more sense. There is still a lot I need to learn.

What I do is use sx264 to encode and set the VBV settings to 200kbps maxbitrate, and 2000 kb buffer size. However Im using CRF so Im not sure if its overriding it for quality reasons or notIt isn't.

[Riki]

Thats one place where its hard for me to understand. Im using the latest version of x264 and I have 2 encodes and the only difference is I change the CRF quality.

x264 --profile main --crf 30 --pass 1 --level 4.1 --ref 1 --keyint 120 --min-keyint 12 --scenecut 40 --bframes 6 --b-bias 0 --b-pyramid none --direct auto --b-adapt 2 --cplxblur 20 --qcomp 0.5 --mbtree --rc-lookahead 100 --qblur 0.5 --sync-lookahead 20 --qpmin 1 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --chroma-qp-offset 0 --partitions none --me dia --subme 2 --no-8x8dct --trellis 0 --aq-mode 2 --aq-strength 1 --threads auto --filter -1,-1
x264 --profile main --pass 2 --bitrate 94 --level 4.1 --ref 5 --keyint 120 --min-keyint 12 --scenecut 40 --bframes 6 --b-bias 0 --b-pyramid none --direct auto --b-adapt 2 --cplxblur 20 --qcomp 0.5 --mbtree --rc-lookahead 100 --qblur 0.5 --sync-lookahead 20 --qpmin 1 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --chroma-qp-offset 0 --partitions i4x4,p4x4,p8x8,b8x8 --me esa --merange 16 --subme 10 --no-8x8dct --trellis 2 --psy-rd 0.8:0 --no-fast-pskip --aq-mode 2 --aq-strength 1 --vbv-maxrate 200 --vbv-bufsize 2000 --threads auto --filter -1,-1

In the end I get a 5mb file and a 1.8mb file, much different quality visually. In bitrate viewer the numbers are also way different for average and peak bitrate.

What Im hoping to figure out is how to encode for a streaming server and have the quality as high as possible, with a low as possible bitrate, in this case 200.

[nm]

Thats one place where its hard for me to understand. Im using the latest version of x264 and I have 2 encodes and the only difference is I change the CRF quality.

x264 --profile main --crf 30 --pass 1 --level 4.1 --ref 1 --keyint 120 --min-keyint 12 --scenecut 40 --bframes 6 --b-bias 0 --b-pyramid none --direct auto --b-adapt 2
--cplxblur 20 --qcomp 0.5 --mbtree --rc-lookahead 100 --qblur 0.5 --sync-lookahead 20 --qpmin 1 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --chroma-qp-offset 0
--partitions none --me dia --subme 2 --no-8x8dct --trellis 0 --aq-mode 2 --aq-strength 1 --threads auto --filter -1,-1

I don't see VBV parameters in this command line. Also, your GUI is using very low settings -- looks like fast first pass to me.

I'd recommend using x264 from the command line so that you get to control things yourself. But use the preset system for quality settings.

[Riki]

I don't see VBV parameters in this command line. Also, your GUI is using very low settings -- looks like fast first pass to me.

I'd recommend using x264 from the command line so that you get to control things yourself.

Should the VBV be in the first pass? Its at the end of the 2nd pass, you just have to scroll the forum to the right very far.

[Riki]

As for the qualiy, its actually pretty good, the video content is a screencapture video so its not anything like a movie.

[nm]

Ok, you lost me there completely. Can you explain again what you tested and post the command lines for both cases?

[Riki]

In my GUI (sx264) Im setting the VBV settings to 200kbps maxbitrate, and 2000 kb buffer size. I'm also encoding using CRF. (all of this you can see in the code I posted). So in my test ALL I changed was CRF from 30 to 18, I left the vbv, and everything else the exact same. The end result is what you would expect from a CRF of 18 and 30, the 18 is much better looking, also the filesize is much different.

So no surprises there I guess, but when I use a program like bitrate viewer to check it the bitrates are also way different, and I was expecting x264 to cap it at 200. This is important for a smooth transmission through the streaming server.

One option is bitrate viewer is just confusing me, and if so, what program can i use to get an accurate check?

[LoRd_MuldeR]

One option is bitrate viewer is just confusing me, and if so, what program can i use to get an accurate check?

VBVChecker, configured with the proper VBV parameters ???

Note: You cannot check VBV compliance without providing the required VBV parameters. Use it as Neuron2 told you here (http://forum.doom9.org/showpost.php?p=1336974&postcount=45) ;)

Elecard Buffer Analyzer (http://www.elecard.com/products/products-pc/professional/buffer-analyzer/) would be another option...

[nm]

In my GUI (sx264) Im setting the VBV settings to 200kbps maxbitrate, and 2000 kb buffer size. I'm also encoding using CRF. (all of this you can see in the code I posted). So in my test ALL I changed was CRF from 30 to 18, I left the vbv, and everything else the exact same. The end result is what you would expect from a CRF of 18 and 30, the 18 is much better looking, also the filesize is much different.
You are not using plain 1-pass CRF but 2-pass VBR with a fast CRF first pass. Apparently sx264 is using the bitrate from the first CRF pass when encoding the second pass. I think VBV should be set in the first pass too to make things work properly.

When encoding with CRF 18, check the --bitrate value that sx264 uses for the second pass.

Also try using plain 1-pass CRF to see if it works differently.

Edit: I think this 2-pass CRF mode was implemented in sx264 because a while ago x264's 1-pass VBV didn't work that well. Now 1-pass VBV is fine, so you don't need to run two passes.

[Riki]

Ok, sorry. I did try it though but its not so easy for me to wrap my brain around, I was hoping some people could re-explain it in a different way so hopefully the lightbulb will come on.

So I use the VBVChecker program, and I put the raw stream of the 2 videos Ive been mentioning. I set the buffer size to 0 for autodetect and I set the bit Rate to 200. I choose 200 cause that is what I se for the Max bitrate in the VBV settings.

So as you can see Im getting the numbers in the image, what do these numbers mean?

[Riki]

Also try using plain 1-pass CRF to see if it works differently.

Edit: I think this 2-pass CRF mode was implemented in sx264 because a while ago x264's 1-pass VBV didn't work that well. Now 1-pass VBV is fine, so you don't need to run two passes.

Very interesting!! I will test this soon. 1 pass is also much faster to encode :)

[Riki]

This is what it spit out. So I was wondering if someone could help me interpret the numbers.

[nm]

You should probably upload those images to some external site and link them to your posts. Attachments take a while to get approved so we can't see your images yet.

[Riki]

oh I didnt realize that.

1 pass CRF 30
http://img5.imageshack.us/img5/7456/1passcrf30x.jpg

2 pass CRF 30
http://img194.imageshack.us/img194/7451/crf30.jpg

2 pass CRF 18
http://img200.imageshack.us/img200/7614/crf18.jpg

[neuron2]

Bitrate is in bits/sec so you should enter 200000, not 200.

[Riki]

oh wow, yeah that makes a huge difference.

When I put 200000 for the bitrate it auto detects that the buffer is 800000.

Like I mentioned before in x264 I set the buffer to be 2000 which would be 2000000 bits I guess. So When I put 2000000 in the Buffer size but Im not sure what to put in the initial fill, but I went ahead and put 1000000. When I run Check no errors come up.

So Im trying to understand the logic and it sounds like there wont be any underflow issues at a 200kbit/s stream with a 2000 kbit buffer. Is that correct?

Also when I drag the file into mediainfo it says the stream is 90 kbps, but I guess thats just the average or something.

[neuron2]

When I put 200000 for the bitrate it auto detects that the buffer is 800000.

Like I mentioned before in x264 I set the buffer to be 2000 which would be 2000000 bits I guess. So for the file you checked you only needed a buffer of 800000.

But be aware that the buffer size is determined by hardware. You should find out what your player device has and use that value.

So When I put 2000000 in the Buffer size but Im not sure what to put in the initial fill, but I went ahead and put 1000000. When I run Check no errors come up. If you haven't set it otherwise with x264, then use 90% of the full buffer size.

So Im trying to understand the logic and it sounds like there wont be any underflow issues at a 200kbit/s stream with a 2000 kbit buffer. Is that correct? If by 200kbit/s stream you mean 200kbit/s transport rate, then yes.

VBVChecker here is just verifying that x264 actually produced correct VBV compliance to the model you defined.

Also when I drag the file into mediainfo it says the stream is 90 kbps, but I guess thats just the average or something. Or something.

[Esurnir]

I recently tried to encode a stream using the "1 frame vbv" settings :
x264 ––tune zerolatency ––intra-refresh ––preset veryfast ––fps 30 ––vbv-maxrate 700 ––slice-max-size 1500 ––vbv-bufsize 24 –o persona.264 persona.y4m
resulting in this :
http://www.mediafire.com/?nc145iqmzhk
I tried to check the vbv using vbvchecker and it tell me that I got an undersized buffer (frame size 25864 bit, largest in the stream then become 28720 bits).

Now after Dark Shikari check it in elecard streameye the largest frame become frame 2462 which is 23424 bit large (see http://doom10.org/index.php?topic=181.0)

Is vbvchecker doing something wrong does elecard overlook something or is x264 vbv model broken ?

[neuron2]

Let's talk about what happens when you specify a buffer of 24000 and initial fill 24000.

For frame 90 (zero-based), I see three NALUs with non-IDR slices having these sizes:

1498
841
894

Interestingly, the size of the first is just below the limit you set for slice-max-size. The total is then 3233 * 8 = 25864 bits. That is bigger than the buffer size.

Please ask Dark_Shikari if he agrees with these NALU sizes and if so ask why they are exceeding the buffer size.

[Dark Shikari]

Let's talk about what happens when you specify a buffer of 24000 and initial fill 24000.

For frame 90 (zero-based), I see three NALUs with non-IDR slices having these sizes:

1498
841
894

Interestingly, the size of the first is just below the limit you set for slice-max-size. The total is then 3233 * 8 = 25864 bits. That is bigger than the buffer size.

Please ask Dark_Shikari if he agrees with these NALU sizes and if so ask why they are exceeding the buffer size.CMA1820 says that there are two NAL units in frame 90 (zero-based), of size 7160 bits and 6992 bits respectively. This does not exceed 24000.

[neuron2]

Open the uploaded file in a hex editor. The first slice of frame 90 is at offset 0x23ebb. Search forward for the next start code and you find the next slice at 0x24496. The difference is 1499 bytes. (VBVChecker said 1498 because it should have included the trailing zero byte. That will be fixed.)

In any case, that looks like evidence that the first slice is in fact 1499 bytes, consistent with the slice-max-size setting.

[Dark Shikari]

Open the uploaded file in a hex editor. The first slice of frame 90 is at offset 0x23ebb. Search forward for the next start code and you find the next slice at 0x24496. The difference is 1499 bytes. (VBVChecker said 1498 because it should have included the trailing zero byte. That will be fixed.)

In any case, that looks like evidence that the first slice is in fact 1499 bytes, consistent with the slice-max-size setting.The slice of size 1499 bytes (11992 bits) is the second slice in frame 89.

Frame 89: 3664, 11992, 6736
Frame 90: 7160, 6992

[neuron2]

I'll double check my frame boundary detection and report back.

[b66pak]

and what is the result of the double checking?
_

[neuron2]

and what is the result of the double checking? I have to get DGDecNV build 2006 out first. You know what they say, patience is a virtue.

[neuron2]

OK, I have results for you.

The frame boundary checking was correct but I was incorrectly accumulating the NALU sizes delimited by those boundaries. I have it fixed and will release it this evening.

With the fixed version for the sample stream, buffer size 24000 passes, while 23000 fails.

If you enter 0 as bufsize for autodetect, it ends up at 24000.

[neuron2]

* Fix problem with frame size determination.

* Fix BFF field-structured stream handling.

* Improve granularity for automatic buffer size determination for small buffers.

http://neuron2.net/dgdecnv/dgdecnv.html (scroll to bottom)

[Blue_MiSfit]

Nice work, neuron2!

Good VBV checking tools are hard to come by. This is an extremely useful tool to me! Worth fifteen hundred pennies for sure ;)

~MiSfit

[PowerGamer]

Buffer Size: This defines the size of the VBV buffer in bits. Generally, it is determined by hardware
in the decoder. For example, STMicroelectronics specifies a 50000000-bit buffer for HD decoding with
its reference software. You should determine this from the specifications of your decoder.

Is this value known for ATI/NVIDIA GPUs?

[Esurnir]

I'm not sure vbv can apply to software player that much, it mostly depend on the software you play it, say Windows Media Player and Flash will tend to "wait" on an "offline" video until it feel it got enough content to play the video seemlessly (hoping the bitrate of delivery doesn't vary too much) so if you got a slow connection the video will be loaded at something like 60% of the video before it start playing (that's a -huge- buffer). Some other player may download for live streaming a value in a certain ammount of "Second" of buffering, like 1 second, 5 second, etc.

Correct me if I say something stupid, but a "bitrate check" is more or less useless for a software player which got an "almost unlimited" buffer, What would be more interesting would be to determine the ammount of buffer necessary to play your video for a certain bitrate.

There may be a "maximum buffer" available on the nvidia card, but that buffer isn't entirely relevant (save maybe to determine what is the maximum frame/slice size your player can play), since your software will hold the bitstream "in ram/on harddrive file cache" and will most likely only feed the bitstream that need to be played imediately.

[cali_axela]

What's the consensus on highest reasonable bufsize to use when encoding a baseline h264 for streaming to older Apple devices (as well as the PSP etc)? 4000k? 8000k? More, less? I assume this buffer needs to be small enough to store in the video player application's active memory, and can't be ridiculously large due to high latency. I've not been able to find any concrete data on any upper limits (or reasonable/reliable best practice size).

A few tests of a 1.5mbps ABR file with a 2mbit bufsize/maxrate seems to work OK, but we would be alright with slightly more initial latency as a tradeoff for more reliable streaming on dodgy 3G connections. Is a 3mbit bufsize reasonable for a 1.5mbit ABR video? 4mbit? 8? What's the common range used in practice for this purpose?

[Blue_MiSfit]

I usually think in terms of how many seconds of buffer. One second is a bare minimum for most sane purposes. Not to say that it's not common practice to do much less than this in the "professional" (lol) world.

Then again.. that's just how I think about it. You ask a very valid question. The bottom line is that you're balancing buffer time / delay until the stream starts playing / memory required versus image quality... basically.

Derek

[cali_axela]

So if I wanted two seconds of buffer, I would set bufsize to 2x maxrate? Or 2x of ABR?

[Blue_MiSfit]

I'm not 100% sure, but I do believe that 2x maxrate is the proper value.

--vbv-init does change things. The default value of .9 implies that the initial vbv occupancy is 90% of the total bufsize.

I'm not a VBV expert, but I do know that even vanilla CBR encoding looks pretty good in most cases (i.e. bitrate==maxrate==bufsize)

Derek

[kieranrk]

So if I wanted two seconds of buffer, I would set bufsize to 2x maxrate? Or 2x of ABR?

2x maxrate.