1pass RC in x264

[chen]

Currently, i am working on the analysis and improvement of the rate control algorithm of x264, but i have no idea about the algorithm in use. Could anyone tell me what the algorithm is and where i could get something about it to read?

[Sirber]

here it is: blurCplx^(1-qComp)

[Backflip]

Also - http://forum.doom9.org/showpost.php?p=643669&postcount=13

[akupenguin]

http://students.washington.edu/lorenm/src/x264/ratecontrol.txt

[chen]

which options of the x264.exe should i choose if i wanted a ratecontrol at 1pass_abr mode? And what about the CQP ( constant QP ) mode?

[bond]

which options of the x264.exe should i choose if i wanted a ratecontrol at 1pass_abr mode?-B

And what about the CQP ( constant QP ) mode? -q

[chen]

unfortunately, i could not find the part corresponding to this mode in the source code. Is it in that my x264 version released at Aug. 01 out of date?

[bond]

unfortunately, i could not find the part corresponding to this mode in the source code. Is it in that my x264 version released at Aug. 01 out of date?for getting a cbr encode you need to modify the vbv settings somehow, but i dunno exactly how this can be done

[chen]

for getting a cbr encode you need to modify the vbv settings somehow, but i dunno exactly how this can be done

But would you mind explain what vbv stands for? i could not find this key word in the mateirials about H.264 i am reading. And as for the modification of the vbv settings, did you mean that we only need to modify the settings when running the x264.exe instead of modifying the source code?

[bond]

But would you mind explain what vbv stands for? i could not find this key word in the mateirials about H.264 i am reading. And as for the modification of the vbv settings, did you mean that we only need to modify the settings when running the x264.exe instead of modifying the source code?vbv stands for Video Buffering Verifier and defines for example the bitrate range you can use when encoding (to avoid too big bitrates)

you can modify vbv via the commandline, no need to go via the source

[DarkZell666]

Hi chen!

All settings available for encoding with x264 (well almost) are available via the commandline switches ... I don't even understand why you wanted to change the sourcecode in the first place ;)

Maybe you should go for x264gui.exe which is included in the latest builds of x264 (http://x264.nl), which is the gui to x264.exe (pretty obvious i think), this would avoid you taking hours figuring out what all those switches mean.

Actually, the "ABR" setting you are looking for is available in x264gui.exe (first line of the main interface), and many of the settings you seem to be looking for are available in the "options" page.

Tips for you (using x264gui.exe) :

- Chose the Average bitrate mode (that is definitely what you were looking for)

- For CQP, set "Bitrate variability" to 100 in the options tab.
Then you can adjust the MinQP and MaxQP both to the QP you wanted (the bitrate specified in the main page will then be "ignored", or rather "overrided").

- For real plain CBR, set "bitrate variability" to 0 in the options tab, and set MinQP and MaxQP to the biggest range possible (ex: 0 - 100+), to avoid disturbing the bitrate if some frames require a very big QP to fit in the constant framesize imposed by the bitrate setting.

Normally this will achieve what you wanted (unless I have got something wrong ... ;)). You can of course test many other combinaisons to get a rather CBR-ish ABR encode, or a rather CQP-ish ABR encode.

Also don't forget to read this topic : http://forum.doom9.org/showthread.php?t=96059 which deals with some other options used by x264 (and other h264-based codecs).

Cheers and have a good time testing ;)

(Edit : this was my first post here, I hope it will be usefull !, and it definitely won't be the last ;))

[chen]

- For real plain CBR, set "bitrate variability" to 0 in the options tab, and ......
This is exactly what i should have thought of.

[chen]

http://students.washington.edu/lorenm/src/x264/ratecontrol.txt
The quoted txt above provides quite a good summary of the ratecontrol algorithm of x264, but this is still not enough for me to go into the datails, because there are hundreds of variables in the source code and complex relations between them. Does anybody have a detailed description of the algorithm? In fact, i have contacted the code writer, but no feedback yet.

[bond]

In fact, i have contacted the code writer, but no feedback yet.akupenguin is the code writer ;)

[chen]

The concept CBR is kinda ambiguous to my way of thinking. Does it mean that the # of bits we allocate to each frame is constant, or that the average bitrate of a certain number of frames -- specifically GOP? What i would like to know is how we explain this "constant" here.

[akupenguin]

x264 supports both modes:

If you set qcomp=0 (i.e. "bitrate variability"), then it tries to make each frame use exactly the same number of bits. It's not quite that precise, but that's its goal.

If you set vbv-maxrate and vbv-bufsize to some nonzero values, then it restricts the average bitrate of a certain number of frames (the number being howevermany can fit in your specified vbv-bufsize (which is specified in kbit)). If vbv-maxrate==bitrate, then it's CBR; otherwise it just restricts the peak bitrate.

You can mix any value of qcomp with the vbv stuff, in which case qcomp controls the local allocation of bits within the averaging window. (As opposed to the non-vbv case, where qcomp distributes bits over the whole movie.)

I think the name "VBV" was taken from MPEG-2 or the like. The H.264 standard calls it "HRD". Theoretically, vbv-maxrate is the rate at which the decoder can read from the medium (DVD, network, etc) and vbv-bufsize is the amount of memory it has available to pre-buffer the incoming stream.

[chen]

static inline double qscale2qp(double qscale)
{
return 12.0 + 6.0 * log(qscale/0.85) / log(2.0);
}

above is the function that converts qscale to qp, and before this, another function rate_estimate_qscale is focusing on the calculation of qscale. I am a little bit confused by this "qscale", because normally what we are doing is to calculate the bitrate R before converting R into qp. So, what is qscale then? And how is it related to R? By the way, is the return value of the above function exactly qp, i.e., the quantisation paramter which holds a value between 0 and 51?

[chen]

if( i_force_qp )
{ rc->qpa = rc->qp = i_force_qp - 1;}
else if( rc->b_abr )
{ ...... }
else if( rc->b_2pass )
{ ...... }
else /* CQP */
{ int q;
if( i_slice_type == SLICE_TYPE_B && h->fdec->b_kept_as_ref )
q = ( rc->qp_constant[ SLICE_TYPE_B ] + rc->qp_constant[
SLICE_TYPE_P ] ) / 2;
else
q = rc->qp_constant[ i_slice_type ];
rc->qpa = rc->qp = q; }

What I cannot understand is the first and the last case. In the first case, we can see that the QP is set to a constant i_force_qp - 1, and the frame type makes no difference. But why is the last case regarded as the CQP mode, where QPs vary according to frame type?

[akupenguin]

qscale is related to QP by, well, that function. Originally, 1st pass complexity was directly divided by qscale to result in the estimated bits for a given frame; I have since tweaked the bit predictor so it's no longer quite linear.
qscale2qp returns a float, but yes, when you round it to an integer and clip to the range [qpmin,qpmax] (default: [10,51]) you get exactly QP.

----

The first case is for if the application wants to override x264's ratecontrol. I don't know of any applications that do so, though I think Måns Rullgård had a use for it. Note that i_force_qp is not a constant: it can be passed into x264_encoder_encode() independently for each frame.

CQP: they are constant over time. That's the same as in XviD's CQP mode.

[chen]

The first case is for if the application wants to override x264's ratecontrol. Note that i_force_qp is not a constant: it can be passed into x264_encoder_encode() independently for each frame.

Do you mean that we can specify a different qp for every single frame? But i can figure out how to set the parameters of x264.exe for this case.

[chen]

static inline double qscale2qp(double qscale)
{
return 12.0 + 6.0 * log(qscale/0.85) / log(2.0);
}
If qp increases by 1, qstep increases by 12.5%, and so does qscale.

[chen]

@akupenguin

"Overflow compensation is the same as in 2pass. By tuning the strength of compensation, you can get anywhere from near the quality of 2pass (but unpredictable size, like +- 10%) to reasonably strict filesize but lower quality."

Would you mind explaining to me the above sentence i quoted from the text you gave me? If not, please follow the routine below?

1) What does "anywhere from near the quality of 2pass" mean? Does it mean that 1pass ABR with an infinite rate tolerance could have almost the same quality as 2pass? If so, why do we bother to use 2pass?

2) What is "(but unpredictable size, like +- 10%)"? The size of what?

3) And "to reasonably strict filesize but lower quality", what is filesize? The size of which file?

[akupenguin]

Do you mean that we can specify a different qp for every single frame? But i can figure out how to set the parameters of x264.exe for this case. I don't know of any applications that do so i.e. you can't do it with x264.exe or any other program. Just the library's API supports it.


Overflow compensation is the same as in 2pass. By tuning the strength of compensation, you can get anywhere from near the quality of 2pass (but unpredictable size, like +- 10%) to reasonably strict filesize but lower quality.
Example: you want to encode a movie in 700 MB.

2pass: will get exactly 700 MB, and maximize quality.

1pass ABR, ratetol=inf: will get somewhere between, say, 600 and 800 MB. Whatever filesize it happens to produce, the quality will be almost as good as a 2pass encode of that same size.

1pass ABR, ratetol=1 (or the like): will get very close to 700 MB (e.g. 699-701), but will have lower quality than either of the above.

[chen]

"Ratetol" is defined as the allowed variance of the average bitrate. But what is the way of calculating this variance? Is this variance simply the square of the difference between the target bitrate and the actual averaged bitrate? Or do we have to calculate the local bitrate of every single picture and then calculate the variance of all these samples? Moreover, what is the unit of this "ratetol"? Is it the real value of the variance or just a percentage?

To make things more clear, let's take the following example:

bitrate: 1,5 Mbits/sec
ratetol: 1

what is the possible range of the actual averaged bitrate? And what if ratetol is set to 2?

[akupenguin]

"Variance" describes ratetol's effect, not its algorithm. I do not calculate variance at all. When the encode finishes, it will have produced some average bitrate. It will not be exactly what you requested, but rather distributed randomly near your requested bitrate. "Variance" refers to that distribution (as a fraction of the requested bitrate).
ratetol is not in any particular units. I tried to make it that a given value of ratetol produced the same %error in filesize independent of movie length, but of course longer movies will tend to average out closer to the target even without enforcing anything.

In practice (see some examples (http://forum.doom9.org/showthread.php?p=653279#post653279)), it looks like ratetol=1 is about +/- .5% bitrate, so ratetol=2 would be about +/- 1%. (This does not, however, scale to very large values of ratetol: it converges asymptotically to some maximum variance depending on the content and other settings.) Also, those are not hard limits; I could construct a pathological video that would throw off 1pass ABR much farther from the requested rate.

bitrate: 1,5 Mbits/sec
ratetol: 1
length: at least several minutes
output: probably between 1,49 and 1,51 Mbits/sec

[chen]

For the CBR case, "Scaling factor is based on a local average (dependent on VBV buffer size) instead of all past frames", and this is done by setting the variable cbr_decay to a value smaller than 1, which gradually decays the effect of past frames. But the source code indicates, cbr_decay is set to be smaller than 1, if only vbv_maxrate / bitrate is smaller than 1.5, which is a condition independent from the value of "qcomp" and "ratetol". This means, whether the scaling factor is based on local average or not has nothing to do with CBR, but totally depends on the ratio of vbv_maxrate and bitrate.

Now that a factor based on a local average is not exclusively used for CBR, could anybody please explain how it makes a difference to use local average rather than the average of all past frames? Moreover, why is it related to VBV?

[akupenguin]

CBR is the case where vbv_maxrate equals bitrate.
OK, so I enable local averaging gradually, not on strict equality. If you request almost CBR (i.e. peak bitrate only a little above average bitrate) then it still doesn't try to perform full long-term bitrate variance.

Note: I said before that you could also get something approaching CBR by setting vqcomp=0. While true, that's very low quality, and still isn't really constant because it doesn't strictly enforce anything. Don't do it. Though reducing vqcomp somewhat is OK in conjunction with VBV-based CBR.

[chen]

CBR is the case where vbv_maxrate equals bitrate.
OK, so I enable local averaging gradually, not on strict equality. If you request almost CBR (i.e. peak bitrate only a little above average bitrate) then it still doesn't try to perform full long-term bitrate averaging.

Now i get it. If maxrate = bitrate, it's no doubt CBR; in the range of maxrate / bitrate < 1.5, the value of cbr_decay is gradually approaching 1, thus the averaging window is accordingly lengthened; beyond that range, the cbr_decay is always set to 1, thus a full long-term bitrate averaging substitutes the local averaging.

But i still can't understand why a local averaging rather than a long-term one can better achieve a constant bitrate. Will you please give a short explanation?

[chen]

@akupenguin

In 1pass_ABR, the qp of an I-frame is predicted by a weighted averaging of the qp of all the past frames. When updating the weighted sum of qps, the qp of an I-frame is multiplied by ip_factor so that it is normalised to p_qp, which can be seen from the following piece of code.

if I_slice
accum_p_qp += qpa * ip_factor; (1)
else
accum_p_qp += qpa;

However, the rest of the code indicates that ip_factor and also pb_factor are used to multiply qscale rather than to directly multiply qp. Why don't we modify (1) in this way:

accum_p_qp += qscale2qp( qp2qscale( qpa ) * ip_factor );

Moreover, why don't we also normalize the qp of a B-frame to p_qp by taking pb_factor into accout?

[chen]

Is it in that B-frames are less important and cost fewer bits, so that it doesn't worth the trouble to apply VBV constraints to them?

[akupenguin]

cplxr_sum is the average of (bits * qscale / rceq)
It is used for computing: next qscale = rceq * cplxr_sum * (factor based on requested bitrate)
The effect of the above formula is to make sure that the bitrate of the area that cplxr_sum was averaged over is near the requested bitrate (if not, the next frames get higher or lower qscale to compensate), while individual frames get qscales proportional to rceq. If you average over the whole video, then it doesn't start adjusting qscales until the total bitrate of the whole video differs noticeably from the requested rate. If you use a local average, then it adjusts qscales much sooner, which is what you need for CBR since the tolerance in bit count is small and constant rather than proportional to total video size.

In 1pass_ABR, the qp of an I-frame is predicted by a weighted averaging of the qp of all the past frames. Exponential weighted average (time constant = 20 frames), so it's really a local average like scaling factor in CBR.

However, the rest of the code indicates that ip_factor and also pb_factor are used to multiply qscale rather than to directly multiply qp.
Moreover, why don't we also normalize the qp of a B-frame to p_qp by taking pb_factor into accout? Both are bugs. (That code used to be in a place where B-frames never reached.)

It's true that I don't apply VBV directly to B-frames (partly because of their lower bit size, and partly because that size is hard to predict if you let their qp vary independently of the adjacent P-frames), but that's irrelevent to the accum_p_qp code.

[chen]

cplxr_sum is the average of (bits * qscale / rceq)
It is used for computing: next qscale = rceq * cplxr_sum * (factor based on requested bitrate)

But what i extracted from the source code is : cplxr_sum is a local sum of (bits * qscale)
and the equation for computing is : next qscale = rceq * cplxr_sum / wanted_bits_window,
where "wanted_bits_window" is a local sum of target bits per frame.

[akupenguin]

But what i extracted from the source code is : cplxr_sum is a local sum of (bits * qscale)
573: rc->cplxr_sum += bits * qp2qscale(rc->qpa) / rc->last_rceq;

next qscale = rceq * cplxr_sum / wanted_bits_window,
where "wanted_bits_window" is a local sum of target bits per frame.
That's what I said: a "factor based on requested bitrate".

[chen]

next qscale = rceq * cplxr_sum * (factor based on requested bitrate)

rceq approximately represents the complexity of the current frame while cplxr_sum is the average complexity of recent frames. It's obvious that the larger the averaged complexity, the larger the qscale, so that the target bitrate is met. But what i can't understand is why rceq is also positively proportional to qscale. For example, if a frame has a larger complexity than its predecessors, rceq is larger, so qscale is larger, than this frame quantised more coarsely. But normally frames of higher complexity deserve more bits, don't they?

[akupenguin]

You'll notice that rceq varies less than linearly with estimated bit size (unless qcomp=0). Higher complexity frames do get more bits, just not as much more as they would get in constant QP mode. (See the original post on my theory of ratecontrol.)

[chen]

@akupenguin

1) In the function vbv_update( ), there is warning message of underflow but none of overflow, as if overflow were never gonna happen. Is it for sure?

2) In the function clip_qscale( ), where VBV constraints are applied, there is a line of code " q = max( q0, q ) ", which indicates that qscale can only become larger under the VBV constraints, which is equivalent to the decrease of bits. This implies again that there is never overflow problem. Why overflow is considered as a trivial thing in x264?

[akupenguin]

VBV underflow is really bad: it means that the codec used more bits than the network can transport (or whatever your streaming limitation is), so there is a good chance of lag / dropped frames / other problems.
VBV overflow is not an error: we used fewer bits than the max available, so quality isn't quite as good as it could have been if ratecontrol had been more precise. This usually happens only in low complexity spots, which are the highest quality anyway. While it would be nice to avoid overflows, it only improves overall quality if you can do it by improving bitrate prediction; clipping to a lower QP after filling the VBV buffer would just make the highest quality frames a little better.

[chen]

I totally agree with yon on the underflow, but for the case of overflow, i have some different opinions.

When it comes to overflow, I don't understand why you keep talking about the quality. Quality is already maximized before VBV is taken into account and VBV constraints simply aim at avoiding overflow and underflow of the decoder buffer. Specifically speaking, to avoid overflow, we have to set a lowerbound of bits. One paper states that this lowerbound is vbv_maxrate / fps, which is explained like this: suppose the worst case when the buffer is full, and we read out the bits of a frame for display. During this 1/fps time length, the number of the bits the decoder receives from the channel is vbv_maxrate / fps. Thus, the frame removed must have no fewer bits than this lowerbound in case an overflow arised.

Simply speaking, if VBV overflow constraint is ignored, we won't lose much quality, but the decoder buffer may suffer an overflow.

[akupenguin]

No, VBV-based ratecontrol aims at maximizing quality while not using too many bits. Calling too few bits a "buffer overflow" is misleading. Nothing at all happens in that case, it just means that the decoder stops prefetching more data (or in multicast, the server sends new frames at framerate instead of as fast as bandwidth allows). No format since VCD has required you to stuff bits just to use up space.

[chen]

No format since VCD has required you to stuff bits just to use up space.
I am afraid i can't quite follow you. Would you mind giving me a more detailed explanation? I guess it could contribute a lot to a better understanding in VBV .

[chen]

In the ratecontrol process in x264, VBV constraints is not the last step. After that, we still have clip( ), and for 2pass, overflow compensation is yet to follow. In this case, how can we make sure that VBV conditions are still met after this clip or overflow compensation? As far as i know, a previous version of x264 did result in a decoder buffer underflow. As for the current version, does anybody have such an experience?

[akupenguin]

You don't. Even if VBV were the last step, it wouldn't guarantee no buffer underflows: it limits based on predicted frame size, and the prediction might be wrong.

[chen]

You don't. Even if VBV were the last step, it wouldn't guarantee no buffer underflows: it limits based on predicted frame size, and the prediction might be wrong.
My current project requires so small a decoder buffer, at most a 2-second buffer with 1,5Mbit/s, that is, 3000kbits in total, that it is even harder to avoid underflow. But i think there is something that we could do to lower the probability of an underflow, and will you please give me some suggestions?

[akupenguin]

By default, qpmax=51, so clipping after vbv will not affect probability of underflow.

Yes, it is possible to improve prediction. We could keep track of the distribution of complexity within a frame, and re-evaluate the ratecontrol decisions partway through.
Note that in my limited experience (I don't actually have any use for CBR, I only added it on request because the current method is simple), almost all underflows occur on the misprediction of I-frames.

[chen]

......, almost all underflows occur on the misprediction of I-frames.
In this case, do you think it a good idea to simply tune the --ipratio so that I-frames cost fewer bits than before?

[chen]

i once indicated that the VBV constraints in x264 ratecontrol ignore overflows, and now my simulation of decoder buffer indeed results in overflows, and will anybody give me some suggestions about how to improve the encoder so that overflow do not occur anymore.

Details of the simulation:

1) Test Sequences: the first 1500 frames of the second half of Matrix3;

2) Encoding parameters: x264.exe -B 1500 --vbv-maxrate 1500 --vbv-bufsize 1500 --fps 25 --bframes 2 --no-b-adapt -r 4 -v -o f:/bitstreamfortest/4sec.264 f:/YUV_Sequences/4sec.yuv 720x576

3) Encoding result: bitrate = 1470.13 kb/s

4) Decoder buffer specifications: bitrate = 1470.13 kb/s; bufsize = 1500; init_buffer_occupancy = 0.9 ( equal to the encoding parameter --vbv-init );

5) Model of simulation: read one frame out of decoder buffer, check underflow; stuff bitrate / fps into decoder buffer; check overflow. Repeat this two operations frame after frame.

Result of the simulation: No underflow, and buffer is at the least 10% filled; Overflow occurs, and the buffer was overflowed altogether a quarter of the whole decoding duration.

[akupenguin]

In this case, do you think it a good idea to simply tune the --ipratio so that I-frames cost fewer bits than before? Yes, for CBR with a small buffer, you might want ipratio=1 or so.

In general: fewer B-frames should result in better bitrate prediction. And regardless of ratecontrol settings, never use more than 1 consecutive B-frame without b-adapt.

Overflow occurs, and the buffer was overflowed altogether a quarter of the whole decoding duration. How do you measure overflowed duration? Is that "a quarter of all frames were smaller than bitrate/fps even though the buffer was already full" ?

Note that the important figure when measuring overflow is not the number of frames, but rather the number of bits. In your case, 1470 vs 1500, so x264 used 2% fewer bits than it could have.

[chen]

How do you measure overflowed duration? Is that "a quarter of all frames were smaller than bitrate/fps even though the buffer was already full" ?
What i actually did was to continue with the decoding by overflow as if nothing happened. For example, if the decoder buffer was 120% full, which is in practice not valid, i ignore this overflow and go on with the decoding. After the decoding is over, the buffer fullness falls back to its original value, which i set as 90%. "A quarter of the duration" is the sum of all the spells when decoder buffer is over 100% full.

Note that the important figure when measuring overflow is not the number of frames, but rather the number of bits. In your case, 1470 vs 1500, so x264 used 2% fewer bits than it could have.
Will you please also give me some suggestions what to do after an overflow occurs? Shall i ignore it and go on decoding or shall i simply introduce one frame delay to avoid overflow?

By the way, is it also necessary to change the --pbration?

[akupenguin]

If the buffer ever exceeds 100%, clip it to 100% and continue. By the nature of VBV, the encoder can't ever use those bits that it missed, so just write them off.

You do not need to cahnge pbratio.

[chen]

Yes, for CBR with a small buffer, you might want ipratio=1 or so.

never use more than 1 consecutive B-frame without b-adapt.

i have tried with these suggestions, and the ipratio = 1 does help a little bit, but b-adapt makes things worse. I still believe that the source code is not perfect in ratecontrol, which simply ignores the VBV buffer overflow.