x264 B-Frames test

[KR]

Hello everyone !

I've done a test recently: I encoded the same clip twice.

The clip consists of 848x480 non-interlaced video @ 29.97 frames/sec, and lasts 833 frames.
I know it is rather short, but still.

In all cases, the following settings were used:



2 pass encode, at 800 kbps.

CABAC: Yes Max IDR-Frame interval: 300 Bitrate variability: 60%
Deblock: Yes, 0-0 Min IDR-Frame interval: 24 Min QP: 10
Max ref. frames: 1 Scene cut threshold: 40 Max QP: 51
Sample AR: 1 : 1 Max QP Step: 4

8x8 DCT: Yes, Chroma ME: Yes, Partition decision quality: 6-RDO, ME Method: Exhaustive search, ME Range: 16
All analysis features enabled (except B-Frames search for case 2).

Here are the variations for the settings:

#1 - (Based on default)
Max consecutive B-Frames: 2, Adaptative: Yes, Use as references: No, Bias: 0, Weighted biprediction: Yes,
Direct mode: temporal
Keyframe boost: 40%, B-Frames reduction: 30 %
#2 - (My settings)
Max consecutive B-Frames: 0
Keyframe boost: 0%

Evaluation consisted of 2 parts:
1. Subjective evaluation via an AviSynth script which allows me to see the two versions side-by-side, and, at the bottom, the difference between each of them and the original.
Result: #2 looks better, and has less distortion from the original.

2. Objective evaluation using Video Quality Studio (latest version 0.4RC3), Feng Xiao's VQM quality metric.
Because of Video Quality Studio's inability to deal with B-Frames, I had to recompress all video with huffyuv in lossless mode (no converting to yuv).
I then deleted the first 2 frames from the #1 version (which were all green), and the last 2 frames from the source and the #2 version. This allowed me to compensate for the B-Frames decoding lag while not inducing any temporal difference between the clips. I verified that Frame 0 of the original was exactly the same (minus encoding artifacts) as Frame 0 of #1 and #2, and that all clips were of the same length.

Results:
VQM
#1 Y U V
Average: 0.978471, 0.314691, 0.253653
Std Deviation: 0.697539, 0.191155, 0.131668


#2 Y U V
Average: 0.478045, 0.260273, 0.21377
Std Deviation: 0.0941015, 0.0621438, 0.0469162

Obviously #2 is better (closer to 0).

Of course it would not make sense to immediatly draw conclusions from this test, because it has been done on only 1 source.
I invite you to conduct similar tests (if possible, with longer clips and at higher kbps) and post your results.

Thanks.

P.S.: Sorry the length of this post.

[berrinam]

Interesting test. I've looked through your post, and I didn't find any mention of the filesizes, though. Obviously it is only fair if you compare files of equal bitrate, and I would be surprised if you can get accurate rate control with only 833 frames. Could you please post the filesizes?

[akupenguin]

@KR:
Have you tried playing with B-frame reduction, and number of B-frames, and bias?

@berrinam:
Ratecontrol will be accurate even on 833 frames, though it may take 3 or 4 passes.

[KR]

As requested, here are the filesizes:

Source (FFV1)
101 652 480 bytes
x264
#1: 2 826 240 bytes
#2: 2 818 048 bytes

#1 (which has B-Frames) is bigger than #2, but #2 is still better-looking.

Remember that for #2 there are 2 differences: No b-Frames and Keyframe boost: 0 % (Keyframe boost Off)

I've always though that B-Frames were useless... Maybe I'm wrong, but I think that the general idea behind B-Frames is just wrong. People on this forum have said that the goal was to compress these frames more, so that lower quants could be used elsewere. Thing is, for the overall quality to be increased, it is necessary that B-Frames are compressed more efficiently (not loosing as much quality), which is clearly not the case here.

Others have said that B-Frames were used only on high-res encodes, to help gain compressibility. It doesn't make sense, because even on high-res encodes, according to my tests, it is still better to disable B-Frames.

Finally, I've read somewhere that B-Frames would only be usefull when an object hiding another moves away and reveals it. Spending as little as 30 sec., thinking about this situation, one can easily see that it is simply useless to use B-Frames: P-Frame encoding is already very efficient in this situation. B-Frames only bring problems, because there has to be code indicating whether information is taken from the preceding or the next frame, and this takes more space. And the fact that they generally are not used as reference is another problem, because the P-Frame that will come after will need more texture info (as opposed to motion vector), because it is based on a frame too distant. And if they are used as reference (I just repeated the test with this setting), man I tell you it's ugly.

It's easy to understand how codecs such as VP6, which apparently do not use any B-Frames, can still be quite efficient.

And let's not forget that B-Frames mean more complexity during encoding and decoding.


Did anyone try this test on other sources ?


By the way, I wish to say thanks to x264 devs for making this great, free codec.

[Joe Fenton]

You misunderstand B-Frames. First, remember what they are - they are called B-Frames because they are BI-DIRECTIONALLY predicted. They use both previous frames and future frames to encode the current frame. There will be content that B-Frames encode better than other kinds of clips, and vice-versa. With a tiny clip like you gave, you probably just had very little content that was B-Frame friendly. An entire episode or movie will generally have lots of B-Frame friendly content and encode better with B-Frames. In fact, I have yet to see anything longer than 15 minutes that looked better without B-Frames.

For an example of B-Frame friendly content, consider a smooth horizontal pan. New video is constantly appearing from nowhere on one side unless you bi-directionally predict the frames. Pans and scrolls usually look terrible without b-frames. Zooming out is another place where b-frames usually help.

I suggest you either use a longer test, or try test on different segments that do different video effects - pans, scrolls, zooms, cuts, dissolves, etc..

[Sharktooth]

It has been discussed to death and the final conclusions is B-frames definatly help compression (and so final quality).
Obviously there are scenarios where b-frames arent needed/wanted (for example lossless encoding).

[omion]

It has been discussed to death and the final conclusions is B-frames definatly help compression (and so final quality).
Obviously there are scenarios where b-frames arent needed/wanted (for example lossless encoding).
I've been kind of wondering why they help with AVC. I thought the main reason to use them in ASP is that nothing depended on them so they could be compressed more without affecting all the upcoming frames.

With AVC, both B and P frames may be used as references, and both B and P frames may be skipped over.

From what I know, the only real remaining difference is that B frames use 2 motion vectors and P frames only use 1. And I don't see how that naturally leads to greater compressibility.

Am I completely overlooking something?

[MeteorRain]

i wonder your setting about i->p and p->b quantizer ratio.
if one is only i&p frame, and quantizer is like 18,22,22,22,22.....,18,22.....
and another is ipb, like 17,20,29,29,20,29,29,20,29....
then the answer is obvious. no need to do such test and we can see some frames improves a little and lots of ones declines a lot.
my behaviour is to set both ratio at 1.0, so that w/o b-f the quantize should like 20,20,20,20,.....
while w/ b-f 19,19,19,19,.....
and then, calculate the PSNR and see the result.

From what I know, the only real remaining difference is that B frames use 2 motion vectors and P frames only use 1. And I don't see how that naturally leads to greater compressibility.

Am I completely overlooking something?
IMHO, when it use "BI-DIRECTIONALLY predicted" feature, there will be less data to deal with (if more, change to p-f!), and cause to high compressibility.

regards,
MeteorRain

[sixide]

From what I know, the only real remaining difference is that B frames use 2 motion vectors and P frames only use 1. And I don't see how that naturally leads to greater compressibility.

Am I completely overlooking something?

I am not an expert, but I think you're overlooking one major thing - low or no motion scenes. In this case, the difference between P-frames n and n+3 for example may be almost nil, allow B-frames n+1 and n+2 to be coded as almost all zeroes and without any intra-coded blocks, taking little to no space. I can't think of a good way to explain the savings in pans and such, but the benefit is similar.

[Caroliano]

From http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-x264.html

bframes: If you are used to encoding with other codecs, you may have found that B-frames are not always useful. In H.264, this has changed: there are new techniques and block types that are possible in B-frames. Usually, even a naive B-frame choice algorithm can have a significant PSNR benefit. It is interesting to note that using B-frames usually speeds up the second pass somewhat, and may also speed up a single pass encode if adaptive B-frame decision is turned off.
weight_b: In typical cases, there is not much gain with this option. However, in crossfades or fade-to-black scenes, weighted prediction gives rather large bitrate savings. In MPEG-4 ASP, a fade-to-black is usually best coded as a series of expensive I-frames; using weighted prediction in B-frames makes it possible to turn at least some of these into much smaller B-frames. Encoding time cost is minimal, as no extra decisions need to be made. Also, contrary to what some people seem to guess, the decoder CPU requirements are not much affected by weighted prediction, all else being equal.

Unfortunately, the current adaptive B-frame decision algorithm has a strong tendency to avoid B-frames during fades. Until this changes, it may be a good idea to add nob_adapt to your x264encopts, if you expect fades to have a large effect in your particular video clip.

[KR]

@Sharktooth:
It has been discussed to death and the final conclusions is B-frames definatly help compression (and so final quality).
I see. Very convicing. There is simply nothing I can reply to such a perfect argument.

BTW, my post was initially not about discussing, it was about testing.


@Joe Fenton
For an example of B-Frame friendly content, consider a smooth horizontal pan. New video is constantly appearing from nowhere on one side unless you bi-directionally predict the frames.
I don't see how B-Frames help. Sure, you won't need I-Blocks on them, but eventually, you will need some. Lets say the scene is an ideal, very short horizontal pan. You encode it with B-Frames, which are not used as references:
#1: IPbbP
then without B-Frames:
#2: IPPPP

In #1: the b frames are based on the first and last P frames. b frames will not introduce any new information, but the last P frame contains a lot of new information.
In #2: Each P frame is based on the preceding frame, and introduces a bit of new information each time.

Why is #2 more efficient than #1 ? Because B-Frames need bits to indicate whether each block is based on preceding or following frame. P-Frames, however, do not need to store such information (It is always based on the preceding frame.)

Perhaps I am missing something. I find it strange to come up with an entirely different conclusion.

I thank you all for engaging in this discussion, but I beg you to do the test with your sources and post your results !

[Joe Fenton]

In #1: the b frames are based on the first and last P frames. b frames will not introduce any new information, but the last P frame contains a lot of new information.
In #2: Each P frame is based on the preceding frame, and introduces a bit of new information each time.

Why is #2 more efficient than #1 ? Because B-Frames need bits to indicate whether each block is based on preceding or following frame. P-Frames, however, do not need to store such information (It is always based on the preceding frame.)


In the case being discussed, the b-frame introduces almost no new info. So whether b-frames helps depends on if the bits needed to indicate which direction the blocks are from is less than the bits encoding new info introduced on the side with p-frames. If you look at how new info is encoded compared to how to reference existing blocks, it takes more space to encode new info or you'd NEVER bother to check for if there are matching blocks elsewhere, even in p-frames.

In fact, x264 not only looks for blocks elsewhere, you can specify it look for blocks a considerable number of frames from the current frame. So it obviously takes less space to encode using blocks many frames away from the current frame than to encode new info. That is just common sense.

[omion]

I don't see how B-Frames help. Sure, you won't need I-Blocks on them, but eventually, you will need some. Lets say the scene is an ideal, very short horizontal pan. You encode it with B-Frames, which are not used as references:
#1: IPbbP
then without B-Frames:
#2: IPPPP

In #1: the b frames are based on the first and last P frames. b frames will not introduce any new information, but the last P frame contains a lot of new information.
In #2: Each P frame is based on the preceding frame, and introduces a bit of new information each time.

Why is #2 more efficient than #1 ? Because B-Frames need bits to indicate whether each block is based on preceding or following frame. P-Frames, however, do not need to store such information (It is always based on the preceding frame.)

Perhaps I am missing something. I find it strange to come up with an entirely different conclusion.There is a minor error in your example, in that B frames and P frames aren't really that simple in AVC:
In a P frame, each macroblock may be one of 6 types, where each type indicates the partitioning and motion estimation mode. B frame macroblocks have 24 different types. Exactly which frames are used for prediction are variable too; the entire codec is really complicated.
The end result is the same, though; the B frame needs to send more data for macroblock types. But that might mean that the B frame is more accurate in other respects, since it can be tuned finer.


@ Joe Fenton:
You're right that the B frame introduces almost no new info, but the next P frame (which it is presumably is based on) has to have a lot of new info.
The point that KR was making is that if you have no B frames, each P frame has 1 frame's worth of new info added to it. However, if you use (say) 2 B frames, the P frame after the two will have 3 frame's worth of new info!
Basically, if there is new info in a scene, it has to go somewhere. Using B frames will only cause that info to get "bunched up" into the P frames. (assuming that makes any sense...) :p


In order to test the efficiency of B frames vs. P frames, I made a slow-pan test clip from the anime I'm currently encoding.
I froze 1 frame of the anime and did a sweep down the middle. Each frame is offset 1 pixel right and 1 pixel down from the previous.

I did two 3-pass encodes, one with and without B frames. The command lines were:
Pass1: x264.exe --qp 20 --stats ".\x264-slowpan.stats" --subme 5 --ref 4 --me hex --merange 16 --bframes 0|5 --weightb --b-pyramid --b-bias 0 --ipratio 1.40 --pbratio 1.30 --qcomp 0.60 --analyse all -o "1 1st SlowPan +B.mp4" ".\SlowPan.avs"
Pass2/3: --pass 3 --bitrate 40 --stats ".\x264-slowpan.stats" --subme 6 --ref 4 --me hex --merange 16 --bframes 0|5 --weightb --b-pyramid --b-bias 0 --ipratio 1.40 --pbratio 1.30 --qcomp 0.60 --analyse all --8x8dct -o "1 2nd SlowPan +B.mp4" ".\SlowPan.avs"I realize that some of the options are the default; I keep them in so that I can tweak them easier

I got the 40kbps final bitrate from the bitrate of the first pass from the B frame clip.

Here's one frame from the results:
With B:
http://people.ucsc.edu/~rswilson/other/b-test/t+B.png
Without B:
http://people.ucsc.edu/~rswilson/other/b-test/t-B.png :eek:

Looking at the command output for both, it seems that the B frames are 90% BSKIP macroblock types, and the P frames are only 70% PSKIP. I don't exactly know why, but that is presumably why the one with B frames looks so much better.

Conclusion:
B frames win BIG for slow pans. Although this was an artificial test, it indicates that B frames are indeed more efficient at dealing with easily-predictable scenes than P frames.

Files:
Output +B[42KiB] (http://people.ucsc.edu/~rswilson/other/b-test/3rd_SlowPan_+B.mp4) -B[39KiB] (http://people.ucsc.edu/~rswilson/other/b-test/3rd_SlowPan_-B.mp4)
Command window output +B[4KiB] (http://people.ucsc.edu/~rswilson/other/b-test/SLOWPAN_+B.txt) -B[3KiB] (http://people.ucsc.edu/~rswilson/other/b-test/SLOWPAN_-B.txt)
Resultant stats files +B[20KiB] (http://people.ucsc.edu/~rswilson/other/b-test/x264-slowpan_+B_3.stats) -B[20KiB] (http://people.ucsc.edu/~rswilson/other/b-test/x264-slowpan_-B_3.stats)

[Sharktooth]

@Sharktooth:

I see. Very convicing. There is simply nothing I can reply to such a perfect argument.

BTW, my post was initially not about discussing, it was about testing.
Well, bframes were tested as well... so maybe if you dont trust me please :search:

[Joe Fenton]

@ Joe Fenton:
You're right that the B frame introduces almost no new info, but the next P frame (which it is presumably is based on) has to have a lot of new info.
The point that KR was making is that if you have no B frames, each P frame has 1 frame's worth of new info added to it. However, if you use (say) 2 B frames, the P frame after the two will have 3 frame's worth of new info!
Basically, if there is new info in a scene, it has to go somewhere. Using B frames will only cause that info to get "bunched up" into the P frames. (assuming that makes any sense...) :p

In order to test the efficiency of B frames vs. P frames, I made a slow-pan test clip from the anime I'm currently encoding.
Looking at the command output for both, it seems that the B frames are 90% BSKIP macroblock types, and the P frames are only 70% PSKIP. I don't exactly know why, but that is presumably why the one with B frames looks so much better.

Conclusion:
B frames win BIG for slow pans. Although this was an artificial test, it indicates that B frames are indeed more efficient at dealing with easily-predictable scenes than P frames.


I think the reason it looks better is you save SO MANY bits in b-frames on the pan that you have more bits available for a better looking p-frame.

[akupenguin]

You're right that the B frame introduces almost no new info, but the next P frame (which it is presumably is based on) has to have a lot of new info.
The point that KR was making is that if you have no B frames, each P frame has 1 frame's worth of new info added to it. However, if you use (say) 2 B frames, the P frame after the two will have 3 frame's worth of new info!
Basically, if there is new info in a scene, it has to go somewhere. Using B frames will only cause that info to get "bunched up" into the P frames. (assuming that makes any sense...)

By that logic, you could say that the whole movie has one movie's worth of new info no matter what frame types it's made of, so how can you compress anything?
The answer to this (and the basic premise of MC/DCT), is that you can usually save bits by concentrating entropy into smaller regions. Even aside from any block-related overhead, it's cheaper to code 4x16 pixels at the edge of a P-frame and then copy pieces into 3 B-frames than it is to code 1x16 pixels 4 times.

[omion]

By that logic, you could say that the whole movie has one movie's worth of new info no matter what frame types it's made of, so how can you compress anything?
The answer to this (and the basic premise of MC/DCT), is that you can usually save bits by concentrating entropy into smaller regions. Even aside from any block-related overhead, it's cheaper to code 4x16 pixels at the edge of a P-frame and then copy pieces into 3 B-frames than it is to code 1x16 pixels 4 times.
You certainly do have 1 movie's worth of new info in a movie, that doesn't make it incompressible. It just means that you need to compress that much data, and shouldn't matter where it gets compressed.

An example:
Let's say the movie is panning down at 16 pixels per frame, and the new info can't be motion-estimated to any previous info. The blocks would therefore be encoded as "I" blocks, no matter which frame they were in. Why would it be more efficient to have all the new rows represented in one frame? The encoding of each macroblock would be the same no matter where it was (an "I" block predicted from the pixels above it)

I tested this theory, with a static scene scrolling down at 16 pixels per frame. It turns out x264 didn't want to use B frames at all, so I forced them with --no-b-adapt.

The file sizes were almost exactly the same. With 0 B frames, the first 4 frame sizes (not counting the initial I frame) were: 3851,3818,3853,3826. With 1 B-frame, they were: 43,7611,45,7603. With 3 B: 48,48,47,15097. Added together, these are all within 1% of each other. All the rest of the frames were similar.

Basically, using B frames with motion at 16 pixels per frame provides no benifit, so x264 decided against using them.

This does not, however, contradict my last test. I suppose with motion less than 16 pixels per frame, it is good to "fill up" macroblocks to contain all new data. However, if the macroblocks already contain new data, it doesn't matter where they are stored.