x264 re-encoding without using b-frames?

[Marin85]

Now, this may sound like blasphemy, but I had this weird idea about re-encoding Blu-ray (movies only, no anime or similar) using 2pass with -bframes 0. Output is 1080p, compression ~50%, of course target is to preserve transperency to the source. I did some tests, and as a reference encode I used 2pass with

--bframes 6 --b-adapt 2 --b-pyramid normal --merange 64 --me tesa,

cet. par.
For the same bitrate (say 1/2 from original bitrate), the P-frames of the b-frame-less encode looked closer to the source than the corresponding B-frames of the reference encode. In fact, even with much lower motion estimation settings, the P-frames still looked better than the B-frames.

Now, having in mind the theory of B-frames, where is the catch? Was I rather lucky with my samples? As usual, various opinions are most welcome! ;)

[MatLz]

Try using --b-adapt 1 instead.
Also --me esa and --merange 24/32 will output maybe 0.1% less quality than your tesa 64... but will be 2 or 3 time faster...

[Marin85]

Try using --b-adapt 1 instead.
Also --me esa and --merange 24/32 will output maybe 0.1% less quality than your tesa 64... but will be 2 or 3 time faster...
Why should I use the fast b-adapt instead of the optimal one for the reference encode? Also, esa vs. tesa is not really the issue at hand. I have enough processing power and the samples are relatively small (~200-500 MB), so I can afford running tesa 64 for the sake of it ;)

[MatLz]

I find --b-adapt 2 broken on fades.

[AnonCrow]

Marin, do you have the different x264 stats output handy, specifically the lines with the PSNR values and avg QPs for the different frametypes ?

[Marin85]

Marin, do you have the different x264 stats output handy, specifically the lines with the PSNR values and avg QPs for the different frametypes ?
Stupid me did not think of saving the log files. But I can re-run the tests the day after tomorrow again, if that is ok with you, as I am currently doing some other tests that will take a little bit longer and I would rather not interrupt them in the middle. Then, I guess, it will be easier to talk in more details.

[Marin85]

Last question before I start making the samples: should I turn off psy or should I instead use --tune psnr? If you ask me, I would rather use --tune psnr, because I have no observations how --bframes 0 looks like compared to, say, --bframes 6.

[LoRd_MuldeR]

If you are going to measure/compare PSNR values, then you must use "--tune psnr", otherwise your results will be botched. The same applies to SSIM and "--tune ssim".

(Psy optimizations, while massively improving the subjective quality, are known to hurt objective quality metrics, such as PSNR or SSIM)

[Marin85]

If you are going to measure/compare PSNR values, then you must use "--tune psnr", otherwise your results will be botched. The same applies to SSIM and "--tune ssim".

(Psy optimizations, while massively improving the subjective quality, are known to hurt objective quality metrics, such as PSNR or SSIM)
I am aware of that, that is why I was asking whether I should turn off psy completely or turn on --tune instead, before starting the encoding process ;)

[LoRd_MuldeR]

I am aware of that, that is why I was asking whether I should turn off psy completely or turn on --tune instead, before starting the encoding process ;)

The appropriate "--tune" for SSIM/PSNR will turn off the conflicting Psy optimizations:

--tune <string> Tune the settings for a particular type of source
or situation
Overridden by user settings.
Multiple tunings are separated by commas.
Only one psy tuning can be used at a time.
- psnr (psy tuning):
--aq-mode 0 --no-psy
- ssim (psy tuning):
--aq-mode 2 --no-psy

[Marin85]

I see now. I was not aware of that before. Thank you for the clarification!

[burfadel]

Your goal is to preserve quality compared to the source. Why tune the psy settings for SSIM or PSNR? By stating you want a transport encode, its visual settings that are much more important. Whilst its true b-frames are lower quality than p-frames, p-frames and b-frames are frame differences to the key-frame, and not representative of frames in their own right. Use of b-frames reduces bitrate used in that location, so you can use that bitrate elsewhere. You could change the pbratio so the b-frames are slightly higher quality, and you could change the b-frame bias to a negative value (say, -20, I wouldn't turn b-frames right off). Also if it worries you, use --bframes 3 (for example).

Try something like:
--subme 10 --me umh --psy-rd 1.00:0.10 --direct auto --trellis 2 --b-adapt 2 --b-bias -25 --merange 24 --aq-mode 2 --aq-strength 0.6 --pbratio 1.2 --ipratio 1.3

The --pbratio is normally 1.4 and the --ipratio is normally 1.4. Setting the pbratio is possibly beneficial in your claimed case, the ipratio could go either way, as although a lower ratio means better quality p frames, it takes higher bandwidth which is taken from keyframes... also changing the ipratio automatically effectively changes the pbratio, since the ipratio is the ratio between i frames and p frames, and pbratio is the ratio between p and b frames. If you raise the quality of the p-frame, the quality of the b-frame is therefore increased as well (even without changing the ratio).

So, in you case, instead of changing both, you may want to just change either, since reducing either will increase the b-frame quality.

[MatLz]

--no-mbtree is required if you want to play with --pbratio.

[Marin85]

Your goal is to preserve quality compared to the source. Why tune the psy settings for SSIM or PSNR? Because others asked about those metrics. And it would be nice to quantify the differences of the observed results. The downside is that the comparison for SSIM/PSNR won't exactly match my obersvations, which are without PSNR/SSIM, due to turning off the psy enhancement.

By stating you want a transport encode, its visual settings that are much more important.
Totally agreed, but is there any other way to perform any meaningful comparative statistics on the results, say, for the sake of "objectification"? I think --bframes 0 is a special case interesting for itself for the reasons I am going to mention below, so I think it would be nice to have some raw data in a meaningful way. I am open for suggestions here.


Whilst its true b-frames are lower quality than p-frames, p-frames and b-frames are frame differences to the key-frame, and not representative of frames in their own right. Use of b-frames reduces bitrate used in that location, so you can use that bitrate elsewhere.
Normally, I would agree, but in this case the scenario basically looks like following: Given the same bitrate, each pair of I-frames looks effectively identical and each P-frame from the --bframes-0-encode looks closer to the source than each corresponding P- or B-frame from the, say, --b-frames-3-encode. Thus, in this very special case, one could see the p-frames of the --b-frames-0-encode in their own right. Further, assuming one retains transperency to the encode at the given bitrate with compression of about 50+%, the bitrate reduction argument seems kind of outweighted.

Also note that the reference encode with b-frames uses --bframes 6 while maintaining very reasonable tradeoff between quality and compression, so one would think it would be normally pretty tough for other encodes under the same bitrate to compete with it (cet. par.)

If one wants to be nitpicky about incremental relative differences between the different types of frames, I guess one could say that in this case there is zero quality reduction from p-frames to b-frames. This in turn, however, is not necessarily the same as what I am describing above, because zero quality penalty from p-frames to b-frames does not necessarily imply that each p-frame from the first encode will look better than the corresponding frame from the second encod, especially if the first encode uses lower motion estimation settings and the second encode uses maxed out ME settings (!).


Setting the pbratio is possibly beneficial in your claimed case, the ipratio could go either way, as although a lower ratio means better quality p frames, it takes higher bandwidth which is taken from keyframes...
That is precisely one of the points here. Why hurting the I-frames by lowering the ipratio when it is not really necessary.


also changing the ipratio automatically effectively changes the pbratio, since the ipratio is the ratio between i frames and p frames, and pbratio is the ratio between p and b frames.
Yes, and this is precisely why that sentence does not make much sense to me :p since ratios are relative to the previous frame type by definition.


If you raise the quality of the p-frame, the quality of the b-frame is therefore increased as well (even without changing the ratio).
This I agree with. However, as mentioned above, I don't see a reason why to do this by hurting the I-frames.


So, in you case, instead of changing both, you may want to just change either, since reducing either will increase the b-frame quality.
Well, tempering with the --bpratio goes quality-wise pretty much in the same direction as setting zero quality penalty from p-frames to b-frames / not using b-frames at all. Thus, if the overall bitrate reduction argument is irrelevant, the question boils down to: How and how effective are bits redistributed when only I-frames and P-frames are present? And is there a type of a movie source where such redistribution fails to achieve transperency, given say 45-55% compression target, no matter what AQ and ME settings are selected?

On a related note: since P- and B-frame quality is achieved in an at least twofold way - once by ipbratios and once by reference frames, what does this amount to mathematically? In other words, how is this overdetermination (or is it rather stabilizing recursion?) balanced out?

[Dark Shikari]

Well, tempering with the --bpratio goes quality-wise pretty much in the same direction as setting zero quality penalty from p-frames to b-frames / not using b-frames at all.B-frames will give an overall compression improvement even if pbratio is 1.

And if MB-tree is on, and you're trying to reduce quality variation between frames, the parameter to mess with is qcomp, not any of the others mentioned in this thread.

[benwaggoner]

Yeah, I'm a bit baffled at the premise here. At a given bitrate, using B-frames will give better visual quality than not. Bidirectional prediction improves compression. A reference B-frame can use fewer bits at similar quality to a reference P-frame, and certainly shouldn't be less.

What happens if you just encode "normally" at your target bitrate? Figure out that baseline before trying crazy tweaks.

[Marin85]

What happens if you just encode "normally" at your target bitrate?The resulting bitrate is pretty much the same as the target bitrate, but same also holds for the "crazzy" encode. In both cases, the target bitrate is achieved. The only differentce is that the P-frames of the "crazzy" encode look closer to the source than the corresponding B- or P-frames of the "normal" encode.

[benwaggoner]

The resulting bitrate is pretty much the same as the target bitrate, but same also holds for the "crazzy" encode. In both cases, the target bitrate is achieved. The only differentce is that the P-frames of the "crazzy" encode look closer to the source than the corresponding B- or P-frames of the "normal" encode.
Well, if it looks better, it is better :)! It's all about whatever gives the best subjective quality in the end.

I'm not sure why what you're doing works in theory, but works in practice is better any day of the week.