x264 randomly "bottoms out" the bitrate in very low weight zone

[Prettz]

This seems to be some kind of bug in the 2-pass bitrate allocation when you use a very low weight zone. My encode is nominally 1092kpbs and I set the end credits to be a 9% weight zone. The credits are your normal scrolling white text against a black background. The credits are about 4:10 minutes long.

When I first looked at the credits after my second pass, I thought, "Oh awesome, the text still looks great! I could have even used a lower weight than 9!" The text was still very readable even in frames where the whole screen was filled with text.

However, watching further, at 2 different times in the last zone the whole screen becomes garbled for several seconds before slowly recovering back to normal quality; it looks like the quantizer just suddenly shoots through the roof and everything becomes horribly blocky.

I'm assuming this is a bug, and not just a limitation of x264's bitrate allocation in such extreme conditions (high-res, forced low bitrate, gigantic GOPs). I didn't see any error lines in MeGUI's log screen.

Here is all of my settings:

Resolution: 704x352

x264 v5.5.8 MeGUI
Bitrate: 1092
AVC Level: Unrestricted

Quantization Matrix: "eqm_avc_hr.cfg"

Deblocking strength,threshold: -4,-4
Keyframe interval: 300
Min GOP Size: 25

VBV Initial Buffer: 0.9 (def)
Bitrate Variance: 2.5 (def)
Quantizer Compression: 0.6 (def)
Temp Blur of est Frame Complexity: 20 (def)
Temp blur of Quant after CC: 0.5 (def)

Chroma ME: on
ME Range: 28
Scene Change Sensitivity: 40 (def)
ME Algorithm: Multi-Hex
Subpixel Refinement: 7 - RDO Level 2

Min Quantizer: 10 (def)
Max Quantizer: 50 (def)
Max Quantizer Delta: 4 (def)
Factor between I and P frame Quants: 1.4 (def)
Factor between P and B frame Quants: 1.3 (def)
Chroma QP Offset: 0 (def)

Macroblock Options: All

Trellis: 2 - Always
Number of Reference Frames: 5
Mixed: on
CABAC: on
No DCT Decimation: on
No Fast P-Skip: on

Number of B-frames: 3
Adaptive of B-frames: on
B-pyramid: on
RDO for B-frames: on
Weighted B-Prediction: on
Bidirectional ME: on
B-frame mode: Auto
B-frame bias: 0 (def)

Zones:
1172 2450 200%
213660 219640 9%


I can upload an .mp4 clip of the credits section, but I don't have my own web hosting. It's 4.65MB.

[akupenguin]

Possible cause: The garbling occurs at I-frames. It is a limitation of the bitrate allocation algorithm in the presence of big GOPs and very low motion. x264 ammortizes the complexity of an I-frame over a bunch of frames, but not over the whole 300 frame GOP. So at 9% weight, the I-frame and nearby P/B-frames get a high QP and are ugly. Later in the GOP, the I-frame is far enough away that its complexity is ignored, and so the frames are considered lower complexity and thus get a lower QP.

Workaround: --cplxblur 300. But this might be detrimental to the rest of the movie.

Solution: use a QP zone instead of a bitrate zone.

[Prettz]

Solution: use a QP zone instead of a bitrate zone.

I was hoping I didn't have to use a quant zone, because I like how weight zones can use as high or low quant as they need to keep the quality stable and bitrate no higher than it needs to be. Maybe my thoughts about "constant quant" are a little outdated, though. But I have no idea what quant to use, either.


Ok, so I just decided to look at it using ffdshow's visualizations to see what the area around the "anomolies" looked like, and to see if I could tell if they did indeed start with an I-frame, and I noticed this. The blocky section starts where the bitrate graph gets screwy. During this section the quants hover around 46-48, whereas in the rest of the clip the quants are generally between 37-42, even in complex frames. Yet, in the bad section, the bitrate graph makes it look like huge amounts of bits are being spent predicting only from the shittiest-looking frames :confused:

Is this what you were talking about, or is this something weirder?

[Prettz]

The attachment still hasn't gone through for some reason, so here's the same image, until it does:
http://img145.imageshack.us/img145/7025/snapshot20060927121136nq5.jpg

This bitrate graph is seriously weird. It doesn't look like what you were talking about, and it looks nothing like what I thought was going on in the problem areas.

[Prettz]

I did a 3rd pass setting the credits zone to quant 41. The quality looks the same as before for most of it. In the sections where it got screwed up before, the quality still takes a visible hit there although it isn't all horribly garbled anymore. And the bitrate graph is just as screwed up as before in there, possibly moreso now.

ffdshow shows the bitrate going as high as 900kbit/s in those sections, while it's between about 60 and 80kbit/s for the rest of the credits.

Out of curiosity, is this problem something that could possibly be corrected in future versions, or is it unavoidable with the way x264 works?