[warning]: OVERFLOW levelcode=4211

[bkman]

What is this error that I'm seeing in 2-pass x264 encodes from rev. 798 onwards?

When I see it in the log, some scenes end up a pale white, and others have macroblocks floating around when decoded with CoreAVC.

Edit: Examples of commandline and output:
ob commandline: "C:\Apps\megui\tools\x264\x264.exe" --pass 2 --bitrate 5062 --stats "G:\Encoding\Recall\recall.stats" --level 4.1 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --no-cabac --subme 6 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --merange 24 --threads auto --thread-input --cqmfile "C:\Encoding\Matrices\M4G-HighDetail-V3.1.cfg" --progress --no-dct-decimate --output "c:\enc\r1.mkv" "c:\enc\rec.avs" --aq-strength 0.65
avis [info]: 1024x544 @ 23.98 fps (3936 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 SSSE3 Cache64
x264 [warning]: OVERFLOW levelcode=4661
x264 [warning]: OVERFLOW levelcode=4661
x264 [warning]: OVERFLOW levelcode=4515
x264 [warning]: OVERFLOW levelcode=4587
x264 [warning]: OVERFLOW levelcode=4587
x264 [warning]: OVERFLOW levelcode=4143
x264 [warning]: OVERFLOW levelcode=4143
x264 [warning]: OVERFLOW levelcode=4161
x264 [warning]: OVERFLOW levelcode=4161
x264 [warning]: OVERFLOW levelcode=4741
x264 [warning]: OVERFLOW levelcode=4741
x264 [warning]: OVERFLOW levelcode=4711
x264 [warning]: OVERFLOW levelcode=4711
x264 [warning]: OVERFLOW levelcode=4167
x264 [warning]: OVERFLOW levelcode=4167
x264 [info]: slice I:83 Avg QP:18.31 size: 69605 PSNR Mean Y:43.29 U:47.24 V:47.55 Avg:44.07 Global:42.27
x264 [info]: slice P:2051 Avg QP:19.66 size: 38599 PSNR Mean Y:42.08 U:47.09 V:47.39 Avg:43.16 Global:42.62
x264 [info]: slice B:1802 Avg QP:22.00 size: 10380 PSNR Mean Y:40.85 U:46.36 V:46.48 Avg:41.97 Global:41.25
x264 [info]: mb I I16..4: 18.8% 58.4% 22.8%
x264 [info]: mb P I16..4: 12.0% 23.4% 5.7% P16..4: 31.4% 20.0% 5.3% 0.0% 0.0% skip: 2.2%
x264 [info]: mb B I16..4: 1.3% 2.7% 0.7% B16..8: 31.6% 2.3% 4.2% direct: 3.5% skip:53.7%
x264 [info]: 8x8 transform intra:57.1% inter:47.2%
x264 [info]: direct mvs spatial:94.2% temporal:5.8%
x264 [info]: ref P 70.4% 16.1% 6.6% 3.8% 3.1%
x264 [info]: ref B 77.9% 14.3% 5.1% 2.6%
x264 [info]: SSIM Mean Y:0.9622327
x264 [info]: PSNR Mean Y:41.542 U:46.759 V:46.980 Avg:42.630 Global:41.936 kb/s:5051.00

encoded 3936 frames, 3.73 fps, 5051.71 kb/s
desired video bitrate of this job: 5062 kbit/s - obtained video bitrate (approximate): 121 kbit/s

[Dark Shikari]

I noticed this with --no-cabac also...

[Manao]

In CAVLC, at very low quantizers (or at low quantizer with some CQMs), you can create quantized coefficients that you can't code.

Two possibilities for you : use CABAC, or don't use the CQM.

Or wait for a fix in x264 (which might not come, since nobody cares about CAVLC)

[Dark Shikari]

In CAVLC, at very low quantizers (or at low quantizer with some CQMs), you can create quantized coefficients that you can't code.

Two possibilities for you : use CABAC, or don't use the CQM.

Or wait for a fix in x264 (which might not come, since nobody cares about CAVLC)That's especially odd, since CQMs require high profile--so there is very little reason to use CAVLC in such a case!

[Manao]

Let me clarify.

Without custom quant, a quantizer below 6 may generate a quantized coefficient too big for CAVLC.

With custom quant all at 1, a quantizer below 30 may generate a quantized coefficient too big for CAVLC.

IIRC, M4G-V3 has a coefficient at 4, so overflow may happen as soon as quantizer 18.

[Dark Shikari]

I could code a solution to this.

1. If the warning is hit during RDO, add a large value to the total bits encoded, to weight against the overflow.

2. If the value is hit during the actual encode, terminate the CAVLC encoding, re-encode the macroblock at 1 QP higher, and repeat the process until the overflow is gone.

[bkman]

Um.. I honestly don't know how --no-cabac got in there! :confused:

So it's ok, I can use other settings :p

[Gabriel_Bouvigne]

Wouldn't it be possible to just clip the coefficient if this case is detected?

(btw, this could be easily reproduced by encoding a totally black/white picture in intra16x16: the second DC transform will overflow the max allowed value)

[Manao]

Clipping is easy, but unefficient quality wise. Raising the quantizer until it doesn't overflow anymore is better.

[Dark Shikari]

This patch (http://paste.videolan.org/3743) fixes the problem in High Profile, and does its best to avoid it in Main Profile and below.