[2-pass] Inaccurate average bitrate

[Atak_Snajpera]

x264 r1649 (64bit) from x264.nl

Avatar BD
C:\>"C:\Users\Dawid\Documents\Delphi_Projects\RipBot264\tools\avs2yuv\pipebuf.exe" "C:\Users\Dawid\Documents\Delphi_Projects\RipBot264\tools\avs2yuv\avs2yuv.exe" "C:\temp\RipBot264temp\job1\job1.avs" - : "C:\Users\Dawid\Documents\Delphi_Projects\RipBot264\tools\x264\x264_x64.exe" --pass 1 --bitrate 2966 --stats "C:\temp\RipBot264temp\job1\job1.stats" --fps 24000/1001 --force-cfr --min-keyint 24 --keyint 240 --frames 232606 --sar 1:1 --level 4.0 --aud --nal-hrd vbr --vbv-bufsize 25000 --vbv-maxrate 25000 --filter 0,0 --ref 3 --bframes 3 --b-adapt 1 --b-pyramid none --subme 7 --aq-mode 1 --trellis 1 --partitions all --me umh --stdin y4m --output "C:\temp\RipBot264temp\video.264" - : 2
y4m [info]: 1280x720p 1:1 @ 10000000/417083 fps (cfr)
x264 [info]: using SAR=1/1
x264 [warning]: VBV bitrate (25000) > level limit (20000)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 4.0
C:\temp\RipBot264temp\job1\job1.avs: 1280x720, 10000000/417083 fps, 232606 frames
x264 [info]: frame I:2585 Avg QP:21.94 size: 77743
x264 [info]: frame P:127011 Avg QP:25.21 size: 22405
x264 [info]: frame B:103010 Avg QP:27.13 size: 5309
x264 [info]: consecutive B-frames: 23.2% 46.3% 14.5% 15.9%
x264 [info]: mb I I16..4: 33.5% 0.0% 66.5%
x264 [info]: mb P I16..4: 21.5% 0.0% 0.0% P16..4: 57.7% 0.0% 0.0% 0.0% 0.0% skip:20.8%
x264 [info]: mb B I16..4: 1.8% 0.0% 0.0% B16..8: 24.4% 0.0% 0.0% direct: 6.7% skip:67.1% L0:28.1% L1:49.6% BI:22.4%
x264 [info]: final ratefactor: 22.92
x264 [info]: coded y,uvDC,uvAC intra: 41.4% 46.3% 15.8% inter: 19.1% 13.6% 1.6%
x264 [info]: i16 v,h,dc,p: 31% 22% 24% 23%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 20% 18% 4% 10% 6% 7% 6% 8%
x264 [info]: i8c dc,h,v,p: 57% 18% 19% 6%
x264 [info]: Weighted P-Frames: Y:3.2%
x264 [info]: kb/s:2963.25

encoded 232606 frames, 15.37 fps, 2963.25 kb/s

C:\>"C:\Users\Dawid\Documents\Delphi_Projects\RipBot264\tools\avs2yuv\pipebuf.exe" "C:\Users\Dawid\Documents\Delphi_Projects\RipBot264\tools\avs2yuv\avs2yuv.exe" "C:\temp\RipBot264temp\job1\job1.avs" - : "C:\Users\Dawid\Documents\Delphi_Projects\RipBot264\tools\x264\x264_x64.exe" --pass 2 --bitrate 2966 --stats "C:\temp\RipBot264temp\job1\job1.stats" --fps 24000/1001 --force-cfr --min-keyint 24 --keyint 240 --frames 232606 --sar 1:1 --level 4.0 --aud --nal-hrd vbr --vbv-bufsize 25000 --vbv-maxrate 25000 --filter 0,0 --ref 3 --bframes 3 --b-adapt 1 --b-pyramid none --subme 7 --aq-mode 1 --trellis 1 --partitions all --me umh --stdin y4m --output "C:\temp\RipBot264temp\video.264" - : 2
y4m [info]: 1280x720p 1:1 @ 10000000/417083 fps (cfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.0
C:\temp\RipBot264temp\job1\job1.avs: 1280x720, 10000000/417083 fps, 232606 frames
x264 [info]: frame I:2585 Avg QP:24.86 size: 58349
x264 [info]: frame P:127011 Avg QP:26.49 size: 22757
x264 [info]: frame B:103010 Avg QP:28.91 size: 5485
x264 [info]: consecutive B-frames: 23.2% 46.3% 14.5% 15.9%
x264 [info]: mb I I16..4: 14.4% 68.0% 17.6%
x264 [info]: mb P I16..4: 2.4% 9.5% 2.0% P16..4: 42.2% 11.8% 2.7% 0.4% 0.3% skip:28.7%
x264 [info]: mb B I16..4: 0.3% 1.0% 0.1% B16..8: 40.3% 5.0% 1.0% direct: 1.6% skip:50.6% L0:37.2% L1:57.8% BI: 5.1%
x264 [info]: 8x8 transform intra:68.5% inter:74.9%
x264 [info]: coded y,uvDC,uvAC intra: 62.4% 65.1% 32.2% inter: 18.3% 15.8% 2.3%
x264 [info]: i16 v,h,dc,p: 32% 19% 8% 41%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 14% 15% 7% 10% 10% 10% 9% 9%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 16% 13% 7% 12% 10% 11% 7% 7%
x264 [info]: i8c dc,h,v,p: 57% 17% 18% 8%
x264 [info]: Weighted P-Frames: Y:3.7%
x264 [info]: ref P L0: 62.4% 22.4% 10.5% 4.4% 0.3%
x264 [info]: ref B L0: 93.1% 6.9%
x264 [info]: kb/s:2973.73

encoded 232606 frames, 11.83 fps, 2973.73 kb/s

[Dark Shikari]

You're complaining about an inaccuracy of 0.26%? Seriously? Is this some kind of practical joke?

[Atak_Snajpera]

This is not a one case http://forum.doom9.org/showthread.php?p=1409119#post1409119
Accuracy is crucial for AVCHD. BTW. Shouldn't be more accurate with longer footage?

[Dark Shikari]

This is not a one case http://forum.doom9.org/showthread.php?p=1409119#post1409119Of course it's not. x264 is not designed to get 99.99999999999% accuracy on bitrate.Accuracy is crucial for AVCHD.Then I suggest you stop using x264 and instead use your magical psychic encoder from space.

[Selur]

@Atak_Snajpera: try to lower qcomp to 0.5 (helped me)

[Dark Shikari]

@Atak_Snajpera: try to lower qcomp to 0.5 (helped me)Lowering qcomp should not affect this at all. If it does, it's sheer dumb luck. It's no different from any other trivial modification to settings -- it's rerolling the dice on something that is 100% random.

[Selur]

than I just had 'sheer dumb luck' :)

[AnonCrow]

Might also consider a slower preset , 3/n-pass encoding or even slow-firstpass.

[LoRd_MuldeR]

Might also consider a slower preset , 3/n-pass encoding or even slow-firstpass.

I don't think using three passes helps here.

It probably would help, if x264 didn't hit the desired target bitrate by accident, i.e. x264 is off by more than what is to be expected. But in this case the deviation from the target bitrate is within the expected range. x264 intentionally doesn't hit the target bitrate 100% exact, because (as far as I know) enforcing the bitrate too strictly would hurt quality. So if you do a third (or even forth pass), x264 still won't hit the bitrate 100% accurate, as it still doesn't try to. I assume using "--slow-firstpass" won't help either, for the very same reason. I wonder if "--ratetol" has an effect on 2-Pass...

[Dark Shikari]

Yes, --ratetol has an effect on 2-pass and does exactly what you think it does: affect how aggressively x264 tries to hit the target :p

[LoRd_MuldeR]

Yes, --ratetol has an effect on 2-pass and does exactly what you think it does: affect how aggressively x264 tries to hit the target :p

Then this should be solution here, right ???

[Dark Shikari]

Then this should be solution here, right ???Of course.

There is a way that x264 can be improved here slightly though. Currently the rate tolerance converges to zero towards the end of the video. But this isn't useful if the end of the video has no bits to adjust anyways (e.g. all black, like the end of the credits). So I've made a commit locally to do the adjustment based on the position in the video bit-wise instead of frame-wise. This reduced misprediction by 50% on a quick contrived test case.

[Atak_Snajpera]

You're complaining about an inaccuracy of 0.26%? Seriously? Is this some kind of practical joke?
I wouldn't complain if bitrate was 8kbps below. The real joke is that average Joe instead of ~4473MB gets 4483MB. Now try to put this on DVD+R without cutting.

[Dark Shikari]

I wouldn't complain if bitrate was 8kbps below. The real joke is that average Joe instead of ~4473MB gets 4483MB. Now try to put this on DVD+R without cutting.Then stop trying to cut it so close. If your software can't author a DVD because it chose too high a bitrate, it isn't x264's fault.

[Atak_Snajpera]

If your software can't author a DVD because it chose too high a bitrate, it isn't x264's fault.
DVD-SL is treated as 4480 MB instead of 4482.62 MB. When encoder is able to not go above calculated bitrate then final size is ALWAYS below 4480MB. Look at my example.
2966 kbps / 2974 kbps = 0.9973100201748487 * 4483 MB (<- size with 2974kbps) = 4471 MB

[Dark Shikari]

DVD-SL is treated as 4480 MB instead of 4482.62 MB. When encoder is able to not go above calculated bitrate then final size is ALWAYS below 4480MB. Look at my example.
2966 kbps / 2974 kbps = 0.9973100201748487 * 4483 MB (<- size with 2974kbps) = 4471 MBSorry, I give up. You refuse to read a single post that I write, so I'm not going to bother writing them for you.

[Atak_Snajpera]

I give up as well Your Highness. Obviously We are not on the same page.

[Audionut]

Yeah, If I'm encoding for media I always leave enough room.

So If the bitrate calc says encode at 3460, I'll encode it at 3400.
Honestly, 60kb/s is neither here nor there when aiming for size.
And I know I won't have to re-encode because the encoder overshoot the bitrate.

x264 intentionally doesn't hit the target bitrate 100% exact, because (as far as I know) enforcing the bitrate too strictly would hurt quality.

Having the encoder hit the exact bitrate would hurt the quality as LM stated.
So, the way x264 works, shaving a few kb/s off the encode and having that encode hit 2950, is better than an encoder made to hit 2966 exactly.
It just means that the end user has to apply a little common sense.

Besides, I challenge you 2 find the difference in quality between 2966 and 29??

[Blue_MiSfit]

LOL why don't you just undershoot the bitrate by like 1%??? I always do at least this much when I'm preparing stuff for optical media.

Derek

[Groucho2004]

Obviously We are not on the same page.
Clearly.

If you don't like the fact that x264 doesn't hit the target bitrate exactly, why don't you use ratetol as suggested? If this doesn't satisfy you - like you're getting 4789.01 Kbps instead of 4789.0 Kbps - how about looking for a different encoder that can do it better?

[MatLz]

I agree with Atak_Snajpera, obviously there is a problem.

There is something broken somewhere, specially when the clip is a 2h40min movie.
My point of view is: more long the video is, more accurate should be the resulting bitrate.
0.26% is really a huge deviation.

@King's bouffons
:p

[Blue_MiSfit]

I would scarcely consider .26% to be a huge deviation. What's wrong with keeping ~1-2% overhead in your bit budget?!

Derek

[MatLz]

I would scarcely consider .26% to be a huge deviation.I think it is huge.What's wrong with keeping ~1-2% overhead in your bit budget?!I think it is OOT.

Well, since I use 2pass mode, I never had this problem; dunno if it's related : I always use slow-firstpass with exactly the same settings of the second pass.

[Audionut]

I think it is huge.

Make a 2 pass encode at 3000 and one at 2992.2

Post back your results showing how that .26% destroys the quality of the second encode.

Well, since I use 2pass mode, I never had this problem;

The op did 2 pass and got this "problem".

I vote you troll.

[MatLz]

Make a 2 pass encode at 3000 and one at 2992.2

Post back your results showing how that .26% destroys the quality of the second encode.That's OOT.I vote you troll.Why did you have quoted only the half of my post ?

[Atak_Snajpera]

If you don't like the fact that x264 doesn't hit the target bitrate exactly, why don't you use ratetol as suggested?
Because I wasn't aware of that option amigo?

If this doesn't satisfy you - like you're getting 4789.01 Kbps instead of 4789.0 Kbps
Don't exaggerate ,ok?

[IgorC]

Each pixel of display won't receive the exact correspondent amount of luminance due to analog distortion. LED luminance isn't constant and varies with temperature and humidity etc. Add to it that the electronic components very hardly have real error less than 1%.

~0.25-0.5% of bitrate reduction isn't perceptible because it's at least 10x times lower than mentioned distortions.

So you can safely reduce bitrate in those fractions of percent to hit specific filesize.

[Blue_MiSfit]

Well said, IgorC!

@MatLz: OK so Audionot was a little snippy, but he brought up a very valid point.

Do a pair of 2 pass encodes. One with bitrate ~2% lower than the other. Be as selective as you want, pick and choose some really tough scenes, and show us any meaningful difference.

You'll have a VERY hard time doing this I think. Then, consider that x264's bitrate deviance in the OP's case was less than .3% (1/6 of your synthetic case).

If you're not willing to do something like this, and continue to insist "0.26% is really a huge deviation"... well... then honestly you're being a troll.

Derek

[MatLz]

[2-pass] Inaccurate average bitrateThat is the topic.
Why persist on the way "try to make a difference between encode@x kbps and same encode@ x-y kbps" ?
Well, yes deviation wasn't so huge, it was so small Atak didn't can put his movie on his dvd.

My last post in this thread.

[Groucho2004]

Why persist on the way "try to make a difference between encode@x kbps and same encode@ x-y kbps" ?
Well, yes deviation wasn't so huge, it was so small Atak didn't can put his movie on his dvd.

This drivel doesn't make any sense.

Apparently some people have serious problems grasping the concept of allowing for deviations from the target bitrate.

I have observed these deviations also in Mainconcept AVC/PRO where it is even worse than with x264 (which even allows to minimize this by simply setting a switch).

[Trahald]

I think the cases have been stated. One group believes the deviation is not acceptable and the other does not. The only thing keeping the thread open is doing his upping the hostility yet changing nothing. Thread closed.