Final Ratefactor after 1st pass, gone? [Archive]

View Full Version : Final Ratefactor after 1st pass, gone?

elguaxo

6th October 2006, 11:51

I usually do 2 passes when encoding in x264 and look at the final ratefactor after the first pass to see if the compressibility is ok.

Lately, after some encodes the final ratefactor was not there after the first pass. For example here:
Log for job job4-1

avis [info]: 720x368 @ 25 fps (133884 frames)
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [info]: slice I:1056 Avg QP:20.71 size: 37168
x264 [info]: slice P:52984 Avg QP:22.89 size: 16192
x264 [info]: slice B:79844 Avg QP:23.98 size: 5063
x264 [info]: mb I I16..4: 18.3% 0.0% 81.7%
x264 [info]: mb P I16..4: 18.3% 0.0% 0.0% P16..4: 72.9% 0.0% 0.0% 0.0% 0.0% skip: 8.8%
x264 [info]: mb B I16..4: 1.4% 0.0% 0.0% B16..8: 21.7% 0.0% 0.0% direct:15.8% skip:61.1%
x264 [info]: direct mvs spatial:99.8% temporal:0.2%
x264 [info]: SSIM Mean Y:0.9769964
x264 [info]: kb/s:1942.2

encoded 133884 frames, 3.45 fps, 1942.26 kb/s

In my tests, it seems to disappear only when No Dct Decimation is checked. Right now I am using MeGUI 0.2.3.2192 and x264 rev. 578A, but I think I have seen this problem since the first time I tried the DCT option when I read about it here (http://forum.doom9.org/showthread.php?t=115604).

Is it a bug?

elguaxo

7th October 2006, 13:58

Sorry to bump this. Is it possible to calculate the Final Ratefactor, as it was showed after the first pass, using the stats file? If yes how? Or where can I download a DOC that explains it? Thanks.

akupenguin

7th October 2006, 22:46

No, the data used to calculate CRF ratefactor is not written in the statsfile. 2pass ratefactor uses different data.

I can't reproduce the bug.

elguaxo

7th October 2006, 22:55

Thanks for the answer. I will make more extensive tests to see exactly what of the options causes this problem.

woah!

8th October 2006, 09:41

only time that happened to me, was when i put -b for bitrate cmd instead of -B

i used 1000 bframes because of that...

elguaxo

24th October 2006, 18:19

I made some 2-pass tests using Sharktooth's profiles (v33). For every test I used MeGUI 0.2.3.2192, x264 r593 (updated by MeGUI's autoupdate feature), MKV container and a CQM. I had no luck with any of the 4 profiles I tried.

HQ-Slow. No Final Ratefactor after the first pass:
Generating jobs. Desired size: 15728640 bytes
No audio encoding. Calculating desired video bitrate directly.
Setting video bitrate for the video jobs to 1253 kbit/s
Setting desired size of video to 15684608 bytes
Starting job job1-1 at 13:28:47
encoder commandline:
--pass 1 --bitrate 1253 --stats "H:\dvd\sample.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --merange 12 --thread-input --cqmfile "D:\megui\extra\M4G_MRM_v2.cfg" --progress --no-psnr --output NUL "H:\dvd\sample.avs"
successfully started encoding
Processing ended at 13:36:09
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 720x352 @ 23.98 fps (2400 frames)
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [info]: slice I:17 Avg QP:15.00 size: 39101
x264 [info]: slice P:768 Avg QP:16.95 size: 12411
x264 [info]: slice B:1615 Avg QP:18.21 size: 3358
x264 [info]: mb I I16..4: 28.3% 0.0% 71.7%
x264 [info]: mb P I16..4: 19.0% 0.0% 0.0% P16..4: 76.0% 0.0% 0.0% 0.0% 0.0% skip: 5.0%
x264 [info]: mb B I16..4: 0.9% 0.0% 0.0% B16..8: 19.4% 0.0% 0.0% direct:40.2% skip:39.5%
x264 [info]: direct mvs spatial:99.9% temporal:0.1%
x264 [info]: SSIM Mean Y:0.9819921
x264 [info]: kb/s:1248.3

encoded 2400 frames, 5.52 fps, 1248.49 kb/s

----------------------------------------------------------------------------------------------------------
job job1-1 has been processed. This job is linked to the next job: job1-2

HQ-Slower. No Final Ratefactor after the first pass:
Generating jobs. Desired size: 15728640 bytes
No audio encoding. Calculating desired video bitrate directly.
Setting video bitrate for the video jobs to 1253 kbit/s
Setting desired size of video to 15684608 bytes
Starting job job1-1 at 13:38:49
encoder commandline:
--pass 1 --bitrate 1253 --stats "H:\dvd\sample.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --thread-input --cqmfile "D:\megui\extra\M4G_MRM_v2.cfg" --progress --no-psnr --output NUL "H:\dvd\sample.avs"
successfully started encoding
Processing ended at 13:46:23
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 720x352 @ 23.98 fps (2400 frames)
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [info]: slice I:16 Avg QP:15.19 size: 37828
x264 [info]: slice P:760 Avg QP:16.94 size: 12500
x264 [info]: slice B:1624 Avg QP:18.21 size: 3365
x264 [info]: mb I I16..4: 29.3% 0.0% 70.7%
x264 [info]: mb P I16..4: 19.2% 0.0% 0.0% P16..4: 76.0% 0.0% 0.0% 0.0% 0.0% skip: 4.9%
x264 [info]: mb B I16..4: 0.9% 0.0% 0.0% B16..8: 19.3% 0.0% 0.0% direct:40.4% skip:39.5%
x264 [info]: direct mvs spatial:99.9% temporal:0.1%
x264 [info]: SSIM Mean Y:0.9819471
x264 [info]: kb/s:1244.4

encoded 2400 frames, 5.38 fps, 1244.58 kb/s

----------------------------------------------------------------------------------------------------------
job job1-1 has been processed. This job is linked to the next job: job1-2

HQ-Slowest. No Final Ratefactor after the first pass:
Generating jobs. Desired size: 15728640 bytes
No audio encoding. Calculating desired video bitrate directly.
Setting video bitrate for the video jobs to 1253 kbit/s
Setting desired size of video to 15684608 bytes
Starting job job1-1 at 13:48:08
encoder commandline:
--pass 1 --bitrate 1253 --stats "H:\dvd\sample.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --thread-input --cqmfile "D:\megui\extra\M4G_MRM_v2.cfg" --progress --no-dct-decimate --no-psnr --output NUL "H:\dvd\sample.avs"
successfully started encoding
Processing ended at 13:55:27
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 720x352 @ 23.98 fps (2400 frames)
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [info]: slice I:16 Avg QP:15.25 size: 37672
x264 [info]: slice P:765 Avg QP:17.09 size: 12816
x264 [info]: slice B:1619 Avg QP:18.36 size: 3198
x264 [info]: mb I I16..4: 29.2% 0.0% 70.8%
x264 [info]: mb P I16..4: 19.8% 0.0% 0.0% P16..4: 77.0% 0.0% 0.0% 0.0% 0.0% skip: 3.2%
x264 [info]: mb B I16..4: 0.8% 0.0% 0.0% B16..8: 18.9% 0.0% 0.0% direct:38.6% skip:41.7%
x264 [info]: direct mvs spatial:99.9% temporal:0.1%
x264 [info]: SSIM Mean Y:0.9819319
x264 [info]: kb/s:1245.5

encoded 2400 frames, 5.56 fps, 1245.69 kb/s

----------------------------------------------------------------------------------------------------------
job job1-1 has been processed. This job is linked to the next job: job1-2

HQ-Insane. No Final Ratefactor after the first pass:
Generating jobs. Desired size: 15728640 bytes
No audio encoding. Calculating desired video bitrate directly.
Setting video bitrate for the video jobs to 1253 kbit/s
Setting desired size of video to 15684608 bytes
Starting job job1-1 at 13:56:52
encoder commandline:
--pass 1 --bitrate 1253 --stats "H:\dvd\sample.stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 7 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --thread-input --cqmfile "D:\megui\extra\M4G_MRM_v2.cfg" --progress --no-dct-decimate --no-psnr --output NUL "H:\dvd\sample.avs"
successfully started encoding
Processing ended at 14:18:27
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 720x352 @ 23.98 fps (2400 frames)
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [info]: slice I:17 Avg QP:14.53 size: 34966
x264 [info]: slice P:905 Avg QP:16.38 size: 12595
x264 [info]: slice B:1478 Avg QP:17.46 size: 2418
x264 [info]: mb I I16..4: 9.9% 75.6% 14.6%
x264 [info]: mb P I16..4: 1.1% 6.6% 1.0% P16..4: 45.3% 26.3% 16.7% 0.6% 0.4% skip: 2.0%
x264 [info]: mb B I16..4: 0.0% 0.1% 0.0% B16..8: 24.0% 1.0% 3.2% direct: 8.4% skip:63.3%
x264 [info]: 8x8 transform intra:75.8% inter:62.1%
x264 [info]: direct mvs spatial:96.1% temporal:3.9%
x264 [info]: ref P 53.5% 15.9% 8.6% 5.1% 3.6% 3.2% 2.4% 1.5% 1.2% 1.1% 0.9% 0.8% 0.7% 0.6% 0.5% 0.4%
x264 [info]: ref B 81.0% 12.1% 2.5% 1.3% 0.7% 0.6% 0.4% 0.3% 0.2% 0.2% 0.1% 0.1% 0.1% 0.1% 0.1%
x264 [info]: SSIM Mean Y:0.9846925
x264 [info]: kb/s:1244.0

encoded 2400 frames, 1.86 fps, 1244.33 kb/s

----------------------------------------------------------------------------------------------------------
job job1-1 has been processed. This job is linked to the next job: job1-2

Sharktooth

24th October 2006, 20:18

Final ratefactor is only available in CRF mode.

akupenguin

24th October 2006, 20:21

Final ratefactor is only available in ABR (1pass or 1st pass).
With CRF, you specify the ratefactor, so you don't need x264 to echo the same number back.

I think I know what's wrong: that's ABR, not CBR. --vbv-maxrate 25000 is considered CBR, even though it's way above the target rate.

elguaxo

24th October 2006, 20:53

Final ratefactor is only available in CRF mode.
Then was it a bug before? It was always there in 2-pass modes and I used it as a compressibily check. It worked for me last month:
Generating jobs. Desired size: 1214251008 bytes
No audio encoding. Calculating desired video bitrate directly.
Setting video bitrate for the video jobs to 1552 kbit/s
Setting desired size of video to 1211864064 bytes
Starting job job2-1 at 22:32:34
encoder commandline:
--pass 1 --bitrate 1552 --stats "H:\dvd\x3.stats" --keyint 240 --min-keyint 24 --bframes 3 --b-pyramid --direct auto --filter -4,-4 --subme 1 --analyse none --me dia --thread-input --cqmfile "D:\megui\extra\eqm_avc_hr.cfg" --progress --no-psnr --output NUL "H:\dvd\x3.avs"
successfully started encoding
Processing ended at 01:55:37
----------------------------------------------------------------------------------------------------------

Log for job job2-1

avis [info]: 720x368 @ 23.98 fps (149751 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [info]: slice I:1445 Avg QP:19.87 size: 35457
x264 [info]: slice P:57545 Avg QP:22.34 size: 13342
x264 [info]: slice B:90761 Avg QP:23.02 size: 4343
x264 [info]: mb I I16..4: 38.2% 0.0% 61.8%
x264 [info]: mb P I16..4: 23.5% 0.0% 0.0% P16..4: 56.4% 0.0% 0.0% 0.0% 0.0% skip:20.1%
x264 [info]: mb B I16..4: 1.0% 0.0% 0.0% B16..8: 15.8% 0.0% 0.0% direct:16.0% skip:67.2%
x264 [info]: final ratefactor: 22.07
x264 [info]: direct mvs spatial:99.3% temporal:0.7%
x264 [info]: SSIM Mean Y:0.9723031
x264 [info]: kb/s:1553.9

encoded 149751 frames, 12.30 fps, 1554.04 kb/s

----------------------------------------------------------------------------------------------------------
job job2-1 has been processed. This job is linked to the next job: job2-2
Starting job job2-2 at 01:55:37
encoder commandline:
--pass 2 --bitrate 1552 --stats "H:\dvd\x3.stats" --keyint 240 --min-keyint 24 --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -4,-4 --subme 7 --trellis 2 --analyse all --8x8dct --me umh --thread-input --cqmfile "D:\megui\extra\eqm_avc_hr.cfg" --progress --no-psnr --output "H:\dvd\x3.mkv" "H:\dvd\x3.avs"
successfully started encoding
Job aborted, deleting output file...Deletion successful.
Processing ended at 11:26:34

Edit1:
I think I know what's wrong: that's ABR, not CBR. --vbv-maxrate 25000 is considered CBR, even though it's way above the target rate.
Thanks! I can delete that anyway, since I am not using it. I will try it now.

Edit2:
Great, it works now! Thanks!!!
Generating jobs. Desired size: 15728640 bytes
No audio encoding. Calculating desired video bitrate directly.
Setting video bitrate for the video jobs to 1253 kbit/s
Setting desired size of video to 15684608 bytes
Starting job job1-1 at 16:43:26
encoder commandline:
--pass 1 --bitrate 1253 --stats "H:\dvd\sample.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --me dia --merange 12 --thread-input --cqmfile "D:\megui\extra\M4G_MRM_v2.cfg" --progress --no-psnr --output NUL "H:\dvd\sample.avs"
successfully started encoding
Processing ended at 16:50:54
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 720x352 @ 23.98 fps (2400 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [info]: slice I:17 Avg QP:15.06 size: 39007
x264 [info]: slice P:760 Avg QP:16.93 size: 12458
x264 [info]: slice B:1623 Avg QP:18.21 size: 3368
x264 [info]: mb I I16..4: 27.8% 0.0% 72.2%
x264 [info]: mb P I16..4: 19.2% 0.0% 0.0% P16..4: 76.0% 0.0% 0.0% 0.0% 0.0% skip: 4.9%
x264 [info]: mb B I16..4: 0.8% 0.0% 0.0% B16..8: 19.6% 0.0% 0.0% direct:40.2% skip:39.4%
x264 [info]: final ratefactor: 18.16
x264 [info]: direct mvs spatial:99.9% temporal:0.1%
x264 [info]: SSIM Mean Y:0.9819720
x264 [info]: kb/s:1246.6

encoded 2400 frames, 5.46 fps, 1246.75 kb/s

----------------------------------------------------------------------------------------------------------
job job1-1 has been processed. This job is linked to the next job: job1-2

Sharktooth

25th October 2006, 03:33