x264 encoding times in CRF and VBR second-pass differ by twice

[Shevach]

Dear experts

i observe a strange issue with x264 (version 0.142, compiled by gcc).
On same content and with almost same parameters encoding time (user time) of single-pass CRF is about twice smaller than that of second-pass VBR.
For example gprof reveals that the function 'x264_macroblock_deblock_strength' in VBR run takes ~19s while in CRF run it takes ~3s.
The function actually does not depend on content but it accesses 'mb.cache' data. Maybe accesses to mb.cache slows down performance in VBR second-pass.

Did anybody experienced similar problem with x264 and can explain the difference in performance between single-pass CRF and second-pass VBR?

[nevcairiel]

Speed and size are not directly correlated, so I don't think you can draw any conclusions from any of this. The size difference may simply be from different settings in the rate-control, which being CFR vs 2-pass VBR are already different.

[LoRd_MuldeR]

On same content and with almost same parameters encoding time (user time) of single-pass CRF is about twice smaller than that of second-pass VBR.

How does the resulting bitrate of the CRF encode compare to that of the VBR encode? Especially when using CABAC, higher bitrate requires more encoding time.

(Also you should post your complete command-line)

[EDIT]

For me the second pass of a 2-Pass ABR encode actually runs quite a bit faster than a single-pass CRF encode:

x264_8bit_x64.exe --crf 22.0 --preset medium --tune film --output foreman_4K.mkv foreman_4K.y4m
[...]
encoded 248 frames, 6.33 fps, 26116.76 kb/s

x264_8bit_x64.exe --bitrate 26117 --pass 2 --stats foreman_4K.stats --preset medium --tune film --output foreman_4K.mkv foreman_4K.y4m
[...]
encoded 248 frames, 8.50 fps, 26466.45 kb/s

[Shevach]

Dear -i lord_mulder

In my runs the bitrates in CRF and second-pass VBR are different but not much (CRF ~9.3 Mbps and VBR ~10.9 Mbps ). This difference in bitrates can't apparently explain a huge gap in performance.

Unfortunately you don't specify how many cores were involved in each run. For example, if in CRF mode 3 cores are involved while in VBR mode 4 cores are involved then VBR is quicker.
i use the command 'time' and compare 'user time'.
Anyway, i used your setting (excepting bitrates and resolution) and found that CRF (=22) gives ~11m of 'user time' and the second pass VBR gives 9m43s of 'user time'.
Probably i set a parameter which slows down second-pass VBR

[Groucho2004]

Unfortunately you don't specify how many cores were involved in each run. For example, if in CRF mode 3 cores are involved while in VBR mode 4 cores are involved then VBR is quicker.Since he didn't specify the number of threads, x264 will default to (logical cores * 1.5) in both cases.

Probably i set a parameter which slows down second-pass VBRQuite possible.
However, Mulders result makes more sense since CRF is probably always a bit slower than the second pass of a VBR encode because most decisions about bitrate distribution, etc. were already done in the first pass.

[Shevach]

With your setting (except resolution and bitrate) i empirically found that setting '--threads 2' in VBR run raises 'user time' to ~15m from 9m43s.

[Groucho2004]

With your setting (except resolution and bitrate) i empirically found that setting '--threads 2' in VBR run raises 'user time' to ~15m from 9m43s.Longer encoding time and lower CPU usage are to be expected if you're setting "threads" to a number less than the number of logical cores of your CPU.

[Shevach]

Longer encoding time and lower CPU usage are to be expected if you're setting "threads" to a number less than the number of logical cores of your CPU.

Dear Groucho2004

as i say 'encoding time' i mean 'user time' in the context of linux 'time' command. Actually 'user time' is a pure time of a process duration while 'real time' is a wall clock time.
i observe counter-intuition cases with x264, in some situations (or x264 settings) 'user time ' of CRF is much smaller than 'user time' second-pass VBR on same content and almost same parameters (excluding --crf and --bitrate). Notice that bitsizes of both outputs are comparable, thus i exclude impact of CABAC.