Strange behavior : first pass slower than 2nd pass.

[jpsdr]

Hello.

Using actually 1332 version.
I'm using the following to encode :

First pass :
x264.exe --profile "high" --preset "placebo" --tune %TUNING% --pass 1 --bitrate %2 --stats %STAT_FILE% --level "4.1" --b-adapt 2 --direct "auto" --subme 2 --trellis 0 --aq-mode 2 --partitions "none" --no-8x8dct --qcomp 0.4 --me "dia" --rc-lookahead 60 --sar 1:1 --no-mixed-refs --slow-firstpass --threads 0 --thread-input --output NUL %E_SRC%
2nd pass :
x264.exe --profile "high" --preset "placebo" --tune %TUNING% --pass 2 --bitrate %2 --stats %STAT_FILE% --level "4.1" --b-adapt 2 --direct "auto" --subme 10 --trellis 2 --aq-mode 2 --partitions "all" --8x8dct --qcomp 0.4 --me "esa" --rc-lookahead 60 --sar 1:1 --threads 0 --thread-input --output %E_DST% %E_SRC%


If on SD size the first pass is a little faster than second pass, on HD size first pass is the same than second pass !!
Results of encoding are the following for a 1240x700 video of 4549 frame (encoded in YV12 with Lagarith 1.3.20) :
First pass : 4,72fps.
2nd pass : 4,71fps.
Tuning was animation, bitrate was 5000.

I've noticed on the first pass, only one CPU is used at almost 100%, all the others are not used, but on the 2nd pass,
all the 8 CPU are used at around 95%...
Is it normal ? Is there some trick to speed up the process ? What trigger the use of all CPU ?

[nurbs]

Your command line is very strange since there are a lot of redundant options in it, but it looks like you are using 16 b-frames with --b-adapt 2 and that usually slows the first pass down a lot. Since it has to do with threading it's especially noticeable since you have eight cores.
If you want to speed it up use sane settings.

[dstln]

Is there a reason you're using slow firstpass and placebo settings? :P Placebo is mathematically better but the difference is less than any measure of statistical significance in almost all cases, and it's more than twice as slow as veryslow... For an example, I don't think I've seen anything that's actually used 16 bframes, and crazy bframes are terrible for first pass regardless of other settings. I'd recommend doing a quick crf test to see how many refs/bframes are actually being used. Usually you won't see much past 6 bframes if even that...

Anyway, if you're going for the mathematically best (possibly on the order of kb or a meg or two over an encode), use placebo. I'm just trying to say that if you want to reasonably max quality, use veryslow. First pass will blaze then.

[nurbs]

He isn't actually using slow first pass. Placebo normally would but he's overriding it with the command line options so that they are almost the same as with a fast first pass and that's why he's surprised that it is slower. Some of the command line arguments in the first pass and most (9 from a quick look) in the second pass are still redundant.

[Chengbin]

That's normal.

b-adapt 2 is not multithreaded yet, so first pass, with your insane settings, will be slow, and only using 1 core. The second pass has all 8 cores working. This only happens when you have insanely slow settings and >4 core CPUs, and you have both

[jpsdr]

Ok, my point is to speed up only the first pass... As b-adapt 2 is present on the 2 pass, and second pass have all 8 core working, the problem is not with this (but i may be assuming wrong). I'm wondering, wich minimal thing will trigger the use of the 8 cores. Is in the first pass increasing a little me or subme and or anything else wich give me the following :
- All 8 core used.
- Settings 'lower' than 2nd pass, so theoricaly faster.
Or the only fact that pass is at 1 will simply disable any kind of multi-threading ???
With the differences of settings between the 2, there must be some kind of intermediate level that will still use the 8 cores.

[nurbs]

Frametype decision is done in first pass, so --b-adapt doesn't do anything in second pass. Multithreading isn't disabled in first pass, it's just that frametype decision isn't multithreaded (as I hinted and Chengbin said).
If you want a faster first pass use less b-frames, if you want more CPU utilization in first pass use slower settings.

[jpsdr]

Ok, thanks, i think now i get the picture. I realy don't know how encoding work, so i didn't know that b-adapt wasn't used in the 2nd pass. If i'm using slowfirst pass it's beacause, i've remembered read something about the fact that some qualtity improvement algorithm (i don't remember exactly what) worked only if frame references were the same between the passes. From what i've understood, fast/normal first pass change the references frames, so it's why i'm not using it.

[nurbs]

Fast first pass doesn't change anything that is important and it wouldn't be any faster than your settings since your encoding speed is limited by the frametype decision. Frankly if you don't know what you are doing just stick to the presets. You got a good rig and you apparently got time to waste so use --preset veryslow.
If you want more or less the same settings you have now you can dox264.exe --preset veryslow --tune %TUNING% --pass 1 --bitrate %2 --stats %STAT_FILE% --aq-mode 2 --qcomp 0.4 --sar 1:1 --output NUL %E_SRC%
x264.exe --preset veryslow --tune %TUNING% --pass 2 --bitrate %2 --stats %STAT_FILE% --aq-mode 2 --qcomp 0.4 --sar 1:1 --output %E_DST% %E_SRC%
Personally I'd also get rid of the --qcomp 0.4, because for one it's changed by some of the tunings and secondly I don't really know what it does and I also doubt that you know. I'd also add a --bframes 5 to both passes for speed.
If you actually want level 4.1 compliant streams you have to add the --vbv-maxrate and --vbv-bufsize for that and also reduce the number of reference frames if your video has more pixels than 1280x720. (Unless the --level 4.1 switch actually does that for you, but I think it just tags the stream accordingly which it will automatically do anyway) At 1920x1080 the maximum allowed number of reference frames at level 4.1 is 4.
Check wikipedia (http://en.wikipedia.org/wiki/H264) for more details on that.

[jpsdr]

Thanks for the suggestions. I don't have time to waste, it's just that i'm having a PC dedicated to video encoding/processing, i used it only for that. And as i'm not a professional of video, it's just for personnal use, i don't have time constraint, so, i don't realy mind if encoding take 2 or 3 days, and effectively, i've bought a good rig (wasn't cheap !!) knowing it will realy help, and even realy don't mind time, i still prefer when encoding take 1 day than 6.