x264 - Same Encode, Different Size

[GamezR2EZ]

I am in the process of comparing the difference in speed with x264 running on a different number of threads.
I have done this before in small scale (10 min clips), I am now doing this for an entire movie.

I will eventually make a graph of threads vs time elapsed.

I decided to also check the various files against each other based on size. I was initially going to hash out each file, assuming the same size.
I was surprised to find that the first and second encode varied about 230k. Every encode produced a slightly different file.
The only change was the number of threads used. Granted this is across a 4gb file, but still, why the variance?

x264 encode parameters:
avs2yuv movie.avs -o - | x264_64 --crf 18 --preset veryslow --tune animation --weightp 1 --bframes 3 --nal-hrd vbr --vbv-maxrate 40000
--vbv-bufsize 30000 --level 4.1 --keyint 24 --b-pyramid strict --slices 4 --aud --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1
--threads=1 --fps 24000/1001 -o movie.264 --stdin y4m -

NOTE: The only difference in the encodes was changing the "threads=" option.

AVS script:

loadplugin("FFMS2.dll")
FFIndex("movie.MKV", indexmask=7, dumpmask=0, overwrite=true)
FFVideoSource("movie.MKV")
AssumeFPS(24000,1001)


PS: Initial results show an almost 92% increase in speed going from 1 thread to 2 threads on an 1 hour and a half long movie.
On 3 threads only 24% increase (over 2 threads). More to come... possibly a pie chart!

[Dark Shikari]

1) threads + VBV isn't deterministic in x264.

2) 3 threads is "only a 24% increase" most likely because you're bottlenecking on the source decode (or you only have a 2 core CPU).

[GamezR2EZ]

Thanks for the reply.

1) VBV itself is not deterministic? Or is it the combination of threads and VBV?

2) Im running a 1090T (6 core) running with 8gb DDR3 1600. Source is on a SSD capable of ~200mbs sustained read (which is not being even approached when encoding).
The AVS source was posted as well as the command line parameters. I see nothing bottlenecking this (other than the limited thread count). Am I supposed to expect a higher speed increase?

EDIT:
Also, the programs avs2yuv is only using 10-20% of 1 core, so it shouldn't be a bottleneck, even if it doesnt support multiple cores.

[Dark Shikari]

1) VBV itself is not deterministic? Or is it the combination of threads and VBV?Both, hence the +.

2) Im running a 1090T (6 core) running with 8gb DDR3 1600. Source is on a SSD capable of ~200mbs sustained read (which is not being even approached when encoding).
The AVS source was posted as well as the command line parameters. I see nothing bottlenecking this (other than the limited thread count). Am I supposed to expect a higher speed increase?Yes, 3 threads should give about 3x speed.

[GamezR2EZ]

Yes, 3 threads should give about 3x speed.

1 Thread: 1.66 fps
2 Threads: 3.12 fps
3 Threads: 4.06 fps

With all 6 threads I have only done a small scale test (at this point) which yielded 8.07 fps.
So I can agree that falls in line with that statement. Thats more like 6 threads is 5x speed though.

My understanding of the answer is:
Given the same input, output will vary (very slightly) when using VBV or a different number of threads.

Is this correct?

[upyzl]

>#3

I still recommend you to try using without avs for thread testing... (use x264 ffms input directly)

maybe you'd better use x264 with demuxer-thread patch (such as sada5' builds (http://sada5.sakura.ne.jp/files/index.php))

and...you just do it for speed test, so I think it no need to set BD-Compatible parameters...right?

[GamezR2EZ]

@upyzl

I have done tests without avs before. However I was going for real world testing this time.
Since I encode for BD-compatibility, I want to know the speed difference I am going to get.

While this is a very simple AVS script (in fact, it doesnt do anything!), I do use AVS options from time to time.
I don't like reworking the encode process as I have all of this automated.

Somewhat off topic though. Thanks for ffms directly suggestion. Its been about 2 years since I looked into that (it wasn't there 2 years ago). Now I can do this on my true love, linux.

[RunningSkittle]

use ffms2 built against ffmpeg-mt then

[AnonCrow]

1 Thread: 1.66 fps
2 Threads: 3.12 fps
3 Threads: 4.06 fps
With all 6 threads I have only done a small scale test (at this point) which yielded 8.07 fps.

Using preset veryslow implies b-adapt 2 which is not multi-threaded ,but then again ,you use only 3 b-frames.

To really test just x264 with different number of threads:

use a raw source , so that decoding speed is not a factor
(or use a GPU-based decoder);

test with the faster presets to find bottlenecks outside x264;

from preset slow downwards, manually set b-adapt 1 and compare the encoding speed and actual
number of b-frames used to b-adapt 2 when using a large number of threads;

do 2-pass encoding tests , so that frame type decision is done in 1st pass.