Poor x264 coding speed in multithreaded environment

[jva]

Hi,

I've been testing encoding of videos into H.264 using the x264 tool with my old and puny 1ghz AMD Duron with 512MB PC-133 SDRAM. I've been thinking to upgrade the machine to some cheap dual core system to boost coding speed.

I had a change to test the coding with a system containing Intel Core 2 duo E6400 cpu and 1GB 800Mhz DDR2 RAM. But, just to notice that compared to my Duron the encoding speed is only roughly double. (e.g. sample video that encodes 4.5fps on the duron, is encoded 10fps using core 2 duo) To be honest, I would have expected to see much more significant speed boost, so something must be wrong here..?

I use multithreading for x264, so that should not be the problem. I've also checked this from the system monitor, both cores are running 100%. Both systems run same system software baseline (e.g. Ubuntu linux 7.10 herd3).

One thing I was thinking is the cpu specific multimedia enhancements, like MMX, SSE?, etc. I am running pre-build binaries of ubuntu and current optimisations enabled are: MMX, MMXEXT, SSE and SSE2. I assume that the problem is not here either?

Any ideas for this performace would be greatly appreciated. If it is a reality that the speed improvement is only this small then I'm going to skip the computer upgrade. Thanks in advance.

Params used for x264 are:
--crf 20.0
--sar 1:1
--partitions all
--aud
--level 4.1
--ref 3
--mixed-refs
--bframes 3
--b-pyramid
--direct auto
--direct-8x8 1
--8x8dct
--analyse all
--subme 6
--me umh
--b-rdo
--bime
--weightb
--trellis 1
--pre-scenecut
--threads auto
--no-ssim
--no-psnr
--progress

[Manao]

You messed something up. The speed gain is at least 5x, not 2x.

[nm]

Perhaps the problem lies in the way you feed the video to x264. Have you tried encoding a raw video file or by using MEncoder instead of piping the video from ffmpeg?

I'd also suggest compiling the latest x264 version yourself with machine-specific optimizations, although it shouldn't matter that much.

[jva]

Perhaps the problem lies in the way you feed the video to x264. Have you tried encoding a raw video file or by using MEncoder instead of piping the video from ffmpeg?

I'd also suggest compiling the latest x264 version yourself with machine-specific optimizations, although it shouldn't matter that much.

I did some further testing. I run 10 seconds of sample video using following test cases:

- ffmpeg pipe, ubuntu binary x264
- local YUV file, ubuntu binary x264
- ffmpeg pipe, own optimized x264 build
- local YUV file, own optimized x264 build

and results are as follows:



encoded 10 seconds from ffmpeg pipe:

time ffmpeg -t 10 -i video.avi -f rawvideo - | x264 - -o video.264 --crf 20.0 --sar 1:1 --partitions all --aud --level 4.1 --ref 3 --mixed-refs --bframes 3 --b-pyramid --direct auto --direct-8x8 1 --8x8dct --analyse all --subme 6 --me umh --b-rdo --bime --weightb --trellis 1 --pre-scenecut --threads auto --no-ssim --no-psnr --progress 832x480

encoded 250 frames, 11.28 fps, 1095.36 kb/s

real 0m22.167s
user 0m35.974s
sys 0m1.028s

-----

encoded 10 seconds from local file:

time x264 video.yuv -o video.264 --crf 20.0 --sar 1:1 --partitions all --aud --level 4.1 --ref 3 --mixed-refs --bframes 3 --b-pyramid --direct auto --direct-8x8 1 --8x8dct --analyse all --subme 6 --me umh --b-rdo --bime --weightb --trellis 1 --pre-scenecut --threads auto --no-ssim --no-psnr --progress 832x480

encoded 250 frames, 11.56 fps, 1095.36 kb/s

real 0m21.631s
user 0m35.178s
sys 0m1.124s

-----

encoded 10 seconds from ffmpeg pipe, own compiled x264

encoded 250 frames, 12.87 fps, 1095.52 kb/s

real 0m19.430s
user 0m30.894s
sys 0m0.996s

-----

encoded 10 seconds from local file, own compiled x264

encoded 250 frames, 13.17 fps, 1095.52 kb/s

real 0m18.985s
user 0m30.642s
sys 0m0.940s


There seems not to be too much difference in local file vs. ffmpeg pipe. I doubt that mencoder will make any difference here. Another thing is that own optimized x264 build squeezes approx 10% away from the encoding time, even though the x264 is compiled with same switches as the ubuntu binary.

I do not have the duron figures here right now, but the encoding result is approx 4.5fps as I stated earlier, for this particular video. If the 5x encoding speed increase is to be expected, it is still quite far away from that.

Starting to wonder whether this core 2 duo machine has some generic trouble. For example cpufreq or some other daemon making the machine to run slower than it should be. I have to figure out, for example, why /proc/cpuinfo says cpu clock frequency to be 1.6Ghz, as it should be 1.86GHz. However, I doubt that 200Mhz clock increase will again make too much difference..

A friend tests the same encoding today with his single core opteron. I'd be interested to see the difference there..

[akupenguin]

I am running pre-build binaries of ubuntu and current optimisations enabled are: MMX, MMXEXT, SSE and SSE2. I assume that the problem is not here either?
You're missing SSSE3. That's not a huge difference, but just one more factor.

I have to figure out, for example, why /proc/cpuinfo says cpu clock frequency to be 1.6Ghz, as it should be 1.86GHz
E6400 should be 2.13GHz

[Cef]

Check the frequency while you're encoding, there's some technology lowering the freq multiplier while cpu is idle. It can be disabled in BIOS settings usually.

[DeathTheSheep]

Maybe you could try a build from http://x264.nl/ and see if there is any difference? It's built a bit differently, fprofiled from CVN directly, so maybe it's worth a shot.