intel core 2 quadro benchmarks

[pyrates]

I noticed that toms hardware posted some benchmarks for the coming up of intel core 2 quadro cpu. But none for the x264 1080p encoding. It certainly does look promising though. Here's the link:

Link (http://www.tomshardware.com/2006/09/10/four_cores_on_the_rampage/page10.html)

Would x264 be able to take advantage of the 4 cores though?

[Sagittaire]

yes but encoding optimisation seem not very good for XviD or x264 ...

[foxyshadis]

The more cores you have, the more of a bottleneck avisynth becomes. But I would expect x264 in general to follow similar curves to mainconcept's H.264 or WME9; since xvid 1.2 uses a much different form of parallelization I have no idea how it scales to 4+ cpus.

[Kostarum Rex Persia]

I just saw encoding results, and I must say: not too bad.

foxyshadis, do you mean that Avisynth isn't optimized well for multi-core environment?

[Egladil]

Anyone tried the multithreading avisynth mod?

http://www.avisynth.org/tsp/

[Revgen]

TSP's plugin works well with many filters. Some (PixieDust, MVTools) don't work well with it.

I've only used 2 cores with it though. I'm not sure what it would do with 4 cores.

[emmel]

The more cores you have, the more of a bottleneck avisynth becomes.

Yes. But why not simply split the file in N-parts, and trivially distribute them over N-cores? And finally join the results. That would scale exactly linearly, or does it? Of course, if N is big, exact load balancing may be problematic (one thread could run much longer than the others). But anyways.

[Manao]

emmel : have a look at Elder ( search on this forum )
Sagittaire : XviD scales quite well, and without quality loss. x264 scales not as well, mainly because some parts aren't or can't be multithreaded, and multithreading implies a relatively small quality loss ( visually quite annoying at low bitrates, meaningless at high bitrates ).

[omion]

Yes. But why not simply split the file in N-parts, and trivially distribute them over N-cores? And finally join the results. That would scale exactly linearly, or does it? Of course, if N is big, exact load balancing may be problematic (one thread could run much longer than the others). But anyways.

I actually made a program that could do that with x264, but it's not quite linear. I got something like 1.95x with 2 processors (although it's been a while since I've used it, so that number may be way off).

The main problem with many processors is with the splitting. There's no easy way to determine where x264 will put an I frames without analyzing each frame, so chances are the split point will be in the middle of a scene. For consistant quality, you need to re-render that scene. Obviously, this will not scale well if your number of processors is greater than the number of scenes in your movie.

I minimized the load-balancing problem by splitting to more parts than cores, then having each core "check out" the next split part. This, of course, increased the number of split frames and therefore the number of re-encoded scenes.

I think the reason that my setup didn't scale perfectly had to do with disk accesses. Two programs were reading and writing in two separate places on the same hard disk.

But as I said, it's been a while since I used that program to encode anything, and it was in the super-pre-alpha stage, so it may be a problem with my optimizations.

[emmel]

Thanks Manao for pointing out Elder. I'll definitely have a look at it.

I just wonder why it has been forgotten? I mean, no news about it for almost 6 months...?

I actually made a program that could do that with x264, but it's not quite linear. I got something like 1.95x with 2 processors (although it's been a while since I've used it, so that number may be way off).
Thats more than close enough to perfect imho.
The main problem with many processors is with the splitting. There's no easy way to determine where x264 will put an I frames without analyzing each frame, so chances are the split point will be in the middle of a scene.
Do you think that this really is a problem? Isn't the key-frame-interval for example in HD-DVD and BD-disks something like max. 15? I fail to see how adding one or two extra key frames every now and then due to the splitting could cause problems.

I think "temporal parallelism" is indeed something worth considering. I just did a series of modest tests with 2000 "normal" PAL-frames, and got convinced that this might the better strategy for me at least:

1) scalar encoding (one thread):
encoded 2000 frames, 20.93 fps, 412.83 kb/s
real 1m35.750s

2) spatial parallel encoding (two threads):
encoded 2000 frames, 30.22 fps, 421.48 kb/s
real 1m6.359s

3) temporal parallel encoding (file splitted in two, one thread each):
encoded 1030 frames, 19.06 fps, 369.70 kb/s
real 0m54.234s
encoded 970 frames, 17.94 fps, 461.25 kb/s
real 0m54.297s

Conclusions (for this test):
- speedup approx. 1.45 when using "spatial parallelism"
- speedup approx. 1.77 when using "temporal parallelism"

Heres my test script:
# encode 2000 frames, one thread:
echo 'mpeg2source("c.d2v").trim(1,2000)' > c.avs
time x264 --threads 1 --thread-input -o c.264 c.avs

# encode 2000 frames, two threads:
echo 'mpeg2source("c.d2v").trim(1,2000)' > c.avs
time x264 --threads 2 --thread-input -o c.264 c.avs

# split by file size (1030+970 frames), one thread each:
echo 'mpeg2source("c.d2v").trim(1,1030)' > c1.avs
echo 'mpeg2source("c.d2v").trim(1031,2000)' > c2.avs
time x264 --threads 1 --thread-input -o c1.264 c1.avs &
time x264 --threads 1 --thread-input -o c2.264 c2.avs &