XviD on multicore not as fast as excpected

[ilhyfe]

...in fact it doesn't encode faster than on 1 CPU. I turned off GMC and (re)encoded a 5 min XviD in 140 seconds on a core2duo @ 3ghz. Then I turned off one core, set the number of threads to one and encoded the video in the same time (140 sec).
Then I started to encode this 5 min sample two times and set the affinity of both vdub's to CPU 0 or 1. Both movie's were ready in 160 sec, so it took 15% more time but I got both done instead of one only.
I am using XviD 1.2-127 dev build, does anyone have an idea why this one isnt working as expected?

[foxyshadis]

Koepi's build is a year and a half old now, you should switch to the one from http://ffdshow.faireal.net/mirror/XviD/ and test again.

To be sure there's no decoding bottleneck, you should make sure you're not starting from a video that uses GMC or Qpel, you should ensure fast recompress is on in vdub, you should test encoding with slower options like qpel, high motion search, VHQ, trellis, to minimize the effect of any decoding bottlenecks.

In more complex processes it's probably that the encoding just isn't slowing you down all that much.

[ilhyfe]

Ok I tested the xvid.cvs.head.MTK.2007.03.10.7z version and it's about 15% faster than my old one. But CPU usage could be higher, I have 55 - 56% in first pass, 65 - 75 in second pass with 2 threads (2 core's). I used a video without QPel and GMC.
I also tried the Core2 version, but it's not faster :)

EDIT: I just tested it with 1 Thread and set the affinity of vdub to core 2. Its still faster then the old codec, but just 10% faster then with 2 threads...

[Awatef]

You should be aware that the SMP optimizations are still experimental in XviD. Most "mature" optimizations will lead to "only" 50 to 60% faster encoding, so don't put the lath too high ;)

[ilhyfe]

You should be aware that the SMP optimizations are still experimental in XviD. Most "mature" optimizations will lead to "only" 50 to 60% faster encoding, so don't put the lath too high ;)

I don't but it could be higher then 10% :)

[celtic_druid]

I can max out both cores here. No idea what performance boost that equates to though.

[henryho_hk]

Celtic_druid, are you using Virtualdub or xvid_encraw?

[celtic_druid]

VDub:
http://img57.imageshack.us/img57/7218/vdubav7.png
encraw:
http://img57.imageshack.us/img57/8890/encrawpe3.png

[squid_80]

FWIW on my Q6600 cpu usage rarely gets over 30%, no matter how many threads are used (tested between 2 - 8).

[Blue_MiSfit]

Brutal. Even with MT Avisynth??

I can get almost full utilization on a dual processor, dual core mac pro 2.66 GHz Xeon, when I have a common source like DV or DVD.. Of course I usually use pretty heavy Xvid settings and filters ;)

~MiSfit

[Lenny_Nero]

If your using windows XP its not very good with multi cores, and cant come close to Win 2000 'out of the box' so to say and needs to be patched to work right with multi cores.

The other important thing is the speed that you send the data to the encoder, if you have only one hard drive that is running the OS and the encoder and the data to and from then its going to be slow. If I use one PATA (or SATA) drive with my encoding box to do everything I dont get much past 35~45 fps for an AGK 2 pass encode. If I run from one Hdd doing nothing but feeding the data to another Hdd doing the same but just receiving the data, with another running the OS and encoding, and they are all on their own controllers I get more than 110 fps with the same settings.

CPU power has something to do with it but not everything.

[squid_80]

Brutal. Even with MT Avisynth?? Yes. I think the problem is the frames are actually encoded too quick, MT avisynth doesn't have time to finish prefetching from mpeg2source.
The other important thing is the speed that you send the data to the encoder, if you have only one hard drive that is running the OS and the encoder and the data to and from then its going to be slow. If I use one PATA (or SATA) drive with my encoding box to do everything I dont get much past 35~45 fps for an AGK 2 pass encode. If I run from one Hdd doing nothing but feeding the data to another Hdd doing the same but just receiving the data, with another running the OS and encoding, and they are all on their own controllers I get more than 110 fps with the same settings.mpeg2source on it's own can decompress frames at 140fps (standard pal res) on this machine. HD speed really only comes into play if you're using uncompessed/huffyuv/lagarith/other low compression source material.
One solution I have found is to load four tcpserver scripts, written in a way so that each script returns every fourth frame of the source e.g.:
Script 1 returns frames 1, 5, 9, 13...
Script 2 returns frames 2, 6, 10, 14...
Script 3 returns frames 3, 7, 11, 15...
Script 4 returns frames 4, 8, 12, 16...
Then have one script combine them all together using tcpsource and interleave. Feed this script to the encoder and you get four seperate threads doing prefetching up to four frames in advance. This seems to work particularly well for hcenc. It would be better if the frames weren't being pushed through the internal tcp/ip stack but works pretty well as is.

[henryho_hk]

Encoded too quickly.... I have never thought of that.... Does it mean that you got similar overall speed with VHQ1 and VHQ4?