CPU usage in SMP

[da franksta]

Hi,

my dual athlon system (1700+ with 1GB RAM) averages a
fine 1.7 RT in CCE (DVD2SVCD 1.05b4). CPU usage doesn't
get over 60%, so I have the feeling it might be faster.

Any ideas how to optimize? What is your CPU usage?

TIA

[Tessai]

Just because an application is multi-threaded, does not mean that it will use 100% of the dual/quad/etc. CPU setup. Multi-threading only works in speeding up an application if the threads can be run independent of one another.

Here's an example:

All of the odd steps of a program operate on one CPU, all of the even steps on the other (keep in mind I'm talking about a dual CPU system). So steps 1 and 2 execute at the same time, because they are able to run independently. Wow! We have a 100% speed increase. However, step 4 requires a result of step 3. Here, we have a problem. Before step 4 can run, it requires that step 3 be completed. Here, we have a 50% speed loss (so to speak). A few more steps pass. Now, step 19 requires the result of step 18. Again, there is a speed loss.

In short, just because you have a dual cpu system, don't always expect your encoding to always use 100% of your CPUs, nor that encoding will take 50% of the time (when compared to a single CPU setup at the same speed). A lot of filters/etc. use single-threaded functions and such, therefore, they are unaffected by a multiple CPU system. Just because CCE supports SMP doesn't mean every filter you're using (I'm not sure about Avisynth and it's filters... I'll have to look into that) supports it. This could (and probably does) explain why you are not getting the "optimal" CPU usage that you are looking to achieve. You should know the limitations of an SMP system BEFORE you build one.

[Kedirekin]

I typically get between 51% and 65% cpu usage in CCE. It depends on what filters I'm using in my avs script.

I believe CCE is fully multi-threaded, and would take full advantage of both CPUs if it could. The problem is our frame serving methods are only single-threaded, and they actually take most of the clock cycles. Of that 51-65%, probably 49% is dedicated to the frame serving, so CCE is actually taking very little of the processor pie.

I did get significantly higher encoding speeds when I fed CCE directly with a pic video MJPEG encoded avi. The pic video codec isn't multi-threaded, but it is much faster than avs frame serving. I got CPU usage of about 75%, and CCE speeds close to 3.0.

Unfortunately this has some rather irritating side effects:
- MJPEG is lossy (though it's highest setting is very good)
- CCE 2.5 suffers from the 2 GB avi file limit (which means, at that highest setting, you can only encode about 8 minutes of video)

The first side effect isn't too nasty, and I believe you can solve the second one by using a later version of CCE.

[Tessai]

If you do use TMPG with SMP enabled, it WILL use 100%. I ran a test with Avisynth using MPEG2DEC using CCE 2.5 and TMPG 2.5. The test AVS script looked like:

LoadPlugin("c:\winnt\system\MPEG2DEC.dll")
mpeg2source("C:\mpegcaps\Rush Hour 2\rushhr2.d2v")
BicubicResize(480,360)
AddBorders(0,60,0,60)

For this test I used Rush Hour 2 in full 29.97 (in other words, without IVTC). My system specs are as follows:

2x PIII 1ghz (133x7.5) @ 1.125 ghz (150x7.5)
Abit VP6
768 megs of PC133 Crucial CAS2 ram
IBM 40gig 60GXP
IBM 60gig 60GXP (test was run with this drive, using the ATA/100 cable and controller onboard)
ATI Radeon 64 meg VIVO (with newest DX8.1 drivers)
SB Live 5.1 (with drivers directly from the cd)
3com SOHO 10/100 NIC
Windows 2000 SP2
DirectX 8.1

Interestingly enough, CCE, on average, would only use about 65% of the CPUs, while TMPG (using readavs.dll) used 100% of the CPUs. Unfortunately, I do not have screenshots for this. What I gather is that there is something missing from CCE in it's multi-threading technique or it may be something different altogether.

Both TMPG and CCE were set to 2 passes. I stopped them each about 10% into the first pass. Minimum bitrate was set at 0, maximum at 2520, and average at 1150kbps. If someone would be willing to duplicate this test (possibly with a comparable system), I'd appreciate it.

[Kedirekin]

I remember, the first thing I did when I got my MP system was try out TMpg. It did indeed use 100% of both processors. I vaguely recall someone mentioning that it has a very good prefetch buffer.

I also remember that TMpg was almost 4 times as fast on my 1 GHz dually than it was on my 500 MHz single processor machine, so the scaling was practically linear across the two processors.

However, even though it uses both processors to their maximum extent, it's still quite a bit slower than CCE. At the time, I was getting 1/4 real time with CCE on the dually, while TMpg was only getting 1/7 real time. And since then, mpg2dec.dll has come along and I've gotten as high as 1.25 real time with CCE.

[Arky]

60% CPU usage sounds to me like Cinemacraft is using most of one CPU, and the OS is allocating other background threads to the other CPU. I am unconvinced that Cinemacraft SP is Multithreaded - as far as I can tell, from the Cinemacraft website, only the Cinemacraft PRO, hardware-assisted version can usefully employ SMP.

If anyone knows otherwise, please let me know, since I am on the verge of building a video PC and would welcome any information on this issue.


Arky ;o)

[da franksta]

60% CPU usage sounds to me like Cinemacraft is using most of one CPU, and the OS is allocating other background threads to the other CPU
this is not the case. read Kedirekin's first post, he explained what's happening.
I am on the verge of building a video PC and would welcome any information on this issue
if you're planning on frameserving, this will be the bottleneck using only one cpu (=50%). CCE can keep up with avisynth easily, using another 10-15%. you gain some speed in smp, but not a lot.
if you're not planning to frameserve, you will gain plenty building a decent smp system.

Da Franksta (i can't believe you dug up my old post :) )