VC1\Expression Encoder 2 SP1 - Limited to only 4 threads?

[Furiousflea]

Hi there as title really, doing a bit more work in expression encoder after upgrading to a Core i7 setup and it looks like the encoder itself can only use 4 threads!?

Am I missing something?

So basically I've got to disable HT in the BIOS everytime I want to encode something with it?:mad:

Thanks, Rob

[benwaggoner]

Yes, this version of the VC-1 codec itself is 4-way threaded. Note the Win 7 version if 8-way threaded, so this isn't an intrinsic limitation.

That said, disabling HT shouldn't give you a different end-to-end point. The OS should naturally get those 4 thread on the 4 real cores. If you're seeing a big speedup with HT off, that's a bug and I'd like to hear some details.

Note that even though the VC-1 encoder SDK is limited to 4 threads, if you're doing multiple bitrate encoding, each bitrate can use 4 threads, and source decode, preprocessing, and audio also happen on their own threads.

[Furiousflea]

Yes, this version of the VC-1 codec itself is 4-way threaded. Note the Win 7 version if 8-way threaded, so this isn't an intrinsic limitation.

That said, disabling HT shouldn't give you a different end-to-end point. The OS should naturally get those 4 thread on the 4 real cores. If you're seeing a big speedup with HT off, that's a bug and I'd like to hear some details.

Note that even though the VC-1 encoder SDK is limited to 4 threads, if you're doing multiple bitrate encoding, each bitrate can use 4 threads, and source decode, preprocessing, and audio also happen on their own threads.

Thanks for your response.

Well, there isn't that much to say. I have threads set to "auto" in Expression Encoder 2 SP1 and with HT on I get a CPU utilization of around 45-55%. With HT off I get a utilization of around 80-85%, both fluctuate a lot constantly in terms of CPU usage around those numbers roughly.

I'm only talking in terms of 2nd pass here and this is with ZERO extra processing, source is an m2ts being decoded with ffdshow.


I noticed that with my old setup, where you helped me a lot I was getting a similar CPU utilization, but understood that apparently I should be getting around 100% on the 2nd pass...I never achieved this though.

I'm on Vista x64

Thanks for your time.

[Leak]

I have threads set to "auto" in Expression Encoder 2 SP1 and with HT on I get a CPU utilization of around 45-55%. With HT off I get a utilization of around 80-85%, both fluctuate a lot constantly in terms of CPU usage around those numbers roughly.
So with HT on it utilizes more CPU - isn't that what you wanted?

(45-55% of 8 logical CPUs is more then 80-85% of 4 CPUs in my book...)

np: Neil Landstrumm - Mask Of Musk (Lord For £39)

[benwaggoner]

Leak's got it right. HT will never get you 100% utilization, since it's not a real extra core. As long as you get > 50% with HT on, than you're getting more processor utilization than HT off at 100%. Of course there's some extra overhead from having to do more threads, but it sounds like you're in the "worth it" range.

The best test is to measure the time an actual encode takes each way. Sounds like having HT on should be 10-20% faster, which is a pretty typical real-world improvement for compression. That said, I haven't actually run EEv2 on a HT machine...

[Furiousflea]

Leak's got it right. HT will never get you 100% utilization, since it's not a real extra core. As long as you get > 50% with HT on, than you're getting more processor utilization than HT off at 100%. Of course there's some extra overhead from having to do more threads, but it sounds like you're in the "worth it" range.

The best test is to measure the time an actual encode takes each way. Sounds like having HT on should be 10-20% faster, which is a pretty typical real-world improvement for compression. That said, I haven't actually run EEv2 on a HT machine...

No because those 8 cpu's aren't really 8 cpus being detected in windows task manager performance tab. Assuming 100% CPU usage is possible without HT on (4 cpus showing in task manager) then your getting around 60% of that. Instead of 85%+ with HT off.

HT doesn't magically double the amount of CPUs in the system which it would have to do to make both your theories possible.

It basically chops the power (not quite, but nearly hence the gains with HT on for optimised programs) in half and doubles the number from my experience...

I could be wrong... :o but I have tested it on\off and it is a lot slower on.

[Furiousflea]

Leak's got it right. HT will never get you 100% utilization, since it's not a real extra core. As long as you get > 50% with HT on, than you're getting more processor utilization than HT off at 100%. Of course there's some extra overhead from having to do more threads, but it sounds like you're in the "worth it" range.

The best test is to measure the time an actual encode takes each way. Sounds like having HT on should be 10-20% faster, which is a pretty typical real-world improvement for compression. That said, I haven't actually run EEv2 on a HT machine...

This simply isn't true from my experience. Take programs like Prime95 that stress your CPU as much as possible, they max out all 8 CPUs showing in task manager to 100%. But you're not getting (nearly) double the processing speed of 4 genuine cores...?

Take encoding with x264, that will give you 100% on all 8 cores in task manager with HT on. But with it off and 4 cores showing at 100% you only lose 5-15% in encoded fps.

[benwaggoner]

This simply isn't true from my experience. Take programs like Prime95 that stress your CPU as much as possible, they max out all 8 CPUs showing in task manager to 100%. But you're not getting (nearly) double the processing speed of 4 genuine cores...?

Take encoding with x264, that will give you 100% on all 8 cores in task manager with HT on. But with it off and 4 cores showing at 100% you only lose 5-15% in encoded fps.
It's been a lot time since I did any encoding on a HT machine, but I thought that it wouldn't show 100% CPU when it was memory bound or whatever. But it was ages ago (XP?) I could be just misremembering.

In the case of EE itself, I wonder if we should look at taking advantage of CPU affinitization in the HT case, as supported by the vC-1 Encoder SDK. That would let us specify that the encoding process is locked to the four real cores. The OS should do that automatically. Vista/2008 and Win 7 are a lot better at NUMA thread management, of course, as they know to combine tasks accessing the same memory to get onto the cores on the socket the meomry is directly attached to.

[Leak]

It's been a lot time since I did any encoding on a HT machine, but I thought that it wouldn't show 100% CPU when it was memory bound or whatever. But it was ages ago (XP?) I could be just misremembering.
Of course simply comparing total encoding time with and without HT instead of just CPU utilization would make it blindingly obvious if HyperThreading is a help or a hindrance here, but I guess that would be too easy... :D

np: Autechre - Iera (Untilted)