WMV9 VCM Encoding and HyperThreading

[Kermee]

Please, no flames on selecting WMV9 VCM as my video format.

I currently archive my HDTV MPEG-2 source materials to WMV9 VCM + AC3 and wrap it in an AVI. I use DVD2AVI and AVISynth to frame-serve into VirtualDubMod and use WMV9 VCM 2-pass VBR to do my video.

Currently, I'm on a P4-2.53 GHz/533 MHz FSB machine and it takes quite a while to do my encodes. I usually process at 1920x1080p/24, 1920x816p/24 and 1280x720p/24. Then mux my AC3 and videostream into an AVI.

My question is moving up to a HyperThreading processor (e.g. 3.06 HT) help speed up the encoding process? I've looked everywhere for information regarding WMV9 and encoding and I see SMP support, but nothing noting anything about it using HyperThreading.

Thanks in advance.

[trbarry]

Dunno.

But beware that most of the PCHD cards are based upon the Janus chips and drivers which have trouble with hyperthreading turned on.

- Tom

[JJ]

hi,

AFAIK Pentium 4 processors have only 1 floating point unit, so the results might be below expectations. I haven't tested it myself though, so other replies will probably be more competent.

JJ

[Beastie Boy]

Kermee, please excuse this slightly off topic post.

I am interested to see that you are using WMV9 for your encodes as I am also coming to the conclusion that this codec gives far superior results to DivX for hi-res/high bitrate encodes.
What bit-rate are you using for these (very) hi-res encodes and have you compared to other codecs?

As regards hyper-threading Anandtech (http://www.anandtech.com) use DivX to test new processors so this could give you a good indication of MPEG4 encoding speeds.

Cheers, Beastie.

[echooff]

Can't really answer specifically, but my encode times were more than cut in half when I momeved from a p4 2.4 to a p4 2.8 dual hyperthreading.

[Atamido]

Anandtech's test looks like it shows at least a 10% increase in speed of DivX encoding when using HyperThreading.

@Kermee: His test directly compares a Pentium 4 2.53GHz with a Pentium 4-HT 3.06GHz. From the article: We'll start off with a "quick" conversion of a DVD rip (more specifically, Chapter 40 from the Star Wars Episode I DVD) to a DiVX MPEG-4 file. We used the latest DiVX codec (5.02) in conjunction with Xmpeg 4.5 to perform the encoding at 720 x 480. We set the encoding speed to Fastest, disabled audio processing and left all of the remaining settings on their defaults. We recorded the last frame rate given during the encoding process as the progress bar hit 100%. The results:

Pentium 4 2.53GHz = 67.9fps
Pentium 4-HT 3.06GHz = 86.6fps

Thats about a 27% inprovement in processing speed. However, resolution, original decoder, audio encoding, and every other factor is going to make a difference here.

[temporance]

It is very difficult to do efficient multithreading with a VCM codec. Because of the VCM API, there is no way to work with multiple frames in parallel, so parallelism must be implemented at sub-frame level.

At an encode rate of 25 fps, the encoder is working on each frame for less than 40 milliseconds. That is not really enough time to farm out load efficiently to each [logical] processor, for them to complete their work, synchronize again and then complete any non-parallel work in a critical section.

Besides, it's very difficult to break the algorithm down to things that can be done in parallel. I'd bet that the overhead of multithreading outweights the benefits of being able to use an extra logical processor.

[Kermee]

Thanks for all the replies.

@trbarry: I also wanted to personally thank you, Tom (trbarry), for the UnDot() function. It's absolutely wonderful and I use it heavily. It was definitely an awesome contribution to the community.

It looks like I will probably upgrade to a P4-3.06 HT and hopefully it will help a lot.

@Beastie Boy: I primarily do two resolutions at this point. For 1280x720p, I usually pick between 4.4 Mbit and 6 Mbit/sec. It depends on how much of a 'visual' feel if at 4.4 Mbit VBR if there are enough scenes which I know WMV9 will assign low-bitrates or not. Anything that's fast-action (e.g. flashing gun muzzles, or sports), I go straight to 6 Mbit VBR. For 1920x816p, I go between 6 Mbit and 6.8 Mbit. Most of the materials I have are from FILM sources and usually IVTC on the 1080i/29.97 or do some sort of SelectEven().Decimate(cycle=1) on 720/59.94 to get back to 23.976.

Looks like I'm stick to a single CPU situation for now and upgrade to a 3.06 HT. Thanks again everyone.

[trbarry]

For 1920x816p, I go between 6 Mbit and 6.8 Mbit.

Generally I have found very few (if any) HDTV sources that seem to have an effective horizontal resolution greater than about 1440 or so. One test of whether to bother trying keeping more is to just downsize (LanczosResize) a clip from 1920 to 1440 and in the same script upsize it right back again. If you can't tell the difference when compared to the original then there is really nothing to lose by encoding it at only 1440 or even 1280 wide.

- Tom

[Atamido]

@trbarry: Would it be effective to do the two resizes and do a diff against each other to see if there is any difference? Or is it better to compare the actual images?

[Neo Neko]

Well a Diff can be effective for finding the exact differences. But it is really rather inappropriate when testing for visual perception. I did a test exactly like what tbarry said some time ago to put to rest the myth that PAL DVD are any better than NTSC DVD simply because PAL is 578 and not 480. I used Lanczos resize and started with a PAL rez frame. Scaled it down to NTSC. Then back to PAL. I really could not notice any percievable differnece. And anything I did notince I would not stand a chance of noticing at 24fps or greater. I did also do a diff. But I had to do some contrast stretching to even make the diff easily visible.

[trbarry]

@trbarry: Would it be effective to do the two resizes and do a diff against each other to see if there is any difference? Or is it better to compare the actual images?

Pamel -

For this I mostly go with the numbers, though it can confuse detail with noise a bit. I made an Avisynth script to do this using Compare() that shows both the numbers and side by side HDTV video awhile back. See my old Let's measure detail (http://www.avsforum.com/avs-vb/showthread.php?s=&threadid=242229) thread over at AVSForum.

- Tom

edit: It would be interesting to redo this with SSIM or whatever that new quality based compare thing is. Again, the idea would be that if you don't lose any quality then don't bother to keep the bits.

[sh0dan]

Regarding HyperThreading you might very well see a slight increase in encoding speed. The general gain is usually around 15-20% compared to same speed without HT.

It does however depend on the codec and the filters you use. Interestingly theory deducts that the more optimized the code you are using is, the less gain you'll have from HT. Needless to say AviSynth is very optimized. :D

Hardwarereview.net has some benchmarks (http://www.hardwarereview.net/Reviews/Intel%20Pentium4%203060/Pentium4-3060.htm) of "realistic" settings using AviSynth/DivX - however they don't show what their realistic settings are.

@trbarry: There are some interesting observations in your thread!

[Wilbert]

This one (http://www.hardwarereview.net/Reviews/AMD%20Athlon%20XP3200/XP3200.htm) gives more information. One test without any avs-filters where PIV+HT outperforms everything, and one with a neutral bicubic resize filter where the PIV-Athlon differences are much closer.

The same tests with PIV-Athlon64 (http://www.hardwarereview.net/Reviews/AMD%20Athlon%2064%20FX51/Athlon64FX51.htm#Divx Encoding), where Athlon64 is the winner when applying a resizer.

[Sharktooth]

Well, its a matter of time.
When codecs will be optimized for x86-64 there will be a performance leap.
DivX network has a 64bit version of DrDivX ready and they say it is damn FAST.
I cant wait to have one of those 64bit mofos in my rig:)