why is Xvid so slow

[Pulp Catalyst]

been using AutoGK and Fairuse Waizard,

my spec is Athlon X2 5200+ (SSE1/2 and 3) dual core 5.2GHz
windows xp 64bit

Xvid 32bit 1.1.3
and Xvid 1.2 64Bit (only a 4% increase in speed which i felt was weird)

2gb DDR2 mem,
ect.

my question is why is Xvid not as fast as i thought would be with such a pwerful Rig,

for Example, i encoded a film using Nero Recode to Nero Digital AVC codec using AthlonXP a long time ago, and it was crawling at something like 45FPS, where as Xvid shot along at 60FPS (1st pass)

2nd pass Xvid was like 20FPS, and nero AVC was 3 FPS!!!!!

did the same thing with my Rig now, and Xvid 1st pass 105FPS and 25FPS on second pass on higest possible setting,

nero recode AVC 296FPS and 96FPS on second pass,



why is Nero taking so much advantage of my new hard ware,
but Xvid doesn't seem to take as much,

i know Xvid does take advantage of SSE2, but does it take Advantage of SSE3 as this would explain a lot, also i couldn't understand why nero uses 89% of total CPU performance, but Xvid only uses 62%,

i'm hoping someone can help me, as i've tried looking around, and from some of the info i've seen, my Xvid encoding should be going faster then it is,

and to have AVC encoding going faster then ASP encoding does not make sence,

to my knowledge AVC encoding is far more complex then ASP,

my only guess was, that both of my CPU cores are not being exploited properly, and maybe Xvid does not Support SSE3 instruction set, and looking at the speed increase from 32bit to 64bit capability 4% doesn't seem right, i know Xvid support has not been around for a while, but i was hoping that this is not the case that would explain the speed problem i'm getting and perhaps some of you more professional people could help, or at least share some light on the problem,

as for the life of me, i can't understand how AVC encoding would be Faster then Xvid (ASP) unless the Xvid is not taking advantage of some of the more advance features found in dual core CPU's today.

summary,
1. Xvid is slower then nero AVC why?
2. Xvid only using 62% of CPU NERO Recode uses 89% why?
3. Xvid 32bit and Xvid 64bit only 4% speed increase why?
4. is Xvid missing SSE3 instruction support?

i hope somebody could help me resolve these issues, i've seen many articles about Xvid not utilizing both core to there full potential, but 62% is redicolous,

i await your experise on this, as i can not find a answer, oh, the xvid codecs are,

Koepi xvid 1.1.3
http://ffdshow.faireal.net/mirror/XviD/

took me ages, nearly 4 hours to find a 64bit Xvid codec that actually worked, but all for a silly 4% increase, it was like a april fools joke on me really, as i honestly thought that was the reason why i was loosing so much speed, but i was so wrong, and very disappointed,

i was gutted, gutted,

[Taurus]

AutoGk uses a "special version" of the Xvid Codec compiled by CelticDruid.
You have to use the one bundled with the AutoGk installer or if you want to update, only take the one from the CelticDruids websides http://esby.free.fr/CelticDruid/mirror/XviD/
And the slowdown you mentioned is not because of the Xvid codec, but the sometimes heavy Avisynth scripting AutoGk performs.
So as long as you don't know the facts, don't blame anything or anyone.

[Pulp Catalyst]

how come when i'm asking questions, people here are taking it as though i'm putting blame, or speaking facts,

i'm just offering my findings and hoping that some one can explain them to me,

not for a second am i putting blame of any kind, i assumed this was a forum to bring together people's findings and try to help one another,

there is no blame to blame, i'm just writing everything that has happened to me, and wondering why this is so,

i don't have the knowledge nor the experience to find or know this myself, so i thought i would try and find out here,

it feels like to me that some are being very defensive about AutoGK, and Xvid, why this is so, i have no idea,

you said "So as long as you don't know the facts, don't blame anything or anyone."

but where in my questions did i put blame or facts, i'm telling you what has happened on my system, and suddenly your now being defensive, for reasons i don't know.

and you said Avisynth scripting is to heavy, because of autoGK, i see, well, i post this in Xvid section, not AutoGK, and the reason that is, because all my encoding applications do the same, i've tried 4, and each one does the same, which the only thing they have in common, you guessed it, XVID, which is why i posted this topic, in....you guessed it, Xvid subject,

but to be honest with you, forget it, your to defensive for my liking, haven't used this forum in a while, and i can see why,

people are to defensive, since when is asking questions, stating facts, not once have i declared that this is a fact, posting ones experience is never a fact,

thanks anyway, besides i've been getting some nice help from another forum, who which i posted exactly the same as here, but nicer responses, or more to the point, friendly responses

apparently SSE3 is not supported in Xvid so i found out, which i was told can be seen in the Xvid encoder control panel,

and i also found out some more info concerning how Xvid pipes, which also explains a lot where some of the speed is lost,

[[P]ako]

There was a MP version of Xvid (for multithreads/multiprocessors). You ought to give it a spin. I am not sure whether it was good using multicore/mp systems as x264 is.

[Taurus]

@Pulp Catalyst
You got me totally wrong.
I wanted to help you out.
If you are offended by the way people are talking in this forum: well, the questions you were asking are around here for years.
Nobody knows about your backgrounds nor your encoding skills.
So, if you want to have comparable results:
Feed some small samples via a short Avisynth script into your favourite encoding gui or use a command line encoder like xvidencraw.exe
or x264.exe.
And I don't think Xvid is slow.
Try other encoders on the market place and you see why :)

Cheers

Taurus

[CruNcher]

1. Because you most probably set it up differently (XviD more complex then AVC, sure you can do that, posting your settings for both would have been nice)
2. Because Koepi xvid 1.1.3 is not MT ready
3. What do you expect from 64 bit ? (and don't forget the whole encoding chain has to be 64 bit from decoder to encoder)
4. If it doesn't say so it doesn't, look in the cpu tab

About the 1pass Speed, Nero Recode 2 (Atemes Encoder Core 1.3) uses a Fast Analysis Method that neither XviD nor X264 use yet (with long runtime source).


i know Xvid does take advantage of SSE2, but does it take Advantage of SSE3 as this would explain a lot

Sorry but this would explain nothing as this can't be the source of the Speed difference you expirience, please don't make claims of things you don't know something about.

[Dark Eiri]

nero recode AVC 296FPS and 96FPS on second pass,

This is insane o.o
You're using the fastest settings, right? Or encoding in a very low resolution.
NO WAY an X2 5200+ would get these speeds.

You should use XviD 1.2.0, always. It's optimized for multithreading. I'm getting ~80fps (1st pass) and ~50fps (2nd pass) for DVD resolution content, with a Core 2 Duo E4300 overclocked to 3 GHz. Not maxed out, though.

[TheRyuu]

@Pulp Catalyst

Your Athlon X2 is not 5.2ghz. It is really 2.6ghz per core. This does not = a total of 5.2ghz.

And xvid, IMO, is not slow. You can change the settings to make it as fast or as slow as you want it to be, but in any case, an xvid encode with the slowest settings will 100% of the time be faster then an AVC (x264) encode with slow settings as well.

Sure, you can make AVC faster then xvid but the quality will be sh!t if you looking to hit a small(er) file size.

Xvid...can be slow, your right on that one.
My encodes that I do noramlly run about 10-20fps, however this is mainly due to avisynth filters, not xvid. Without filters, the same encode can run anywhere between 80fps (first pass) and 30-40fps (second pass).

I'll run through your questions and answer them to the best of my ability:
1. Xvid is slower then nero AVC why?
Xvid can be slower, it entirely depends on the settings your using. You can of course make an AVC encode much slower then a xvid encode as well.

2. Xvid only using 62% of CPU NERO Recode uses 89% why?
First off, xvid's threading model isn't as good as x264's (this becomes apparent when using quad core, however the new version of xvid_encraw I think makes up for this a little by threading the caching of frames?, or something to do with frame serving) (I've never used Nero Record...maybe it has a better threading model as well?). Second, sometimes your avisynth script can cause this as well. You can try the MT version of ayisynth to try and get your CPU usage higher.
3. Xvid 32bit and Xvid 64bit only 4% speed increase why?
I have no idea. Maybe 64bit just isn't that good. Or it could be something with xvid, I honestly don't know.

4. is Xvid missing SSE3 instruction support?
Maybe. x264 doesn't make use of SSE3 instructions AFAIK as well. This is probably because the speed difference will be pretty much 0 when compared with SSE2. SSE3 wasn't that big of a step IIRC. And no, it wouldn't explain a lot since SSE3 wouldn't really make it that much faster (probably why it doesn't have it in the first place)

Take it for what it's worth. I'm no expert on Xvid or h264 (especially Nero Record, I use x264).

[blizard]

@Pulp Catalyst

Your Athlon X2 is not 5.2ghz. It is really 2.6ghz per core. This does not = a total of 5.2ghz.



Wizboy are correct that it depends on both your setting for each codecs decoder and encoder and how complex each frame are to compress within those parameters. AVC (H.264) make use of a different way to handle the same video, so it is not an easy task to compare Xvid (ASP) from simple speed.

Direct to OP:
I react also about how you misunderstand AMDs performance rating (PR). Not even Intel make use of clock speed on CPU to indicate performance level among different version of their own products as it is misleading. To find performance you will need visit hardware forum that focus on benchmarking system and with focus on CPU performance. There are plenty of test where Xvid or DivX have been used to illustrate differences in both system and between single or multithreaded application. Don't forget LAME mp3 (which also can found in MT version) and some other application.

The reason why Xvid64 aren't that good today is probably because it have not been in much development since multi threading came to desktop computer and here we have most 32 bit systems (all x86-64 based CPU can run 32 bit OS). You have to be aware that it exists two main branches of Xvid today: Xvid 1.1.3 (single) and Xvid 1.2.0 (multi thread - support). Both are in development and based for 32 bit which your Wind XP64 should run just fine. Version number are the key here to what kind of support you will see for multi threading on Xvid.

From this link to AMD compare (http://products.amd.com/en-us/DesktopCPUResult.aspx?f1=AMD+Athlon%e2%84%a2+64+X2+Dual-Core&f2=5200%2b&f3=2600&f4=1024&f5=AM2&f6=&f7=90nm+SOI&f8=&f9=2000&) you will see that an Athlon64 X2 5200+ have to core that each perform at the same clock speed of 2600 MHz or 2.6 GHz. Both core have a direct link to memory (RAM), so they have some limitation to access system resources (HD, video card etc) and will not give you a faster from pure clock speed, but will allow sharing resources (through multi threading) in a more fluid way. You can run transcoding task while you do other CPU demanding stuff and you system will still run without too much problem.

The main problem will now be how your harddisk perform and what kind of setup you use for harddisk; single or two; raid or non-raid system etc.


Crunsher:
You claim:

3. What do you expect from 64 bit ? (and don't forget the whole encoding chain has to be 64 bit from decoder to encoder)

I think your statement is incorrect as when you encode something in 64 bit you will not need a decoder that are also 64 bit to be able to play it later. In other words you will not create a video/audio that are pure 64 bit, but it must follow ISO MPEG-4 based specification either to Xvid or some other MPEG-4 ASP based profile. That is what make a video/audio able to be "portable" without a need for any specific "player" as long as video/audio follow those standard within MPEG-4 and those variation that are supported. The same goes for any other video/audio standard like MPEG-2 or MPEG-1.

The only time I know of a "decoding chain" will be needed are when 64 bit is in use for 64 bit based filter to do the job, then it must be able to handle 64 bit and player must be able to support those filter. But there is no reason that you can not mix 32/64 bit as far they don't make use of any call to kernel. Antivirus is a good example on one of those kind of mixes where a 32 bit GUI will be used, but must be able to connect with kernel in 64 bit mode. That is also a good explanation to why the transition to "pure" 64 bit have taken so long time. There is not much to gain by transforming or develop both 64 bit and 32 bit software when there is no need.

In the discussion above I suppose that we are talking about Windows XP x64 bit edition pro., an OS based on 64 bit kernel.

[Manao]

XviD doesn't use SSE3, but since it has almost nothing to gain from it, it's no big deal ( SSE3 = float operations, useless for video encoding, and 1 slightly faster load instruction that isn't faster on AMD anyway... ). x264 supports SSE3 and SSSE3, but there again, only SSSE3 does matter ( and according to akupenguin, was a bit of a disappointment ).

Since XviD hardly does any 64 bits computation, 64 bits helps only because it offers more registers. Those helps on C based code, and on specially written assembly functions. The problem is that XviD doesn't spend a lot of time in functions written in plain C, and that there are afaik no specially written 64 bits assembly functions. So a gain of 4% is imho not surprising at all.

I think your statement is incorrect as when you encode something in 64 bit you will not need a decoder that are also 64 bit to be able to play it laterOf course not, but you misunderstood him. When encoding something, you need first to decode the content. And its better if that decoder is 64 bits if you use a 64 bits encoder ( note : I think it's possible to use a 32 bits decoder through directshow together with a 64 bits encoder, but i'm not sure ).

Apart from that, the figures posted for Nero Recode seem very high. It would really help if you gave us more details : configuration, resolution, avisynth script...

[Episodio1]

I only see XviD 1.1.2 and 1.1.3 in Celtic Druids page.

I can't find 1.2 in Google either (the only links link to koepi.org) :(

[TheRyuu]

I only see XviD 1.1.2 and 1.1.3 in Celtic Druids page.

I can't find 1.2 in Google either (the only links link to koepi.org) :(

http://ffdshow.faireal.net/mirror/XviD/

Head MTK is latest 1.2 version compiled with the ICL compiler.

[Episodio1]

Thanks, wizboy.

I guess I have to download the latest "head MTK": 25-july. ^_^

[Sergey A. Sablin]

Of course not, but you misunderstood him. When encoding something, you need first to decode the content. And its better if that decoder is 64 bits if you use a 64 bits encoder ( note : I think it's possible to use a 32 bits decoder through directshow together with a 64 bits encoder, but i'm not sure ).

all threads and modules inside 64-bit process shall be 64-bit as well. there is no way to use 32-bit dlls inside 64-bit process.
same of course for 32-bit processes.

[foxyshadis]

You can do it with pipes, sockets, or shared memory, but that's about it. A true x64 optimized xvid could probably see at least 10% gain, but everything's so fast today that I suppose no one feels it's worth the trouble.

Don't forget to set threads=2 in your xvid 1.2, I'm pretty sure it defaults to 1.

[CruNcher]

Sure, you can make AVC faster then xvid but the quality will be sh!t if you looking to hit a small(er) file size.

That is wrong, even with faster (encoding) speed it can look better then XviD (more detailed, no ringing, no moving walls, less flickering, less blocks) if correctly tuned for a special purpose (compression scenario), at lower filesize.

[TheRyuu]

That is wrong, even with faster (encoding) speed it can look better then XviD (more detailed, no ringing, no moving walls, less flickering, less blocks) if correctly tuned for a special purpose (compression scenario), at lower filesize.

True, but I was giving like an "extreme" like the one he is probably having (290fps AVC encoding gonna be better quality then xvid? yea.... sure....)

[snowden]

Thanks, wizboy.

I guess I have to download the latest "head MTK": 25-july. ^_^

I'd also like to know if this is v1.2 of Xvid

I would also appreciate it if someone could tell me, do I need to configure the Xvid encoder settings in anyway to make use of the multithread capability (e.g. edit the 'Number of threads to '1' or '2' or something) or will Xvid do it automatically?

[Sharktooth]

if it has the threads option then it should be v1.2...

[snowden]

if it has the threads option then it should be v1.2...

yes ok, but do i have to change the number of threads option or not?