XviD Athlon Optimized PLUS SOURCES

[eLeKtriK EyE]

Frontier released the binary along with the so wanted sources (whose absence led to locking the previous thread).
Get them at the following URL:
http://net314.myphone.gr/xvid_amd.html

[Enigmax]

Yes :) , this released increases the speed 5 - 10 %.
Good Work.

Greetings

[Selur]

little Question:

Will/can these optimisations be included in dev4 ?

Cu Selur

[Eyecannon]

Isn't it kinda dumb to use the stable binaries, since they are pretty old?

[TheXung]

w00t

[kxy]

Is this safe to use? Has anyone test it for the whole clip?

[Frontier]

Yes it is safe to use for everyday use.

[Lobuz]

@Frontier
What IDCT is used? Simple or Walken?
Maybe You can build that optimisations into that exact development Koepi's version. I'm sure He will send You those sources.

Regards
Lobuz

[sh0dan]

The only changes are that AMD's optimized memory copy routines are being used instead of memcpy (default, slow C-version). The actual version used is uManaics Alpha (unstable), with the sources from this page (http://umaniac.leffe.dnsalias.com/alpha/alpha.html).

Other platforms should also get a gain from this. And yes - it should be quite easy to integrate as a part of future XviD's. What's really amazing is how much an improvement it is. It does however require Integer SSE, so it will not work on P2 / Early Celeron.

The legal status of this code is however a bit "funny", as it is provided as free download by AMD. It is copyrighted AMD, but that doesn't mean that it is automatically non-useable for GPL. They do not disallow redistribution, but doesn't grant it either. I guess the best method to find out would be to contact the e-mail in the code. (Even though I cannot imagine why they wouldn't allow it).

[TheXung]

Oddly enough. these AMD memory routines have been discussed before here http://www.xvid.org/modules.php?op=modload&name=phpBB2&file=viewtopic&t=90
and had been dismissed by isibaar. shows ya just how long I've been following this project

[drebel]

Any PSNR tests using this version?
If one wants to use trellis ,I think he could select it from another unstable version and then change the .dll without hitting "load defaults" button.As Frontier said, the code for it exists

[visigotik]

Originally posted by TheXung
Oddly enough. these AMD memory routines have been discussed before here http://www.xvid.org/modules.php?op=modload&name=phpBB2&file=viewtopic&t=90
and had been dismissed by isibaar. shows ya just how long I've been following this project

So is this a GPL issue? the AMD sources (http://www.amd.com/us-en/Processors/DevelopWithAMD/0,,30_2252_2272_2274,00.html) themselves only have a simple copyright note, they don't mention any licence and at AMDs site the licence page (http://www.amd.com/us-en/Processors/DevelopWithAMD/0,,30_2252_2272_2274^2937,00.html) appears to apply to "AMD Athlon(tm) Software Developers' Kit ("SDK")".

[sh0dan]

I don't see any real problem - somebody should just post a mail to AMD and ask - and the problem is solved with 99% probability - I've just had too many beers for it now. ;)

[TheXung]

Originally posted by drebel
Any PSNR tests using this version?
If one wants to use trellis ,I think he could select it from another unstable version and then change the .dll without hitting "load defaults" button.As Frontier said, the code for it exists

ok, ok, maybe there's a misunderstanding about this occuring from the translation, but just let me explain in english.

1. All official branches or commits to the code are available from the CVS server. Visit the xvid homepage on details to access it.

2. When frontier says this is a mod of koepi's build with umaniacs patch, what he really meant is that this is modded from the official latest dev-api-3 branch which I assume he got from downloading it at umaniacs site. He didn't do a modication of koepi's binary.

3. Just to make this clear, this is dev-api-3.

4. There are unofficial additions to the code. Trellis for dev-api-3 being one of them. Trellis is basically only in dev-api-4. Syskin had backported it to dev-api-3 so koepi could distribute a dev-api-3 version with it. It's not actually in the code from CVS. Koepi is the only one who has this code, along with people who asked for it.

[TheXung]

Someone really should do a comparison of this build and umaniac's latest build with the same settings. They should be the exact same output file.

[Koepi]

This build has just all memcpy()'s replaced by memcpy_amd().

A memcpy can't produce artefacts - as long as it's correct, and it is.

We're working on something without using external sources for dev-api-4, so no need to use otherwise copyrighted stuff.

Regards
Koepi

[cipher]

Just a little feedback ---- with skal's new algorithm for fdct, dev-api-4(don't ask me how/where I got the binary:D) is, on my machine(athlon xp 2600+), approximately 13-16% faster than the abovemetioned amd-optimized binary. And, I agree with Keopi that his,Nic's and uManiac's binaries are quite optimized for AMD. If I remember correctly, their builds had ~0.5 higher PSNR,which is pretty huge difference, when encoding on an AMD computer than an Intel one, of course aiming the same filesize.:)

btw, this amd-opt. binary gives brown-yellow-like bars on the left side of the picture.(Sorry, it seems like all I report is kinda bad news.:devil:But I'm telling the truth ;))

I didn't compare it to Keopi's latest, coz you guys can do it(or maybe have done it) and Keopi's latest is really very stable so wouldn't have too much variations over different computers, yeah I'm too lazy.:D

bye

edit:
damn I forgot to put the settings, alright, here we go:D:
1st-pass
Quant. Type h.263
Motion Search Prec'n -6
vhq - 4
Use Chroma Motion
Chroma Optimizer
Max I interval -240
B frames: 2/1.50/0.75

[Koepi]

Ok, I did my own tests now just to see what this is about finally.

Using dev-api-4, replaced every standard memcpy with memcpy_amd.
Using GMC, QPel, VHQ4, MPEG quant...

The speed increase is *caugh* overwhelming: I got 0.1-0.5% more speed! (If at all.) This makes up for max. 10 minutes on an 8hrs encode. Everyone make up his/her own mind about these numbers.

Regards
Koepi

[Frontier]

Originally posted by Koepi
Ok, I did my own tests now just to see what this is about finally.

Using dev-api-4, replaced every standard memcpy with memcpy_amd.
Using GMC, QPel, VHQ4, MPEG quant...

The speed increase is *caugh* overwhelming: I got 0.1-0.5% more speed! (If at all.) This makes up for max. 10 minutes on an 8hrs encode. Everyone make up his/her own mind about these numbers.

Regards
Koepi

The above mentioned memcpy replacement for amd, it is just designed to perform by bypassing the cache of the l2 stream, thus giving better results. For most AMD systems (XP/64 excluded), the speed difference lies within the limits koepi described; however on higher freq buses (400MHz+) and the 64 archs, the benefits are more than these (on my 333MHz XP 2600+ I get 3-6 fps more than the latest unstable koepi release). The situation is getting even better when I compile the sources against AMD math libraries (proprietary, not publicaly released).

@cipher: the problem you mention does not relate to the optimized memcpy routine, but to the actual XviD code.

I'm also doing some tests regarding building against Intel math libraries (with SSE2 support).

[Koepi]

Frontier:

try using the amd memcpy on the sources i sent to you - take a build with usual memcpy, do a test, and then the optimized one. I use different quality flags in that build which didn't make it into CVS out of unknown reasons.

Then we could see the impact it has - with the _same_ quality.

Varying different parameters at the same time will bias the results - that's why i used my current dev-api-4 test to compare against. Else you'd compare things which aren't comparable.

It would be nice if you could do that - I'm busy right now and can't do that tests for at least another day.

Best regards
Koepi

[TheXung]

The speed comparisons with Koepi's last unstable build I believe are invalid because there have been more changes to dev-api-3 than just those two bug fixes that umaniac reports. I honestly do recall syskin making a change to the i/b/p decision after koepi's last build, which accounts for filesize and speed differences. Everyone is in essence comparing this build to an older version of dev-api-3.

Again, this build should be compared with umaniac's latest build.

[Frontier]

Currently I can't do more tests, since I do not have my development system here. I expect my new computer next week, when I'll try to do some tests. Right now I'm focusing on the latest *OFFICIAL* 0.9.2 stable release and I'll apply the memory optimizations on it first.

I can assure you that I'm using my build for over two weeks now, have encoded over 100 movies and no problems exhibited whatsoever.

[Koepi]

Well, to put the final results in here, head over to this mail by GomGom:

http://list.xvid.org/pipermail/xvid-devel/2003-November/003627.html

...who did quite a lot of coding and testing for that matter. but now the results are finally clear.

regards
Koepi

[mikeson]

Originally posted by Frontier
I can assure you that I'm using my build for over two weeks now, have encoded over 100 movies and no problems exhibited whatsoever.
100 movies in two weeks? What's your system?! And you've ever got time to watch them all? Maybe I'm missing a point... :)

[Frontier]

Originally posted by mikeson
100 movies in two weeks? What's your system?! And you've ever got time to watch them all? Maybe I'm missing a point... :)

No, I haven't watched them all, not even the 10% of them. On home I have an Athlon XP 2800+ which on average encodes 10 movies/day (it operates all the time) and on work I have two more AXP 2600+, two P4 2.4GHz and a couple of Celerons. On all Athlons I use my codec and during that time (two weeks) encodes movies without a glitch.

[TheXung]

OK, I finally got around to doing some tests and you guys are delusional or have problems doing a controled test.

1/4 pel, chroma motion, 2 b-frames @ 150:100, vhq-4

clip at 608x256 1000 frames:
Frontier: 1:08 minutes
uManiac: 1:08 minutes

same clip at 612x260 1000 frames:
Frontier: 1:22 minutes
uManiac: 1:19 minutes

my system:
Athlon XP at 11*200 = 2.2 ghz

[Koepi]

I don't see a difference to GomGom's or my test results there?

Regards
Koepi

[Aktan]

I'm confused. What's wrong with the fact that the increase is tiny? If the optimizations do not change the image quilty but only affect the speed, then who cares its a little increase? Every little bit counts. So what if it only save one second. Frontier already did the "work." I just don't understand why it seems the devs don't like this optimization at all.

[kxy]

for me it average 1 sec gain per 1000 frame, I did 5 full length movie with this test.

quarter pel, chroma motion, 3 b-frames @ 150:100, vhq-4

[BoNz1]

I'm confused. What's wrong with the fact that the increase is tiny? If the optimizations do not change the image quilty but only affect the speed, then who cares its a little increase? Every little bit counts. So what if it only save one second. Frontier already did the "work." I just don't understand why it seems the devs don't like this optimization at all.

Also, the fact that they are AMD specific doesn't make too much sense either, I think the feeling from the XviD developers was that if there are things that needed to be speeded up it is better to optimize the code for more platforms ie generic mmx rather than have special memcopy code that can only be used on AMD. BTW I think enough tests have been done now to show that it isn't helpful so arguing that it is by saying every second counts isn't going to convince anyone. Sorry thats just the way it is. Anyway, dev-api-4 is very fast compared to api 3 so I wouldn't worry about speed at all. A great deal of effort has been put in to make sure it is very fast.
EDIT: Sorry, I'm not totally sure about the amd k6 thing.

[ominte]

Also, the fact that they are AMD specific doesn't make too much sense either, I think the feeling from the XviD developers was that if there are things that needed to be speeded up it is better to optimize the code for more platforms ie generic mmx rather than have special memcopy code that can only be used on AMD. BTW I think enough tests have been done now to show that it isn't helpful so arguing that it is by saying every second counts isn't going to convince anyone. Sorry thats just the way it is. Anyway, dev-api-4 is very fast compared to api 3 so I wouldn't worry about speed at all. A great deal of effort has been put in to make sure it is very fast. Well I'm sorry BoNz1 but your comments don't seem to reflect the wishes of the XviD developers. Personally I cannot state what their aims are regarding optimisations but can you explain why they formerly placed a pole on their webpage asking which cpu everyone used. It is most likely that the XviD developers were trying to gauge which systems should be focused on for optimizations. I do not believe they have a fear of specifically optimizing for a platform.

Also, whilst earlier claims about the extent of the speed increases have now been disproven by GomGom this does not dispel the fact that their are improvements. kxy makes a good point, if the optimizations have been implemented and tested then it hardly seems appropriate to leave them out of XviD merely due to spite or laziness.

[celtic_druid]

Originally posted by Edouard Gomez
Hmmm, my code is not the AMD one, that is what the doom9 guy did. His
hack was 1/ ugly 2/ not acceptable in XviD because of the copyright
issues.
from the XviD mailing list as per Koepi's link. I would suggest that you go and at least read it before making comments on the issue.