XviD 1.0.0 build by Jawor, codename "Schadenfreude"

[Jawor]

Get it here (http://www.mat.uni.torun.pl/~jawor/xvid).

This build was meant to be only for friends, but since its performance is really good, I've decided to make it available for everyone.

How to install:
After unzipping, right-click the file "xvid.inf" and choose "Install...". Voila!

How to unistall:
The package does not include an unistaller. To unistall the codec, just remove it from the codecs list in the Control Panel.

Disclaimer:
Please consider this build an experiment. If installing or using these binaries will cause any damage, don't blame me! Use it at your own risk. See "license.txt" for details.
The package does not contain a DirectShow decoder.

Currently I'm trying to optimize the binaries for specific architectures.

P.S. "Schadenfreude" is German for "malicious joy".

[TorgoGuy]

Apparently the difference between this and Koepi's is as your readme says "This is a bugfix release optimized for SPEED." In what ways have you optimized it? Different compiler settings?

Thanks!

[Jawor]

Not only the settings were different, the compiler was / is different. Koepi uses ICL, I use mingw32-gcc. I used
-O3 -fomit-frame-pointer -march=i686 -mcpu=i686
Now I'm going to build for specific CPUs using something like
-O3 -fomit-frame-pointer -march=pentium3 -mfpmath=sse
I suppose Koepi also optimized his build for speed in some way :D

[SoonUDie]

Jawor, any plans for a P4 optimized release?

[athos]

Jawor, not that it matters much, but -march=cpu-type implies -mcpu=cpu-type (or, in 3.4, -mtune=cpu-type).
Also, I thought that -mfpmath=sse was implied by -march=pentium3, but apparently it is not. I think that -msse is implied by -march=pentium3 though. I will try out -mfpmath=sse for my next p3 targetted release of ffshow.

Oh, and good idea with cpu-optimized release of xvid 1.0!

[CruNcher]

hmm Koepis build is well optimized the settings where long tested

[athos]

Originally posted by CruNcher
hmm Koepis build is well optimized the settings where long tested
indeed, but there is no harm in trying? also, there could be a few % performance to be gained by targetting specific architectures, i imagine that koepis is pretty general.

[Koepi]

Ok, now that you take the mouth that full i'd like to see some numbers, how mingw/gcc outperforms the optimized ICL build. (I remember that gcc still doesn't optimize as well as icl does, but maybe that changed - proof that wrong!)

I'm waiting :-)

Regards
Koepi

[PowerMacG4]

Originally posted by Koepi
Ok, now that you take the mouth that full i'd like to see some numbers, how mingw/gcc outperforms the optimized ICL build. (I remember that gcc still doesn't optimize as well as icl does, but maybe that changed - proof that wrong!)

I'm waiting :-)

Regards
Koepi

It probably won't outperform it without heavier optimizations. On my PPC7400 Apple's custom gcc "-fast" flag works wonders, producing code 5-15% faster than just -O3. It includes the following optimizations:

-O3
-funroll-loops
-fstrict-aliasing
-fsched-interblock
-falign-loops=16
-falign-jumps=16
-falign-functions=16
-falign-jumps-max-skip=15
-falign-loops-max-skip=15
-malign-natural
-ffast-math
-mdynamic-no-pic
-mpowerpc-gpopt
-force_cpusubtype_ALL
-fstrict-aliasing
-mtune=G5
-mcpu=G5
-mpowerpc64

Don't use the above optimizations for x86, but a good starting point would be this:

Compile a binary with only gcc -O3 and another binary with only gcc -Os. Compare the speeds with a bunch of tests, and whichever one is faster, keep.

Then you can add other options like -ffast-math and -fomit-frame-pointer and CPU-specific tunings.

try:
gcc -march=athlon-xp -O3 -fstrict-aliasing -ffast-math -ftracer --param max-gcse-passes=4 -malign-double -funroll-loops -pipe -fomit-frame-pointer -msse -mfpmath=sse,387

and see how that performs.

There is always a tradeoff between binary size and speed...

[d'Oursse]

i have made several test for RC4 with Koepi's build and mine's (compiled with gcc coming from wingw 3.1.0), with my flags (almost those given by PowerMacG4) and Koepi's build was always faster.

[PowerMacG4]

Originally posted by d'Oursse
i have made several test for RC4 with Koepi's build and mine's (compiled with gcc coming from wingw 3.1.0), with my flags (almost those given by PowerMacG4) and Koepi's build was always faster.

Yeah, gcc isn't really optimized for building win32 binaries at all. Microsoft's optimizing VC++ 7.1 compiler is pretty much free now, so you can use that. It generates code magnitudes smaller than gcc. It's too bad, because I like gcc.

[athos]

Originally posted by PowerMacG4
There is always a tradeoff between binary size and speed...
At the sizes we are talking about (a couple of megs) I'd go for speed all the way.

[SiXXGuNNZ]

edit: 35fps 1st pass, 18fps 2nd pass

MSP: 6 - Ultra High
VHQ: 4 - Wide Search
I-Frame: 240
Use Chroma Motion
Quants: 2-31
B-VOPs: 2/1.50/1.00
Packed Bitstream
Closed GOV
Custom MPEG Matrix(fox home/kill bill)

I didn't know what chroma optimizer was, so I had that off, I will use it in the future

system: amd 2800+(2.08 GHz), 512 pc 3200 ddr, asus nforce2 mobo

my script

LoadPlugin("C:\Program Files\AutoGK\filters\mpeg2dec3dg.dll")
LoadPlugin("C:\Program Files\AutoGK\filters\undot.dll")
LoadPlugin("C:\Program Files\AutoGK\filters\VSFilter.dll")
mpeg2source("D:\DVD Ripping\Kill.Bill.Volume.1.(2003).DVDRip.XviD-SxGz\KBV1.d2v")
#trim(startframe,endframe)
crop(4,64,712,352)
Undot()
VobSub("D:\DVD Ripping\Kill.Bill.Volume.1.(2003).DVDRip.XviD-SxGz\VTS_01_0")

I cropped away the black bars and did not resize, I notice I get a huge speed increase if I do not resize the encode, I wish the aspect ratio option was working for xvid, but for now I use zoom player to resize it to 16:9 during playback

one thing that would have been cool is, if I could set cartoon mode just for one zone, since there is a cartoon section of the film.

[Jawor]

Originally posted by SoonUDie
Jawor, any plans for a P4 optimized release?
Get it here (http://www.mat.uni.torun.pl/~jawor/xvid)
Originally posted by athos
Jawor, not that it matters much, but -march=cpu-type implies -mcpu=cpu-type (or, in 3.4, -mtune=cpu-type).
Also, I thought that -mfpmath=sse was implied by -march=pentium3, but apparently it is not. I think that -msse is implied by -march=pentium3 though.
It doesn't matter much indeed.
Originally posted by PowerMacG4
It probably won't outperform it without heavier optimizations
So I'll try to optimize my builds much heavier :D
Originally posted by PowerMacG4
It's too bad, because I like gcc
Yeah, I like it too. It doesn't require any commercial license ;)

[SeeMoreDigital]

Originally posted by SiXXGuNNZ
I cropped away the black bars and did not resize, I notice I get a huge speed increase if I do not resize the encode, I wish the aspect ratio option was working for xvid, but for now I use zoom player to resize it to 16:9 during playback. This option works fine, it all depends what player and what DS filter you're using.

If you mux the video stream into an MP4 container and have 3ivX's codec installed their directshow decoder will play back the encode at the correct AR even in Win Media Player.

That said it would be great if XviD's decoder was able to correctly AR anamorphic AVI's too.

There are other ways of forcing XviD encodes to use other decoder filters but this will involve changing the encodes 4CC and installing more codecs


Cheers

[bond]

Originally posted by SeeMoreDigital
If you mux the video stream into an MP4 container and have 3ivX's codec installed their directshow decoder will play back the encode at the correct AR even in Win Media Player.of course it does, in contrary to AVI the MP4 container was designed for working together with mpeg-4 video streams

That said it would be great if XviD's decoder was able to correctly AR anamorphic AVI's too.it would be even better if the xvid decoder would handle to anamorphic resize mp4 files :D

sorry couldnt resist ;)

[SeeMoreDigital]

Originally posted by bond
...it would be even better if the xvid decoder would handle to anamorphic resize mp4 files :D

sorry couldnt resist ;) I like that idea too. Are you going to add it to Koepi's XviD's suggestion box (http://forum.doom9.org/showthread.php?s=&threadid=76415) or shall I?


Cheers

[Jawor]

@PowerMacG4: I tried
gcc -march=pentium3 -O3 -fstrict-aliasing -ffast-math -ftracer --param max-gcse-passes=4 -malign-double -funroll-loops -pipe -fomit-frame-pointer -msse -mfpmath=sse,387
but unfortunately it didn't make any difference. Thanks anyway.

I compared -O3 with -Os and the first one gave me a faster binary.

It is my first gcc build for win32 (so far I used this compiler to build for Linux and Solaris), so had no idea what performance can I get. It seems to me that indeed gcc is not the best choice for win32 builds :(

[pogo stick]

I have P4 and I tested Koepi's and P4 Jawor's compils with the same 2 VirtualDubMod jobs (about 2,5 hours both jobs).
And Koepi's was a little faster. Up to 1 minute for each pass (VirtualDubMod job control doesn't show seconds).
Contents of first pass stat files and size of video files are exactly the same.
But contents of video files are different.
Strange, because I thought that different compils should produce exactly the same files.

[Jawor]

:stupid:
I just noticed that Koepi used a slightly different source code. Mine comes from CVS branch release-1_0_0 (13.05.2004). That could explain the difference :rolleyes:

Perhaps I should try with the source code from Koepi's page.

This P4 comparison (http://forum.doom9.org/showthread.php?s=&postid=497771#post497771) gave a little different result.

[pogo stick]

Originally posted by Jawor
This P4 comparison (http://forum.doom9.org/showthread.php?s=&postid=497771#post497771) gave a little different result.
Maybe it's because I didn't count seconds.
VirtualDubMod job control info gave me:
Koepi's
1 pass: 0:59
2 pass: 1:30
P4 Jawor's
1 pass: 1:00
2 pass: 1:31
But time difference is not that big to notice.

[twist3d]

ok, I did a "real life" 2-pass, 30min clip from "the cell" R2 DVD,using jawor's amd athlon optimized vs. koepi's 1.0 build.

specs: msi kt6v, athlon xp 2800+, 1024mb ddr333 and xp pro

dvd2avidg 1.30rc5 - idct: 32-bit sse mmx, field processing: none
avisynth 2.5.5a - .avs below:

LoadPlugin("d:\dvd2avi\mpeg2dec3dg.dll")
mpeg2source("C:\test.d2v")
crop(4,80,714,416)
Lanczos4Resize(640,256)


for encoding I used avs2avi 1.37 (commandline), high priority, quiet mode.

xvid settings were: 2-pass, q-type mpeg, adaptive quantization, qpel,
use chroma motion, turbo ;-) enabled, max i-frame 250, trellis enabled, target bitrate 1400. all other xvid options as defaults.

results:

koepi:
1st 14min 51s (50.78fps)
2nd 25min 02s (30.13fps)

jawor:
1st 14min 52s (50.71fps)
2nd 25min 17s (29.84fps)

so koepi's build is still king of the hill :)

[The Link]

so koepi's build is still king of the hill
That's probably true but I think that the results are too close to make absolute and clear "better than" statements (which you did not to my pleasure). The reason for my comment is that there seem to be many people out there who constantly claim that gcc is worse than msvc/intel/whatever. These people should take a closer look and make more specific/differentiated comments on this issue. All compilers have their strength and weakness and I think in this special case gcc could show some of its strength even more if one has in mind the different background of these compilers.

Above wasn't meant against people in this thread or the xvid forum (not at all!), it was just my personal opinion regarding statements I read more than once in several web boards (not only the doom9 forum).

Sorry for bothering!
The Link

[twist3d]

that the results are too close to make absolute and clear "better than" statements

yes exactly. I think that if any available x86 compiler was used, it couldn't give more than couple of seconds gain/loss. imho, if someone truly wants to make a "special build" they must optimize the actual xvid source code (which i think is already tuned to maximum :D )

[Neo Neko]

Originally posted by PowerMacG4
Yeah, gcc isn't really optimized for building win32 binaries at all.

I would not say that. It's performance is quite on par. As we have seen neither one is leaving the other in the dust. They are quite comparable in most instances.

Originally posted by PowerMacG4
Microsoft's optimizing VC++ 7.1 compiler is pretty much free now, so you can use that. It generates code magnitudes smaller than gcc. It's too bad, because I like gcc.

Actually that is not exactly true. Yes MSVC and their IDE tend to generate smaller binaries. In fact it can be a huge difference. But do you know why? It's not because it is a better compiler. ;) It is traditional Microsoft tactics. Using their install base to spin the results of their products. The reason MSVC binaries tend to be smaller is that they rely on large shared libraries that tend to come with all Windows installs. [msvcrtxx.dll IIRC] I am currently writing a basic wxWidgets app. It compiles and runs great with Mingw under Win32. But simple programs are easily 1Mb or more. That is because instead of loading the code in as a shared library it is actually being put in the binary itself. The lions share of the 1Mb size is the wxWidgets calls to the Windows interface. Now lets say I compile that library as a Windows shared library. It is still a little over 1Mb in size. But now my actual program binary is in the low tens of kilobytes 25Kb or so. Compiling the same program in MSVC with similar settings gives nearly identical sizes! So the question then becomes why don't those who write the default install targets for Win32 under GCC try to make large use of shared libraries to reduce binary size and save disk space like they do under Linux or other build targets? [I think the answer is that you can't rely on your consumers always having dependancies installed in the less educated Windows world.] In fact it should be possible to compile a program under GCC using the Windows shared libraries to get similar sized binaries. It's just currently alot more work. They are though both good compilers.

[ookzDVD]

On my machine, Xp 1700+, jawor's 2.5% faster than koepi's.

[saratoga]

I'd be amazed if gcc is faster for P4s. Until recently i don't think it actually had scheduleing for the P4 (i read somewhere that it used the old P3 code). Its supposidly better for the Athlon though.

Inte's compiler should be faster for both processor though.

[twist3d]

On my machine, Xp 1700+, jawor's 2.5% faster than koepi's.

how bout giving us resolution, clip length, settings used, avisynth script etc? if it's 2.5% faster on 5min clip, it doesn't matter much.

[athos]

Originally posted by twist3d
if it's 2.5% faster on 5min clip, it doesn't matter much.
Yes it does, if the speed increase holds for longer clips.

[CruNcher]

hmm i can't belive that long time members still think that minor flag changes or compiler changes can couse dramatic speedups, that's false and most tests here are flawed, because they include the IO in their test procedure and even only for the stats file this would be not a good method to test speedups in any kind also you had to completly and i mean really completly reset the memory as we all know windows likes to bork this up and that plays a key role here also.
Better way to test Speedups in my opinion is via the commandline encoder it is far more sufficient for that in my eyes and you can disable IO usage completly. That's how i optimize and test builds
and i do this not only with 1 run but with 3 for each profile setting
1 test consists of 3 profile settings running 3 times so 9 runs i think that's at least a little sufficient to see speedups.

[Zep]

Originally posted by CruNcher
most tests here are flawed, because they include the IO in their test procedure

exactly was I was about to post but you beat me to it :D

To help get IO out of the picture large rez from small files
test could be done. Try to get all the time spent on encoding
in the Xvid core and not disk IO -> MPEG2DEC3dg -> avisynth
trying to feed 50+ FPS to Xvid. Anything above like 20 FPS
is testing that stuff more than the codec.

IMHO HDTV 720p .ts (huge rez , small file) would be a good choice
(use a ram disk for the clip should help too)

[tiki4]

(Scratching my head.)

I think, if you really want to test the optimization of the C compilers against each other, you have to throw out all assembler stuff. To my understanding the C compiler does nothing with the parts that are written in assembler (and I guess most time-critical parts of XviD are written in assembler now).

tiki4

[Jawor]

Get the Athlon XP optimized build here (http://jawormat.republika.pl).

[twist3d]

Yes it does, if the speed increase holds for longer clips
yes I agree. but what I tried to explain was that all these tests should be done with 30min-2h encoding tests (and with various clips).
if I encode 1min clip and it encodes 10% faster doesn't mean a thing. I tested the athlon build and it is few seconds slower than koepi's, I'll test the XP build during weekend but I'm not expecting much as I said few posts back :D

[twist3d]

ok, so here are the latest test results using jawor's athlon xp optimized:

source:
matrix r2 pal dvd, chapters 1-8 ~30 min clip
who framed roger rabbit r2 pal dvd, chapters 1-5 ~30min clip

- tested using avs2avi 1.37, dvd2avidg 1.30 final, avisynth 2.5.5a
- same dvd2avi & avs2avi settings as before using "the cell"

.avs file this time for encodes:

LoadPlugin("C:\apps\dvd2avi\mpeg2dec3dg.dll")
mpeg2source("C:\test_1.d2v")
crop(4,80,712,416)
BicubicResize(640,256)

; for roger rabbit
crop(16,12,688,552)
BicubicResize(640,352)


- xvid settings, profile: everything disabled except q-type: mpeg, b-vops 2,1.50,1.0, closed gov, quantization settings: default, no trellis, motion: use chroma motion, max i-frame 250

all other settings as default.

koepi's build wins both tests.

jawor's athlon xp optimized "the matrix" :
1st 00:13:26.964 (51.45 fps)
2nd 00:20:01.288 (34.56 fps)

koepi's build "the matrix" :
1st 00:13:10.218 (52.54 fps)
2nd 00:19:22.488 (35.71 fps)

1st pass difference in 30min clip: ~17 sec
2nd pass difference in 30min clip: ~39 sec

jawor's athlon xp optimized "roger rabbit" :
1st 00:17:09.583 (40.39 fps)
2nd 00:25:21.687 (27.33 fps)

koepi's build "roger rabbit" :
1st 00:17:01.315 (40.72 fps)
2nd 00:24:45.095 (28.00 fps)

1st pass difference in 30min clip: ~8 sec
2nd pass difference in 30min clip: ~36 sec

conclusion (and my 2c): after these tests i'm 99% convinced that koepi's build is "the one" to use for now. i will test new optimized builds if there really us some speed increases, even if they are few seconds. if you have time and are willing, post your results here with optimized builds.

[ookzDVD]

The latest Jawor's dated 20 May 2004, somehow do not perform
better than Koepi's. :(