Multithreaded XviD - official thread

[ilhyfe]

Any basis for that? I'd say 3 threads. Definitely no more threads than cores, what made you think that!

I never tried that before on a quadcore but I get best speed with 3 threads. Can you explain why?

[olnima]

...and with Dual-core (E6850)? 1 or 2 threads? I tried 3, CPU-usage raises without getting more speed.

Olnima

[ilhyfe]

...and with Dual-core (E6850)? 1 or 2 threads? I tried 3, CPU-usage raises without getting more speed.

Olnima

I had best resultes with 2 threads.

[pixelk]

Somebody just asked on my blog, where can we find with the latest MT avisynth ? I would like to try to get the maximum fps from my quad-core, and if I have the time post the result of my tests.
Any advice about how I can (with the latest nightly build) get the highest fps ?

[LigH]

Using celtic-druid's "head" build (http://mirror.celticdruid.info/XviD/XviD.cvs.head.MTK.exe, 2007-07-25); on an AM2, the "Threads" edit field is disabled (grayed out) with a "2". Is this the expected behaviour? And is this still the most recent Win32 build?

@ celtic-druid: Would be nice to include the version "1.2" in the filename too. Several members of the german forum were confused about "not finding a v1.2 build", not knowing sysKin's remark:
So, here we go:
What happened: Multithreaded XviD code is now committed to CVS. From now on, all "xvid head" or "xvid 1.2.x" versions have it.

[pc_speak]

@LigH. Interesting. Had the same grayed out problem also. Remembered I had Koepi's XviD-1[1].1.3-28062007.exe installed.
Uninstalled it AND celtic-druid's. Ran a registry cleaner over the system. Reinstalled celtic-druid's XviD.cvs.head.MTK.exe.

Went into 'Configure Decoder' for a quick peek at the settings. Then went into 'Configure Encoder' to set my defaults. "Threads" edit field now enabled. Set it to 4. Quad core. :)

[LigH]

I see -- XviD is not XviD, regarding installers... Always uninstall! :D

Happy New Year!

[philippas]

:thanks:

[88keyz]

For the last little while I have been playing with compiling an XviD release based on the CVS tarballs. It has taken me a while to figure it out but I think I have it now. This release includes both the encoder and decoder and comes in an EXE installer that creates config shortcuts for both. Also included is a complete uninstaller should there be any problems. If anyone would like to test the first new XviD release of 2008 then I would be curious to know how my compile stacks up. Please uninstall any previous XviD releases from your system before installing this one. Based on the XviD 1.2 code this release fully supports SMP systems.

XviD_1.2.127-29032008.exe (http://rapidshare.com/files/103434080/XviD_1.2.127-29032008.exe)

Please keep in mind that I am not a developer and that I have only compiled the CVS release available from Xvid.org for curiosity.

:)

[pc_speak]

Installed your compile as per instructions.
Seems fine. Nothing got broken. :D

Configured encoder OK. Could set threads to 4.
Used decoder & encoder. Still OK.
Uninstall worked fine.
I'll leave it installed for a bit and let you know.

:)

[88keyz]

Thanks for the feedback. I've done some testing and everything seems to work fine for me but I'm sure others out there push the codec closer to its limits than I do.

[pc_speak]

Did about 8-10 hours on my quad core machine over the weekend. All worked just fine. Congratulations.

[88keyz]

A small update to the previous codec release. Based on the January 10th tarball code. Includes new icons and a codec info link.

XviD_1.2.127-09022008.exe (http://rapidshare.com/files/90468184/XviD_1.2.127-09022008.exe)

The older compile is still being hosted for now. Thanks to all of you that have downloaded and tried this compile.

:thanks:

[Buggle]

Maybe someone can give me directions on where to find a changelog for the newest changes in Xvid, like the nice listing of x264 found here (http://trac.videolan.org/x264/log/trunk/common). I am really interested in what kind of experimental fixes and changes are before I put it in action. I have been searching for a while now, but cannot find anything but some mailings that do not get me any further.

[Ranguvar]

*bows to 88keyz*

Here, Buggle. It's as far as I have. Note that
Xvid-1.1.2-01112006 and Xvid-1.1.3-28062007 do not include the changes in XviD-1.2.*



Xvid-1.1.3-28062007:
- {core}: Fixed possible security issue in mbcoding.c


Xvid-1.1.2-01112006:
- {core}: Fixed bug when frame-drop (N-VOP) feature is used in combination with packed B-frames
- {core}: Fixed potential crash on AMD64/EMT64 architecture.
- {core}: Fix for visual_object_verid vs. video_object_layer_verid problem.
- {core}: Ensure intervening bytes are preserved in BitstreamInit()
- {vfw}: Prevent segfault when encoding application calls compress_end with NULL codec context
- {vfw}: Profile definitions updates.


XviD-1.2.-127-25022006:
Changelog to XviD-1.1:
- {core}: New experimental SMP support.
- {core}: Trellis improvements (according to sysKin).
- On uniprocessor machines set number of threads to 0!


XviD-1.2.-127-07012006
Changelog to XviD-1.1:
- {xvidcore} Experimental SMP support (2 threads hardcoded). Patch for P- and B-frames from sysKin applied by hand.
- {xvidcore} Trellis improvements (according to sysKin).
- {xvidcore} Bumped bitstream version to 42, you never know (41 is XviD-1.1.0-final).


XviD-1.1.0 final build.
Changelog:
- {core}: Field interlaced decoding.
- {dshow}: Additional fourcc support.
- {vfw}: Small updates.


XviD-1.0.3
Changelog:
- {xvidcore} Fixed trellis optimization overflow for quant 1 & qpel modes (motion search was done twice, one in SAD mode and one in RD mode)
-Fixed MV clipping with non valid DivX 5 based sequences.
-Fixed RGB 16 bit C functions.
-Fixed posible VOL header corruption for fps=1 encodes.
-DC misprediction caused by bad value clipping (bug forwarded to the ffmpeg project too).
VFW frontend
-Fixed mismatching of hintswidgets.


1.0.0 RC4, Codenamed "Hola"
xvidcore
-GMC 1 warp point (DivX5)
-GMC 2 warp point fix
-Minor postproc code fixes
-Motion Vector clipping fix for stressing test cases.
-Problems caused by wrong cooperation of bframes and frame dropping code.
-Decoder provides quant information in stats.
VFW frontend
-Multiple instance memory leak fix.
-Improved bitrate calculator.
-Some other minor changes.
-DShow frontend
-Release packages have all needed files to build from source.


1.0.0 RC3, Codenamed "Ni Hao"
xvidcore
-Workaround for dev-api-3 decoding that causes psychedelic color effects for non modulo 16 encodes. dev-api-3 builds were mostly used by win32 users during the transition from 0.9.x series and 1.0.x.
-Buffer overflow reading in decoder (read up to bytes to far).
VFW frontend
-Bitrate calculator fixes.
-Status window fixes, GMC frames get counted too now.
-Mod4 / YV12 resolutions encoding fixed.
DShow frontend
-Updates and cleaning.


1.0.0 RC2, Codenamed "Jambo"
xvidcore:
-Decoder bugfixes (GMC+interlaced).
-Changed the DivX packed user string to version 999 so DivX decodes XviD packed bitstreams like it should have done before.
-Fixed YVYU colorspace space (was using Y as V channel)
VFW frontend:
-Added bitrate calculator.
-Output a DLL linking library when compiling with MSVC.
DShow frontend:
-Video flipping fixed.
-Added MP4V to the supported FourCCs.
-Command line driving.


1.0.0 RC1, codenamed "Niltze"
xvidcore
-Scaled zones should now work in 2pass 1&2.
-Qpel is disabled during first pass now.
-Bug in PP using MB quants badly initialized.
-Changed Win32 build type to DLL.
VFW frontend
-Changed linking policy. Links against xvidcore.dll. The vfw component is now xvidvfw.dll.
-GUI improvements.
-Added PP options as in DShow frontend.
-Added easier constant quant encoding as most of users complained though the feature was available thanks to zones.
DShow frontend
-Changed linking policy. Links against xvidcore.dll.
-Better seeking.
-Fixed colorspace usage.


1.0.0 beta3, Codenamed "Selam"
xvidcore:
-Defaulted back to VGA 1:1 PAR.
-Enabled SSE2 assembly code for IA32 platforms.
-Improved and bugfixed two pass:
-better frame size scaling.
-better defaults.
-handles up to 2TB target filesizes
-1st pass disables automatically CPU hungry features.
-Added fast ME replacement routines.
-Added Post Processing to decoder:
-Deblocking.
-FILM noise.
-Various Bugfixes.
VFW frontend:
-Added AR widget.
-Added "Turbo mode" that enables core fast ME routines.
-Removed DXN profiles from the profile list.
-New defaults.
DShow decoder frontend:
-Added PP widgets.


1.0.0 beta2, Codenamed "Ciao"
xvidcore:
-MPEG4 compliance is back (beta1 was missing the VOS header)
-matrix quantization is finally thread safe
-improved vop type decision
-2pass2 plugin: min key interval was a misleading name. It's been renamed to kfthreshold. And as the kfthresholding behavior was a bit -too aggressive, it's been disabled until we decide how it should behave w/o hurting quality
-single plugin: fixed quant capping
-interlacing artefacts fixed.
VFW fontend:
-some misuses of xvidcore were fixed
-min key frame widget renamed and moved to the 2pass panel
Debian package:
-small errors in the control file reported and fixed by Nicolas Boos


1.0.0 beta1, Codenamed "Aloha"
-New API.
-New Motion Estimation system
-with SAD based algorithms,
-or Rate Distortion optimized algorithms.
-Dynamic frame type decision based on a fast motion estimation pass
-Support for bvops and svops (up to 3 warp points).
-QuarterPel precision.
-Trellis optimization for h263 and MPEG quantization schemes.
-Special mode for cartoons/anime like futuruma/the simpsons or any anime with flat color areas.
-Mod 2 resolution support.
-Two pass algorithm is now part of xvidcore.

[Buggle]

Thanks Ranguvar, but I meant more like the changes in the nightly builds, or to the cvs, like the changes noted between the latest svns of x264.

[88keyz]

That I'm aware of the Xvid team doesn't publish info about the 1.2.x family of builds. They have only ever been available as CVS source code and there is no info on the site about changes made to the code. According to the site each night at midnight they simply upload the latest source tarball, nowhere is there any info about version until you open the tarball, where all it gives you is a date. I have never seen release notes for any of the 1.2.x family of releases other than what Koepi published on his site when he released his first compile of the 1.2.x code.

XviD-1.2.-127-25022006
Changelog to XviD-1.1:
- {core}: New experimental SMP support.
- {core}: Trellis improvements (according to sysKin).
- On uniprocessor machines set number of threads to 0!

To the best of my knowledge the major change made was that the encoder will now use multiple processors. I'm sure there have been other tweaks along the line though, sysKin might be the guy to answer this question.

[professor_desty_nova]

Go to Celtic Druid's page http://celticdruid.no-ip.com/xvid/ and choose changelog next to the XviD link. The latest that he recorded is in the last page of the forum post (but only goes to midle of 2007).

[Buggle]

A small update to the previous codec release. Based on the January 10th tarball code. Includes new icons and a codec info link.

XviD_1.2.127-10012008.exe (http://rapidshare.com/files/82784043/XviD_1.2.127-10012008.exe)

The older compile is still being hosted for now. Thanks to all of you that have downloaded and tried this compile.

:thanks:

Have encoded lots and lots of stuff with your latest build, already. Haven't run into problems on neither my dual, nor my single core. And its fast, that's really cool :D. Now I encode in second pass at almost realtime speed on my 2800+ Barton. The last time I tried Xvid that was significantly lower (a few months ago), if I recall correctly. Maybe that's just some whishful thinking ;)

Good goin'!

[Cyberace]

Any updates on SMP (multi-processor) DECODING support?

[sysKin]

Any updates on SMP (multi-processor) DECODING support?

I definitely don't plan to try. What for? I can't imagine any SMP-capable computer not decoding xvid on single thread already (unless it's an ancient dual-cpu).

And anyway, what's the point of using XviD for decoding. Other decoders exist.

[OterLabb]

A small update to the previous codec release. Based on the January 10th tarball code. Includes new icons and a codec info link.

XviD_1.2.127-10012008.exe (http://rapidshare.com/files/82784043/XviD_1.2.127-10012008.exe)

The older compile is still being hosted for now. Thanks to all of you that have downloaded and tried this compile.

:thanks:

Did a little testing on this release on my Q6600, with 5 threads (which seems to be default) vdub uses 80-90% of the total CPU, all four cores. But with 5 threads, the fps is dropping and overall encoding takes longer time.. Found out that 3 threads seemed to work best, wich seems to be like 1.1.3 Final build, concerning fps. Anyone else getting more fps on q6600 with this build?

[Ranguvar]

anyway, what's the point of using XviD for decoding. Other decoders exist.
Gah. Bad logic.

What's the point of using Xvid? Other MPEG-4 ASP encoders exist.

;)

[sysKin]

Gah. Bad logic.

What's the point of using Xvid? Other MPEG-4 ASP encoders exist.

;)

Good logic. XviD compresses better than others.
XviD decompresses identically to others.

[Ranguvar]

Good logic. XviD compresses better than others.
XviD decompresses identically to others.
Yes, but through demand and work, it could work better.

[squid_80]

Decoders must produce identical output. There are already faster decoders than xvid, so why waste time speeding up xvid's decoding to try and match them when you're going to get the same identical results?

[TripleA]

I remember back in the dawn of time when I first started using XviD that the encodes I made required insane hardware requirements such as 800MHz P3s and the like.

I suppose slightly higher requirements are to be expected with the enhancements since then (that was pre-B-frames, btw), but I don't think there is any modern CPU that couldn't decode XviD on a single core without breaking a sweat or, indeed, leaving power-saving mode. Well, maybe VIA's C3 can't. But then the C3 isn't gonna benefit from multi-core optimizations, is it?

[squid_80]

The Duron 800 in my car has no probs playing back SD clips, with only 384mb ram and gps software running at the same time.

[EuropeanMan]

^ SQUID80........please help me out man...

your xvidencraw.exe file for some reason crashes on me... :( i sent you a PM and have a thread open in this forum as well...

PLEEEEEEEEAES HELP

[squid_80]

^ How not to file a bug report.

You didn't say what the problem was, so I didn't pay attention.

[audyovydeo]

Maybe an offtopic question, but it's the only thread where I saw v 1.2 mentioned - is xvid 1.2.0 the new project as described here :

http://www.xvid.org/Xvid-Codec.2.0.html

???


cheers
audyovydeo

[Ranguvar]

No.

The Xvid 1.2.x branch has seemed to focus on SMP (multi-threading) and trellis.

[hajj_3]

so how much faster is the latest 1.2 build on a dual or quad core core2duo than 1.13?

ive got a core2duo overclocked to 3.6ghz and will be getting a 45nm quad core in 2months.

also any plans to add sse4 support? divx 6.7 is 60% or something faster with sse4 capable cpus.

thanks!

[clsid]

divx 6.7 is 60% or something faster with sse4 capable cpus.That is not correct. IT IS A MARKETING STUNT.

They added 1 specific routine to DivX that performs really well with SSE4. However, that particular routine is pretty much useless in real life. So DivX encoding does not really benefit from SSE4.

[hajj_3]

That is not correct. IT IS A MARKETING STUNT.

They added 1 specific routine to DivX that performs really well with SSE4. However, that particular routine is pretty much useless in real life. So DivX encoding does not really benefit from SSE4.

i've seen reviews of 45nm quad penryn cpu's encoding movies to divx 6.7 on anandtech site and its literally about 60% faster. independant review, nothing to do with the divx company.

[Ranguvar]

i've seen reviews of 45nm quad penryn cpu's encoding movies to divx 6.7 on anandtech site and its literally about 60% faster. independant review, nothing to do with the divx company.
Link?

[Ranguvar]

so how much faster is the latest 1.2 build on a dual or quad core core2duo than 1.13?

ive got a core2duo overclocked to 3.6ghz and will be getting a 45nm quad core in 2months.

also any plans to add sse4 support? divx 6.7 is 60% or something faster with sse4 capable cpus.

thanks!
For the first, try and see. There's a very new build posted just a little bit ago.

For the second, I doubt it. It's been asked of x264 a lot, and they basically said their own software routines were faster than using SSE4. So I presume it's pretty much the same with Xvid.

[sysKin]

SSE4 only makes things faster if you use very inefficient algorithm to begin with. Basically what you need is a full search, which is pointless and horribly slow. Then, this full search can be made 60% faster.

This is pretty much what DivX did, they invented particular algorithm for the purpose of using SSE4 with it.

[Mgz]

Link?

http://www.anandtech.com/cpuchipsets/intel/showdoc.aspx?i=3195&p=4

[clsid]

I have more trust in the expertise of sysKin, Manao, and akupenguin.

http://forum.doom9.org/showthread.php?t=133567&highlight=divx+sse4
http://forum.doom9.org/showthread.php?t=124881&highlight=divx+sse4

[lazik]

Divx in one test is slower n*times then Xvid.
Try to capture DV source with real-time compression. Yes I know is tricky test, but there is no DV capture tool that can use Divx codec. ;)

[foxyshadis]

SSE4 optimizations only work with DivX set to Insane. That's exactly what anandtech did. If you just use any of the slow speeds (let alone fast), you'll see no boost from sse4.

[cmw]

Erm, sorry if this has been answered prior in this thread, as I'm not done reading trhough, but this seems very strange to me. I use the batch semi-automatic script to encode with xvid, have the xvid built from a few days back (posted prior in this thread) installed and use the neweset xvid_encraw from squid80. I have an Intel Core 2 Duo E6600, Windows XP SP2 32bit.

Commandline input:

start "TGHQ-30 - Pass 1/2" /b /wait /normal "C:\autok2\bin\xvid_encraw.exe" -zon
es 0,q,3,KO -threads 2 -progress 100 -max_key_interval 300 -packed -quality 5 -v
hqmode 1 -max_bframes 2 -bquant_ratio 162 -bquant_offset 0 -qtype 1 -qmatrix "C:
\autok2\matrix\eqm_v3ulr_rev3.xcm" -nochromame -turbo -lumimasking -bitrate 1373
-pass1 "C:\Temp\dvd\tmp\extrem.pass" -type 2 -i "C:\Temp\dvd\extrem.avs"

As you can see, it specifies threads=2

Subsequent encoding:


xvid_encraw - raw mpeg4 bitstream encoder written by Christoph Lampert 2002-2003


Trying to retrieve width and height from input header
xvid [info]: Avisynth detected
xvid [info]: Input colorspace is YV12
xvid [info]: Input is 640 x 352, 29.970fps (30000/1001), starting from frame 0
xvid [info]: Number of frames to encode: 214758, Bitrate = 1373kbps
xvid [info]: xvidcore build version: xvid-1.2.0-dev
xvid [info]: Bitstream version: 1.2.-127
xvid [info]: Detected CPU flags: ASM MMX MMXEXT SSE SSE2 TSC
xvid [info]: Detected cpus = 2, threads requested = 1, threads in use = 1
xvid [info]: Threaded input reading active

Only 1 thread requested? Only 1 thread used? Umm... help pls :)
Does it have something to do with the threaded input reading active? What does that mean? oO

Edit: I've just installed the MT version of avisynth. Encoding speed went up considerably, however, xvid still displays that only 1 thread is used :/

Edit2: I just tried and used an older version of xvid_encraw, and it says cpu's used = 2 there. The line about threaded input reading ist missing there though, so I guess it has something to do with that. Encoding speed is pretty much the same on both versions though.

[squid_80]

1 thread for encoding + threaded input = 2 threads. If you want to try forcing 2 encoding threads use -nothreadedinput or -threads 3, but you'll probably lose speed.

[88keyz]

As a little birthday present to myself I decided to compile a new tarball.

XviD_1.2.127-29032008.exe (http://rapidshare.com/files/103434080/XviD_1.2.127-29032008.exe)

Enjoy.

[clsid]

I hope you know that the tarball is generated automatically, and not just whenever something has changed. Judging from the file dates inside the tarball, nothing has changed since your previous build.

[88keyz]

No, I didn't know that. But to be honest I don't really care either, it was just fun to release a compile on my birthday. Its not like its a lot of work, probably took less than 5 minutes to do so I'm not worried about any time I may have wasted.

:rolleyes:

[audyovydeo]

sorry guys which is the official link for the 1.2.x branch tarballs ?
I couldn't trace it through xvid.org ...


thanks
audyovydeo

[clsid]

It's right there in the download section:
http://downloads.xvid.org/downloads/xvid_latest.tar.gz

[totya]

As a little birthday present to myself I decided to compile a new tarball. This one is based on the February 9th release, just like me!

XviD_1.2.127-09022008.exe (http://rapidshare.com/files/90468184/XviD_1.2.127-09022008.exe)

Enjoy.

Thanks!

My problem with xvid (all version) : preset manager is unusable. Only 1 (one!) user preset available. This is impossible. Different preset required for tv capture, avi recompress etc... :(