Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 ASP

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 18th December 2005, 16:54   #1  |  Link
sysKin
Registered User
 
sysKin's Avatar
 
Join Date: Jun 2002
Location: Adelaide, Australia
Posts: 1,167
Multithreaded XviD! [updated - now bframes too!]

Edit much later: this was experimental and now is obsolete. New thread, discussing official code, is here: http://forum.doom9.org/showthread.php?t=107783

Hi everyone

I just finished my "multithreaded xvid motion estimation for p-frames" project ^____^

First, the patch: http://syskin.is.dreaming.org/smp.patch . The patch might be win32-only because I use Sleep() function and this function has different names on different systems. Apart from that, it's x264's code so should work on many OSes.

Second, a two-threaded build: http://syskin.is.dreaming.org/xvidcore-2threads.7z

Third, a 4-threaded build: http://syskin.is.dreaming.org/xvidcore-4threads.7z

[edit]Third and a half, a single-threaded reference build: http://www.aziendeassociate.it/cd.asp?dir=/
Use the xvid.cvs.head.2005.12.10.7z build. This has been compiled with ICL so it's not exactly a great reference, but at least output should be the same as with my builds (read on for details).

Fourth, some random speed results (defaults, vhq4, NO bframes. X2 4200+ CPU).

1 thread : 1:48
2 threads : 1:05
3 threads : 1:06
4 threads : 1:08
16 threads: 1:11

Now, some info: p-frame's ME is multithreaded. This means it's faster on systems that can use it, like my AMD X2. The slower ME is, the bigger the effect, so use VHQ4 for extra difference.
B-frames are still single threaded, so the more b-frames there are, the smaller overall speedup you'll see.

Output is, or should be, binary identical to single threaded build.

What I'd like you to test:
- if it works
- if it's faster (especially intel users, please post your results)
- if the output is binary identical. Use Celtic Druid's 10 Dec CVS-Head build and compare - should be the same.

Have fun!

Radek
__________________
Visit #xvid or #x264 at irc.freenode.net

Last edited by sysKin; 24th February 2006 at 12:25.
sysKin is offline  
Old 18th December 2005, 17:07   #2  |  Link
bond
Registered User
 
Join Date: Nov 2001
Posts: 9,779
first!

ok i dont have a dualcore cpu, so i cant say much
__________________
Between the weak and the strong one it is the freedom which oppresses and the law that liberates (Jean Jacques Rousseau)
I know, that I know nothing (Socrates)

MPEG-4 ASP FAQ | AVC/H.264 FAQ | AAC FAQ | MP4 FAQ | MP4Menu stores DVD Menus in MP4 (guide)
Ogg Theora | Ogg Vorbis
use WM9 today and get Micro$oft controlling the A/V market tomorrow for free
bond is offline  
Old 18th December 2005, 17:11   #3  |  Link
Doom9
clueless n00b
 
Join Date: Oct 2001
Location: somewhere over the rainbow
Posts: 10,570
can you just replace xvidcore.dll in the system32 directory, without any registration?
__________________
For the web's most comprehensive collection of DVD backup guides go to www.doom9.org
Doom9 is offline  
Old 18th December 2005, 17:12   #4  |  Link
sysKin
Registered User
 
sysKin's Avatar
 
Join Date: Jun 2002
Location: Adelaide, Australia
Posts: 1,167
Quote:
Originally Posted by Doom9
can you just replace xvidcore.dll in the system32 directory, without any registration?
Yes. Only VfW and dshow are registered, and xvidcore is just a simple library opened by these two.
__________________
Visit #xvid or #x264 at irc.freenode.net
sysKin is offline  
Old 18th December 2005, 17:28   #5  |  Link
Doom9
clueless n00b
 
Join Date: Oct 2001
Location: somewhere over the rainbow
Posts: 10,570
should I use a CSV Head (http://www.aziendeassociate.it/cd///....2005.12.10.7z, 1.2) build or the latest 1.1 build (http://www.aziendeassociate.it/cd///XviD.1.1.cvs.exe) ?
__________________
For the web's most comprehensive collection of DVD backup guides go to www.doom9.org
Doom9 is offline  
Old 18th December 2005, 17:43   #6  |  Link
sysKin
Registered User
 
sysKin's Avatar
 
Join Date: Jun 2002
Location: Adelaide, Australia
Posts: 1,167
CVS Head build (better trellis than 1.1). I'll update the main post now.
__________________
Visit #xvid or #x264 at irc.freenode.net
sysKin is offline  
Old 18th December 2005, 18:41   #7  |  Link
AssassiNBG
Registered User
 
Join Date: Mar 2005
Location: Pleven, Bulgaria, Europe.
Posts: 45
Hmm I was never able to get 100 CPU usage while encoding, but this thingie kinda gives me a hope. It's finally doing ~60% CPU Usage. It was 50% before (which means it was using only one thread of my P4 3 GHz HT.)
AssassiNBG is offline  
Old 19th December 2005, 10:44   #8  |  Link
sysKin
Registered User
 
sysKin's Avatar
 
Join Date: Jun 2002
Location: Adelaide, Australia
Posts: 1,167
Now with b-frames, too!

I just coded multithreaded b-frame's ME.

Patch (p- and b- together): http://syskin.is.dreaming.org/smp2.patch

Binary (2 threads): http://syskin.is.dreaming.org/xvidco...ads-bframes.7z

This time, output is not binary identical. This is by design - in theory, the more threads for b-frames the smaller encoder efficiency, however in practice the difference is nil (and up to ~1% bitrate for infinite number of threads).
__________________
Visit #xvid or #x264 at irc.freenode.net
sysKin is offline  
Old 19th December 2005, 12:56   #9  |  Link
Doom9
clueless n00b
 
Join Date: Oct 2001
Location: somewhere over the rainbow
Posts: 10,570
Quote:
It's finally doing ~60% CPU Usage. It was 50% before (which means it was using only one thread of my P4 3 GHz HT.)
Singlethreaded XviD in VDub uses about 80% CPU on a real SMP system (HT is largely imaginary.. it can hurt as much as it can help and intel may actually dump it in the future and just focus on physical cores which are the real deal). So you see.. dump that intel marketing blablah and get a second core.. it's well worth it.
__________________
For the web's most comprehensive collection of DVD backup guides go to www.doom9.org
Doom9 is offline  
Old 20th December 2005, 01:09   #10  |  Link
gilligan2
Registered User
 
Join Date: Apr 2005
Posts: 50
I replaced the files in the system 32 folder. Do i need to do anything with the patch itself and,if so,can someone tell me what to do.

I have a dual Xeon system so i would be curious to see if it speeds up my encoding time at all.

TIA,

G
gilligan2 is offline  
Old 20th December 2005, 02:03   #11  |  Link
nightrhyme
║┐║ XviD N00B ║┐║
 
nightrhyme's Avatar
 
Join Date: Apr 2004
Posts: 223
A little test from a Hyperthreaded P4 (Complete system specs at bottom)

xvid.cvs.head.2005.12.10


2.threads with B frames


xvid.cvs.head.2005.12.10 is also able to get 42fps. 42fps was max for both builds




System Specs:
CPU: P4C Northwood 2.4@3.0 Ghz, 800@1000 Mhz FSB, Cooled by Zalman CNPS7000A-CU
MOBO: Asus p4p800 Deluxe, bios 1019. MAM enabled, Turbo mode
MEMORY: 2 x 512MB Mushkin pc3200 Special (2-2-3-5-8-64T-64Ásec)
GRAPHIC'S: Sapphire 9800pro ,Clock Default, (Cat 4.12), VGA-Silencer
HD's: P-ATA: WD1200JB 8MB Cache 120 GB, WD2000JB 8MB Cache 200 GB, Maxtor DM 9+ 160GB, 8mb Cache
PSU: LC-POWER LC6550G 550W
SOUND: Onboard Soundmax
OS: WinXP SP2, INF Update 6.2.1.1001.
Monitor: Samsung 959NF, 19" Diamondtron
__________________
"We are all Newbies at something"
-- NightRhyme
nightrhyme is offline  
Old 20th December 2005, 02:26   #12  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
eh.. usual HT BS...
copy both CPU graphs with a screen capture... get the right one and flip it horizontally...
place it over the left graph and look at how edges are perfectly aligned...
oh... if you sum the cpu load at every istant you get "magically" 100%...
that's why multithreading an app to make it run faster on intel HT CPUs is useless...

Last edited by Sharktooth; 20th December 2005 at 02:33.
Sharktooth is offline  
Old 20th December 2005, 06:43   #13  |  Link
sysKin
Registered User
 
sysKin's Avatar
 
Join Date: Jun 2002
Location: Adelaide, Australia
Posts: 1,167
Quote:
Originally Posted by nightrhyme
A little test from a Hyperthreaded P4 (Complete system specs at bottom)

xvid.cvs.head.2005.12.10 is also able to get 42fps. 42fps was max for both builds
Aha! "Total time (estimated)" is visibly lower in 2-threaded build.

Never expect significant improvements from HT only - HT doesn't give you more computing power, it can just plug execution stalls in one thread with instructions from another thread.
The worse program optimization, the more HT can give you. "Unfortunately" XviD is pretty well optimized
__________________
Visit #xvid or #x264 at irc.freenode.net

Last edited by sysKin; 20th December 2005 at 08:14.
sysKin is offline  
Old 20th December 2005, 16:35   #14  |  Link
gilligan2
Registered User
 
Join Date: Apr 2005
Posts: 50
Quote:
Originally Posted by gilligan2
I replaced the files in the system 32 folder. Do i need to do anything with the patch itself and,if so,can someone tell me what to do.

I have a dual Xeon system so i would be curious to see if it speeds up my encoding time at all.

TIA,

G
I'm willing to do some testing with a true 2 CPU machine (Xeon 3.06 x 2 with hyperthreading enabled and/or disabled ) if anyone can explain to me how to apply the patch or exactly the steps i need to follow to test out the multithread versions.

Thanks,

Tom
gilligan2 is offline  
Old 20th December 2005, 17:24   #15  |  Link
sysKin
Registered User
 
sysKin's Avatar
 
Join Date: Jun 2002
Location: Adelaide, Australia
Posts: 1,167
Quote:
Originally Posted by gilligan2
I'm willing to do some testing with a true 2 CPU machine (Xeon 3.06 x 2 with hyperthreading enabled and/or disabled ) if anyone can explain to me how to apply the patch or exactly the steps i need to follow to test out the multithread versions.
- Install any XviD. It's probably best to use any modern build (like Celtic Druid's), but in theory even 1.0 would do.
- replace xvidcore in windows\system32 directory.
__________________
Visit #xvid or #x264 at irc.freenode.net
sysKin is offline  
Old 20th December 2005, 19:02   #16  |  Link
gilligan2
Registered User
 
Join Date: Apr 2005
Posts: 50
OK,thanks,i'll try it out .I thought i had to do something with the "patch" you posted up above, So i'll just use the 4 thread or 2 thread core along with the celtic druid Xvid and try it out.Thanks for your help !!

G
gilligan2 is offline  
Old 20th December 2005, 19:08   #17  |  Link
CEC
Chaos Energy Creator User
 
CEC's Avatar
 
Join Date: Sep 2005
Posts: 49
I've got a 26% speedup here! I used the 2 threads file! I have a P4 HT 3 Ghz!

- it works!!!
- it's faster!!!
- the output isn't binary identical because I used the old Koepi 1.1.0 beta 2 build!!!

Nice work!!!!
CEC is offline  
Old 20th December 2005, 19:19   #18  |  Link
gilligan2
Registered User
 
Join Date: Apr 2005
Posts: 50
OK,i went from 59 minutes for a certain clip i am encoding to a little under 30 minutes so it really sped up the process for me !!

Thanks again.

I used the ICL 9 version dated 9/19/2005 and the 4 thread core dated 12/20/2005.

g

Last edited by gilligan2; 20th December 2005 at 19:32.
gilligan2 is offline  
Old 20th December 2005, 19:25   #19  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
Quote:
Originally Posted by CEC
I've got a 26% speedup here! I used the 2 threads file! I have a P4 HT 3 Ghz!

- it works!!!
- it's faster!!!
- the output isn't binary identical because I used the old Koepi 1.1.0 beta 2 build!!!

Nice work!!!!
use the 1.2 CVS for speed comparison.
Sharktooth is offline  
Old 23rd December 2005, 16:17   #20  |  Link
Warrex
Registered User
 
Join Date: Dec 2005
Posts: 69
Hi,

I have an Athlon X2 3800+ and encode mainly with XVID 1.03 (might move to newer version and use Divx profiles when there is a final) for my standalones. One is quite old so I use the following settings for Xvid:

AS@L5, Mpeg and no B-Frames, no GMC, no QPel

Xvid 1.03: 168s
Xvid 1.2 cvs: 144s
Xvid 1.2 cvs with xvidcore-2threads: 144s

Divx 6.1 (Home Theatre Profile, Balanced, no B-Frames): 87s


Yes, I overwrote xvidcore.dll in windows/system32. Used same Avisynth script and newest version of Virtualdubmod. No idea why speed did not go up. Any clues?


With these settings and with this clip SSIM was also better when using Divx:

Xvid 1.2 cvs: 0.9705 (78.7)
Divx 6.1: 0.9748 (81.5)

Anyway: Great to see you working an SMP support!

Edit: Output was identical.

Last edited by Warrex; 24th December 2005 at 16:49.
Warrex is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.