parallel encoder beta6a: The fastest encoder in town

[708145]

I'd like to announce a new release of ELDER, a parallEL encoDER.

Current state (ELDER_beta6b.exe) includes:
* a GUI for all modes
* SMP and windows cluster support (distributed encoding)
* xvid 2pass
* xvid 2pass with improved rate control (qcomp)
* xvid with crf parameter (will do a 2pass with same bit distribution as x264)
* x264 1pass crf
* x264 1pass abr
* x264 2pass
* avisynth input
* mp4 output

You'll need PXPerl. There is a copy of the original page up here: PXPerl-5.8.7-6.exe

bis besser,
T0B1A5

[bond]

sounds very interesting. is there a speed/quality tradeoff and if yes, how big is it (any values?)

[708145]

Originally Posted by bond

sounds very interesting. is there a speed/quality tradeoff and if yes, how big is it (any values?)

If you follow the links, you'll find a bit of info.
As the RC is very simple for now, quality should be lower with this alpha.

edit: As of version alpha4 the size is hit quite accurately and the quality drop is very small. But please test for yourself

This will be fixed for the betas. I just wanted to have a working version out ASAP so you guys can start messing with it.

For speedups I wait for ppl testing it and telling me. I have access to SMPs and clusters running Linux, but I can't test on Windows SMP or multicore.

My guess: speedup should be _very_ linear, like 1.9x-fold on 2 cores and 3.9-ish on 4 cores.

bis besser,
T0B1A5

[708145]

Hey guys. It's been over a week and I got almost no feedback!

You don't need HT or SMP to test it. Just download and run it with your favourite avs script.

But for sure I'd be interested in speedup results of HT and SMP systems as well :P

bis besser,
Tobias

[scb]

Hi 708145!
Thank you in advance for Elder for win32!!! Would love to test the alpha version on my windows/smp setup but I have no idea how to install the thing :\

Would you (or some other kind person) let me know exactly how to get it installed, set it up and run it? I promise to report the results here.

tia!

[unskinnyboy]

Originally Posted by scb

Hi 708145!
Thank you in advance for Elder for win32!!! Would love to test the alpha version on my windows/smp setup but I have no idea how to install the thing :\

Would you (or some other kind person) let me know exactly how to get it installed, set it up and run it? I promise to report the results here.

tia!

I haven't tested ELDER4XVID myself, but from what I can see it isn't any difficult. The downloadable file is in .tgz format which is Unix Gzip/Tape Archive and can be decompressed by the latest WinRAR if you have it installed. Once you open the folder ELDER4XVID_alpha1 inside the .tgz, read the file README.txt which should answer your questions. Nothing to install. Extract the contents of the archive into a folder.

[scb]

thanks unskinnyboy

Got it. OK....reading through the readme file...still need help.

* needs a PERL interpreter to run; tested under cygwin...do I just google PERL cygwin, find it and install it? Does it have an installer?

Have been using GK, never used mencoder, so have some problems there.

I figure some of the ELDER parameters can be had from the *.avs via GK.

Here are the parameters that I'm unsure about:
-f number of frames; this is temporary as it can be obtained using the avs (gk?)
-c number of chunks; should be set a bit larger than maxpar<-- huh?
-p 1st pass options to mencoder<-- no idea
-q 2nd pass options to mencoder; must end with "bitrate="<-- no idea

Pretty sure (somewhat sure? hmmm...no idea) that the aspect ratio would be part of mencoder options? As above, I've no idea how to use mencoder so have no idea how to set aspect ratio to desired target of 1.333. Also, how/where do I set the output size to the desired 512x384?

As audio conversion needs to be done separately, is there a SMP-capable audio converter anyone knows of? (this is not ELDER-related per se, just me trying to get faster conversion)

tia for any help! (apologies if my alpha-teste newbieness grates)

[unskinnyboy]

Originally Posted by scb

* needs a PERL interpreter to run; tested under cygwin...do I just google PERL cygwin, find it and install it? Does it have an installer?

I suppose so. Elder.pl, as you can see, is written in Perl and Perl is an interpreted programming language, so you need a Perl interpreter. Cgywin is the environment Tobias tested this in. You need not have Cygwin installed.

Originally Posted by scb

Here are the parameters that I'm unsure about:
-f number of frames; this is temporary as it can be obtained using the avs (gk?)
-c number of chunks; should be set a bit larger than maxpar<-- huh?
-p 1st pass options to mencoder<-- no idea
-q 2nd pass options to mencoder; must end with "bitrate="<-- no idea

You will understand this better if you look at the example in the README:

* encode blafu.avs with 30000 frames to a 10MB XviD using 32 chunks and 2 CPUs:
./ELDER.pl -a blafu.avs -c 32 -m 2 -b 10485760 -f 30000 -p "- -ovc xvid -xvidencopts pass=1" -q "- -ovc xvid -xvidencopts pass=2:bitrate=" -o blafu.avi

-f --> you can get this by loading the avs into VirtualDubMoD, pressing Ctrl+-> and going to the end of the file. If you want to encode till the end of your movie, then take the last frame number as the input to -f.
-c --> to divide the movie into chunks while encoding to enable parallelization, I *think*.
-p --> your first pass encoding options. If you look at the example above, it is "- -ovc xvid -xvidencopts pass=1".
-q --> your second pass encoding options. As per the example, it is "- -ovc xvid -xvidencopts pass=2:bitrate=". As you can see, it ends with "bitrate=".

To know more about the encoding options & to answer your aspect ratio questions, check the MPlayer documentation.

[scb]

thanks again unskinnyboy

yeah...checked the example before my last post. Figured I could get the -f value from gk but you're method would work perfectly, too.

Thanks also for the mencoder doc link

ok...so....
Grab and install ActivePerl here: http://www.dzsoft.com/getperl.htm (the 'With MSI Installer' version, I'd guess?)

Unpack all the alpha version files to it's own directory

Rip dvd to h/d, open prompt, type command (still trying to figure out the exact command), hit 'enter' and let it run?

Question: Will ELDER/mencoder automatically find everything it needs for the XviD codec or do I have to put a file into the ELDER folder?

Regarding selection of the AR=1.333 and 512x384 issues....this looks like something I'd need to add to the -p and -q parameters (?). Found this in the mencoder doc (thanks for the link):
"aspect=<x/y | f (float value)>
Store movie aspect internally, just like MPEG files. Much nicer solution than rescaling, because quality is not decreased. MPlayer and a few others players will play these files correctly, others will display them with the wrong aspect. The as- pect parameter can be given as a ratio or a float- ing point number."

Looks like I'd need to add aspect=1.333 somewhere in the mencoder parameter string (?)

For the sizing, looks like I'd also add this to the mencoder parameters:
-vf scale=512:384

Took a look at videohelp.com's page in hopes of finding a guide but nada on mencoder.

Ugh....hate being too tech-illiterate to just set it all up and get to the testing part

It'd be much appreciated if someone better (more experienced) than me with this package could just give me the info to get going on this....

Other notes....on the project page, known issues:
'fps is hardcoded to 25 (will be fixed in alpha2)'....does this mean I'd get a max of 25fps per cpu (50-ish fps total, possibly 98fps for the 2cpu with HT-enabled)? Am currently getting around 77fps 1st pass, 58fps 2nd pass using GK.

'in the input avs there needs to be a single "#trim()" line to tell ELDER where to fill in trim commands (fixed in alpha2)'....so need to open *.avs in notepad and add this parameter at the top? What (if anything) do I need to put in the ()?

Anyway, again, if anyone feels like writing up a quick guide for 'How to install and test ELDER4XVID alpha for windows', it'd be SOOOO appreciated!!

I'm now gonna go take a couple aspirin and lay down for a little while hehe

[708145]

thanks unskinnyboy for the explanations

Originally Posted by scb

yeah...checked the example before my last post. Figured I could get the -f value from gk but you're method would work perfectly, too.

The -f is becoming obsolete in alpha2 since I learned how to get that info from the avs.

edit: unfortunaltely this is harder than I thought... so bear with me as I'm trying to solve that issue.

Originally Posted by scb

Grab and install ActivePerl here: http://www.dzsoft.com/getperl.htm (the 'With MSI Installer' version, I'd guess?)

Rip dvd to h/d, open prompt, type command (still trying to figure out the exact command), hit 'enter' and let it run?

Question: Will ELDER/mencoder automatically find everything it needs for the XviD codec or do I have to put a file into the ELDER folder?

With active perl I guess you have to remove all "./"s from the script. Apart from that your plan sounds right

no extra xvid needed, it's built-in in mencoder.exe

Originally Posted by scb

Looks like I'd need to add aspect=1.333 somewhere in the mencoder parameter string (?)

For the sizing, looks like I'd also add this to the mencoder parameters:
-vf scale=512:384

set aspect as mencoder option.
I'd suggest resizing in the avs script. So you could use GKnot to set up the avs for you.

Originally Posted by scb

Ugh....hate being too tech-illiterate to just set it all up and get to the testing part
It'd be much appreciated if someone better (more experienced) than me with this package could just give me the info to get going on this....

I try :P

Originally Posted by scb

Other notes....on the project page, known issues:
'fps is hardcoded to 25 (will be fixed in alpha2)'....does this mean I'd get a max of 25fps per cpu (50-ish fps total, possibly 98fps for the 2cpu with HT-enabled)? Am currently getting around 77fps 1st pass, 58fps 2nd pass using GK.

'in the input avs there needs to be a single "#trim()" line to tell ELDER where to fill in trim commands (fixed in alpha2)'....so need to open *.avs in notepad and add this parameter at the top? What (if anything) do I need to put in the ()?

OK. I'll publish alpha2 later which doesn't have this restriction.
You don't have to care about the trim() and not about the fps.

Originally Posted by scb

Anyway, again, if anyone feels like writing up a quick guide for 'How to install and test ELDER4XVID alpha for windows', it'd be SOOOO appreciated!!

I'm now gonna go take a couple aspirin and lay down for a little while hehe

and about the guide: I try to make this whole program much more user friendly. Some things (like the -f) were just for my convenience so I could release ELDER sooner.

testing is tough

bis besser,
Tobias

[708145]

Originally Posted by 708145

The -f is becoming obsolete in alpha2 since I learned how to get that info from the avs.

OK. I'll publish alpha2 later which doesn't have this restriction.
You don't have to care about the trim() and not about the fps.

I'm sorry I couldn't get rid of the -f option.
I'll write a proper changelog tomorrow but as you can see some issues have been ironed out already with alpha2.

The download link is in the first post of this thread.

bis besser,
Tobias

[scb]

Hi 708145

Which PERL interpreter do you recommend to try?
I've never done any real avisynth stuff so not entirely sure about how to edit one (except that I know they open in notepad), what the resize command would be, etc, etc (guess I could read up on it, though).

Will try to make a first go at it this weekend.

I'm just thrilled by the prospect of finally being able to take advantage of smp for xvid! Thank you 708145!

PS- If anyone out there would care to make an actual guide, it'd be SOOOO helpful

[708145]

Originally Posted by scb

Hi 708145
Which PERL interpreter do you recommend to try?
I've never done any real avisynth stuff so not entirely sure about how to edit one (except that I know they open in notepad), what the resize command would be, etc, etc (guess I could read up on it, though).

Well I recommend cygwin since it allows to run Unix-style scripts. But I guess any PERL interpreter would do (if the "./" were deleted from the script).
The reason why I like to leave them in is that I want the same ELDER script to run on both linux and windows.

About the avisynth: You can use GKnot and let it generate the avs for you.

bis besser,
Tobias

[708145]

Originally Posted by scb

As audio conversion needs to be done separately, is there a SMP-capable audio converter anyone knows of? (this is not ELDER-related per se, just me trying to get faster conversion)

I intend to add audio encoding at a later stage once video is working.

bis besser,
Tobias

[stephanV]

I've created a little guide which (hopefully) explains the settings and what programs are needed to run ELDER4xvid.

http://www.geocities.com/grotesteph/ELDER/guide.html

have fun

[Kostarum Rex Persia]

Wow,very good work, stephanV, keep going. Thank you for that good guide.

[708145]

Originally Posted by stephanV

I've created a little guide which (hopefully) explains the settings and what programs are needed to run ELDER4xvid.
http://www.geocities.com/grotesteph/ELDER/guide.html

Thank you very much

bis besser,
Tobias

[708145]

I just released alpha3!

alpha3:
* avimerge.exe added; merging step activated
* distribution style changed: one part all exe files the other the scripts
* changelog added
* comments added
* warning if chunks are too small added
* size and bitrate options added

bis besser,
Tobias

[DarkNite]

Nice additions, will test if they're broken or not as soon as current encode is finished.

[708145]

Originally Posted by DarkNite

Nice additions, will test if they're broken or not as soon as current encode is finished.

How did you know it's broken :]
I fixed a few things in alpha4... at least in runs fine for me.

alpha4:
* sometimes wrong location for trim(): fixed
* StageC didn't start: fixed
* automatic selection of number of chunks if not specified
* delete intermediate files

bis besser,
Tobias