x264 parallel encoder

[708145]

Development continues! :)


Notice: Development of this branch of ELDER is frozen. I might include it to the main branch at a later time.


The main branch here http://forum.doom9.org/showthread.php?t=100766 contains x264 modes as well.
edit2006-12-15: the beta5 branch contains a x264 2pass with GUI now!
edit2007-06-29: beta6b released with x264 3pass support!

The old x264 only version can still be found here:
http://www.funknmary.de/bergdichter/projekte/ELDER/ELDER4X264_beta4d.exe

bis besser,
T0B1A5

[708145]

Changelog for beta1:
* initial release based on alpha5 of ELDER4XVID
* uses XVID 1st pass and X264 2nd pass
* fixed bug when trim() was used in the input avs
* use xvid_encraw for 1st pass (stage A and stage B)
* consistent naming for temporary files
* parallelism broken

bis besser,
T0B1A5

[Guest]

Please stop crossposting, per forum rule 8. Thank you. :readrule:

[Revgen]

By crossposting do you mean the thread (http://forum.doom9.org/showthread.php?t=100766) he posted in the Xvid forum?

That thread isn't the same subject as this one yet the titles describing the threads are the same.

This seems more like a rule #9 issue.

[708145]

Please stop crossposting

I don't expect x264 users to read the xvid forum. And this IS software for x264. That's why it makes sense to me to post it here.

Please note that the post in the xvid forum announces software for xvid. :)

bis besser,
T0B1A5

[708145]

By crossposting do you mean the thread (http://forum.doom9.org/showthread.php?t=100766) he posted in the Xvid forum?

That thread isn't the same subject as this one yet the titles describing the threads are the same.

This seems more like a rule #9 issue.

Feel free to propose a better title.

bis besser,
T0B1A5

[Revgen]

How about X264 parallel encoder for windows - beta1

That way It's differentiates itself from the xvid one.

[Sirber]

x264 can already "parallel" by using more than 1 thread. I don't see the point... :confused:

[Guest]

Same title and both recent posts gave a changelog for beta1. If it looks like a duck and quacks like a duck...

[708145]

Same title and both recent posts gave a changelog for beta1. If it looks like a duck and quacks like a duck...

Both topics changed :)
And btw, even the changelogs are slightly different.

The two scripts share some code but will share less in later revisions. For example I use the same 1st pass code for both ATM.

bis besser,
T0B1A5

[708145]

x264 can already "parallel" by using more than 1 thread. I don't see the point... :confused:

a) current parallelism does not scale far
b) current parallelism does not scale well

ELDER is supposed to scale approx. 60x on 64nodes.
But as I wrote before... current beta is is "parallelism challenged" due to an issue with Active Perl.

I will be able to sort out the scaling part myself but I need testers for the quality aspects of ELDER which are testable with one CPU machines easily by either using defaults or setting "-m 1" aka selecting to use 1 CPU max.

Another planned feature is pause/resume which should come handy for most users as well.

And I have to start yet another thread sometime as I plan to include SNOW encoding as well :D

bis besser,
T0B1A5

[foxyshadis]

I'm going to be getting a 4-core (or 8-core, haven't decided yet) server to upgrade our main office, when I do I'll give your stuff a test.

How exactly does it work? Does it split each frame into x blocks like x264? Does it split the movie (on 1st-pass-generated IDR chunks perhaps)? A more unique method?

[708145]

How exactly does it work? Does it split each frame into x blocks like x264? Does it split the movie (on 1st-pass-generated IDR chunks perhaps)? A more unique method?

I split the video scene accurately into chunks on the temporal axis. For more details refer to the project page :)

bis besser,
T0B1A5

[Hellworm]

I tried ELDERchap, but when I tried the elderchap_testrun.sh it tolds me in the first line:
cat: chapter.info: Datei oder Verzeichnis nicht gefunden
then it continues, but it seems to do nothing ( its ready in an instant ) and when I looked into the jobfiles they were empty.
To me it looks like something with gettting the chapter info from mplayer isnt working. ( the info is coming from mplayer, isnt it? )

[708145]

I tried ELDERchap, but when I tried the elderchap_testrun.sh it tolds me in the first line:

then it continues, but it seems to do nothing ( its ready in an instant ) and when I looked into the jobfiles they were empty.
To me it looks like something with gettting the chapter info from mplayer isnt working. ( the info is coming from mplayer, isnt it? )

elderchap is obsolete and only worked/s on linux.
please try elder.

bis besser,
T0B1A5

[Hellworm]

I'm sorry. I forgot to mention, that I am using Linux so I can only use elderchap.
So can you help with my Problem?

[708145]

I'm sorry. I forgot to mention, that I am using Linux so I can only use elderchap.
So can you help with my Problem?

gen_chapinfo.pl is supposed to print the chapter info (hence the name)
it will only work when the DVD is in the drive NOT with a ripped DVD on HD.

Anyway I suggest to wait for beta2 which I'll port to linux (wine) soon. Elderchap is not always frame accurate, ELDER is. :)

bis besser,
T0B1A5

[Hellworm]

I used a real DVD.
It's a shame that there won't be a native Linux program. :(
Will the port use wine for avisynth or will there be a possibility to use mplayers input?
If the actual program consists of perl scripts, it should be much easier to port it directly to Linux, than using wine.
Can you outline how the Programm is working? On the Project page is not that much Information.

Anyway, thank you for your efforts, I think elder is a program with great potential! :thanks:

[708145]

I used a real DVD.
It's a shame that there won't be a native Linux program. :(


Once avisynth 3.0 will be released for linux it will be linux native :)


Will the port use wine for avisynth or will there be a possibility to use mplayers input?
If the actual program consists of perl scripts, it should be much easier to port it directly to Linux, than using wine.


I never found out how to do frame accurate seeking in mplayer => :(


Can you outline how the Programm is working? On the Project page is not that much Information.


As written it splits the first pass into chunks of same duration (A). Then adds a correction stage (B) to find out correct scene changes on the chunk borders. The second pass is then conducted scene accurate and bitrate accurate (C) and an optional correction (D) is added to hit the target size more accurately.


Anyway, thank you for your efforts, I think elder is a program with great potential! :thanks:

Thanks for appreciating the work done so far. I hope to have beta2 ready next weekend. This is no promise though :D

bis besser,
T0B1A5

[Hellworm]

Once avisynth 3.0 will be released for linux it will be linux native :)

avisynth 3.0 is already available it's just that you can't do much with it ( you can already render fonts :D )

But anyway, I asked in mplayers irc and there really is no method of frame accurate seeking. I'll try to look in the mplayers source, perhaps I can add it myself, but I havent that much time so it could take a while.

As written it splits the first pass into chunks of same duration (A). Then adds a correction stage (B) to find out correct scene changes on the chunk borders. The second pass is then conducted scene accurate and bitrate accurate (C) and an optional correction (D) is added to hit the target size more accurately.

I read that ( I'm Sorry for my bad English, I meant describe, not outline ), what I exactly wanted to now is, how are the stages done, for example B: is it doing an additional first pass, around the borders, and then analyzing the stats file, or is it overlapping the chunks in stage A to get the info. Does bitrate accurate mean that every scene is done with the desired bitrate or is the second pass calculation of the codec done, to give every scene the same quality, but a different bitrate?
Perhaps you can write that details on the project's page? There isn't much info on it.

I hope to have beta2 ready next weekend. This is no promise though

Thats nice to hear, I'll test it when it's coming! :p

[708145]

I read that ( I'm Sorry for my bad English, I meant describe, not outline ), what I exactly wanted to now is, how are the stages done, for example B: is it doing an additional first pass, around the borders, and then analyzing the stats file, or is it overlapping the chunks in stage A to get the info. Does bitrate accurate mean that every scene is done with the desired bitrate or is the second pass calculation of the codec done, to give every scene the same quality, but a different bitrate?
Perhaps you can write that details on the project's page? There isn't much info on it.


I updated the Wiki:

ELDER Description (parallEL encoDER):
* ELDER Does a parallel 2 pass encode that tries to use the same bit allocation as normal 2pass
* First pass is divided into 2 stages:
* Stage A divides the stream into chunks of the same size
* Stage B corrects the scene changes on the chunk borders (encode from last I-frame of previous chunk to 2nd I-frame of current chunk).
* The second pass is divided into 2 stages:
* Stage C encodes a part of the chunks in parallel both scene and bitrate accurate (a bitrate is assigned to each chunk to give all of them approx. the same quality).
* Stage D corrects for size under- or overshoot

bis besser,
T0B1A5

[Hellworm]

I updated the Wiki:

Thats more informative, thank you! :thanks:

[708145]

new version is up.

beta2:
* added installer
* muxing to mp4
* show progress
* debug output suppressed
* compile perl scripts on windows or ship perl.exe with ELDER
* improved defaults for chunksize

bis besser,
T0B1A5

[bond]

detected fps = 25.00
detected frames = 300
using 2 threads to encode 1 chunks with approx. 300 frames each.
Total size will be 1075200 bytes.seems to be wrong, why 2 threads for 1 chunk? why not 2 chunks?

[708145]

detected fps = 25.00
detected frames = 300
using 2 threads to encode 1 chunks with approx. 300 frames each.
Total size will be 1075200 bytes.seems to be wrong, why 2 threads for 1 chunk? why not 2 chunks?

nope it's correct. 1 thread will just sit idle.
if there are too few frames ELDER will use fewer chunks.

edit: there are 2 defaults in the code: preferred chunksize (2000 frames) and minimum chunksize (350 frames).

bis besser,
T0B1A5

[bond]

ic, i now tested it with 1000 frames (1 core only) and the output only had 999 frames!

[708145]

ic, i now tested it with 1000 frames (1 core only) and the output only had 999 frames!

silent update to version beta2b

bis besser,
T0B1A5

[bond]

silent update to version beta2bfixed the framenumber problem :)

[708145]

new release :)

beta3:
* parallelism fixed

bis besser,
T0B1A5

[Revgen]

I tried this version with my AMD Athlon X2 4600+ and ActivePerl 5.8.7 Build 813 with -m set to 2 and the video encoded fast but not right.

My clip is 40 sec. long but ELDER only encoded 19 sec.

I'll do some more tests later today when I have more time.

[708145]

I tried this version with my AMD Athlon X2 4600+ and ActivePerl 5.8.7 Build 813 with -m set to 2 and the video encoded fast but not right.
My clip is 40 sec. long but ELDER only encoded 19 sec.
I'll do some more tests later today when I have more time.

Thanks for testing.

The mp4box version that ships with beta3 is buggy and does not mux correctly in some/many cases. If anyone knows a newer + more stable build please point me to it.

Plus I fixed a few other bugs in beta4... release soon: I'll just include latest mp4box, encraw and x264 and there you go. No need to test an ancient version of ELDER :P

bis besser,
T0B1A5

[708145]

just released beta4a of ELDER4XVID. The x264 version is right around the corner as well.

[blubberbirne]

Hi Tobias, alter Namensvetter :)

I love your Encoder :)

Image (http://img397.imageshack.us/img397/5058/elderx264beta3test4gd.jpg)

Speed is good :) With "normal" settings i get ~25fps, thats best score ever on my computer (X2 3800+)

Can't wait so see next Beta

[708145]

Hi Tobias, alter Namensvetter
Can't wait so see next Beta

Hey blubberbirne,

you didn't have to wait long it seems :) there is a new beta out NOW.
beta4b uses the new framework of ELDER. If all goes well there will be many more betas in the next couple of weeks (next in queue is an xvid crf mode and linux cluster mode)

bis besser,
T0B1A5

[blubberbirne]

wow,

i'm going to test it.

hope you write a gui in the near future. Creating batch files will coast a lot of time *g*

[708145]

hope you write a gui in the near future. Creating batch files will coast a lot of time *g*

Why is everybody asking for a GUI? :rolleyes:
It is on the TODO for a long time but I considered other things more important (like make it work).
But since this is the case now, I'll think about a GUI when I have some spare time. Maybe someone wants to contribute?

bis besser,
T0B1A5

[blubberbirne]

This batch doesn't work.....

perl ELDER4X264_start.pl -a G:\cut\Bettina_Cramer_Blitz-05_02_06.avs -m 2 -b 1150 -o G:\cut\Bettina_Cramer_Blitz-05_02_06.mp4 -q " --level 5 --ref 3 --mixed-refs --bframes 3 --b-pyramid --b-rdo --weightb --subme 7 --trellis 1 --analyse p8x8,b8x8,i4x4"

I get 0kbit bitrate. Filesize for a 2min clip is ~600kbit :helpful:

And here the same.......

perl ELDER4X264_start.pl -a G:\cut\Bettina_Cramer_Blitz-05_02_06.avs -m 2 -b 1150 -o G:\cut\Bettina_Cramer_Blitz-05_02_06.mp4 -q "--ref 10 --mixed-refs --bframes 3 --b-pyramid --b-rdo --weightb --filter -2,-1 --subme 7 --trellis 1 --analyse all --8x8dct --progress --no-psnr"

I think, there is a bug in your new version

EDIT:

http://img387.imageshack.us/img387/8877/bug5ph.jpg

[Kostarum Rex Persia]

Well, GUI is much more easy for everyday user.

708145, how much you can optimize(to the limit) encoding speed for dual-core machines? 5-10 %, perhaps. And, do you planning to introduce SSE3 support in your x264 parallel encoder?

Thanks for your answers.

[blubberbirne]

Now i leave the -q parameter an it works? Very interresting. Looks like beta4a is very buggy. wiht your old beta it works perfect.

[LRN]

For GUI, you should contact to developers of MeGUI, StaxRip and other frontends.

[blubberbirne]

Of course, it would be very nice, if megui can handle your decoder :)

I found the error while it don't work: i relplaced the x264 in you old beta with a newer one. this one can handle more settings. setting which youre release don't handle.

So, everthink works fine :)

[KWAGGA]

Hi there Guys! Yes I believe a Gui would be a great idea for this encoder. I am not scared of cmd line app, but for all the settings ect it would be sweet to do it via a gui! Maybe I will try making a Java Interface :) (gimme +5 years though *<>* )

Kw

[ndkamal]

What is the gain of performance with this encoder ?

[foxyshadis]

Since it's just a multi-pass command-line, it makes perfect sense to integrate it into MeGUI (once that settles down), instead of wasting a lot of time making a half-assed gui for it, no offense to Tobias. By the time they're done you should just be able to design a form with all the options, set it as a 3 or 4-pass job, and let it rip. No worrying about all the rest of the GUI messiness.

Mr. Rex, I don't think you can code SSE3 in Perl. ;)

Oh yeah, just in time for dual-core laptop coming tomorrow, sup.

[708145]

What is the gain of performance with this encoder ?

There are some speedup figures (up to 88%) in the xvid thread. Speedup is very likely the same with beta4b in both xvid and x264 mode.
Have a look here:
http://forum.doom9.org/showthread.php?p=780648#post780648

bis besser,
T0B1A5

[bkman]

Do I understand correctly that you avoid the discrepancy of joining separately encoded chunks? But the output stream isn't bit identical to the standard encoder in single thread mode, is it.

[708145]

Since it's just a multi-pass command-line, it makes perfect sense to integrate it into MeGUI (once that settles down), instead of wasting a lot of time making a half-assed gui for it, no offense to Tobias. By the time they're done you should just be able to design a form with all the options, set it as a 3 or 4-pass job, and let it rip. No worrying about all the rest of the GUI messiness.

Mr. Rex, I don't think you can code SSE3 in Perl. ;)
.

Yeah, adding it to MeGUI would be great. But I prefer to handle multi-pass in ELDER since I'm about to fix the resume feature which enables to cont. partial encodes with little overhead :D
MeGUI can't do that (yet).

@Mr. Rex: No need for SSE3 in the perl script. We are talking about optimizing a couple of ms! And as ELDER uses all existing codec optimizations it is real fast anyway.
A better optimization is an improved load balancing in 2nd pass which will come soon. Still having the 1.95x speedup on X2 in mind :p

bis besser,
T0B1A5

[708145]

Do I understand correctly that you avoid the discrepancy of joining separately encoded chunks? But the output stream isn't bit identical to the standard encoder in single thread mode, is it.

Not bit identical but if you look at the bitrate distribution throughout the video it is _almost_ the same.
The difference stems from the fact that a normal 2pass always has the chance to correct bitrate over/undershoot in the following frames. ELDER does the same _except_ on chunk borders! Hence a small difference.

The larger the chunks the smaller the difference will be.
The smaller the chunks, the better the load balancing and thus speedup.

I'm inventing a clever way to find an adaptive tradeoff right now :D But current ELDER defaults ain't that bad, IMO.

bis besser,
T0B1A5

[MeteorRain]

can this program be extended to a LAN with lots of computers encoding parallelly?

[708145]

can this program be extended to a LAN with lots of computers encoding parallelly?

Of course since this is the reason I started ELDER development.
The first ELDER version could do so but scaled poorly (~11x speedup on 24nodes)

But I will not do a windows cluster mode. Cluster mode will be linux only.

bis besser,
T0B1A5

[MeteorRain]

Of course since this is the reason I started ELDER development.
The first ELDER version could do so but scaled poorly (~11x speedup on 24nodes)

But I will not do a windows cluster mode. Cluster mode will be linux only.

bis besser,
T0B1A5
囧 only support linux?
BTW, is it possible to hack the x264 core and make it a C/S mode encoder? whose server is a AVS Reader + mp4/mkv/raw Writer, while the clients are the encoder thread?
in local mode, it works in namepipe, and in LAN, works on TCP/IP network, just like MySQL works?

;) just my thought :)

[Revgen]

I tried using the new version with more advanced settings than I used the last time, and I ran into probelems.

Using this script:

perl ELDER4X264_start.pl -a E:\3s-Goo~1\goofs2.avs -f 16501 -m 2 -b 1500 -c 4 -o E:\3s-Goo~1\goofs.mp4 -q "--pass2 --keyint 240 --stats ".stats" --ref 5 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --subme 6 --analyse all --8x8dct --me umh --merange 20"

Gave me a 4.38 MB file for an 11minute 38 second video and the results are awful. Obviously, it's not encoding at 1500 kbps.

Is there a reason why?


EDIT: Okay I eliminated --pass2, --stats, and the --mixed refs parameters and the video doesn't look awful anymore. Unfortunately the muxing bug from the last version still remains. My file is 5:30 instead of 11:38.

This doesn't seem to be an mp4box problem, since ELDER4XVID doesn't have this issue.

[Hellworm]

But I will not do a windows cluster mode. Cluster mode will be linux only.

So, is the hole cluster going to be linux or is the server windows (because of avisynth)?
What will be streamed over the network, the already decoded raw video or the undecoded stream ?

[blubberbirne]

Most parameters of x264 doens't work :(
But, this paramaters are a must have.

Deklocking parameter is one of them

[708145]

EDIT: Okay I eliminated --pass2, --stats, and the --mixed refs parameters and the video doesn't look awful anymore. Unfortunately the muxing bug from the last version still remains. My file is 5:30 instead of 11:38.

This doesn't seem to be an mp4box problem, since ELDER4XVID doesn't have this issue.

The muxing command is almost the same (excpept for .264 and .m4v file extensions of the chunks) => I really think it is mp4box.
Maybe it handles different raw streams / different extensions differently.

As for the "missing" options: I know what goes wrong and it is #1 on the TODO :D

bis besser,
T0B1A5

[708145]

So, is the hole cluster going to be linux or is the server windows (because of avisynth)?
What will be streamed over the network, the already decoded raw video or the undecoded stream ?

Finally it will be avisynth3 on linux all the way. As a prototype I might go for 2.56 under wine. Either way it's the undecoded stream that is fed to the clients.

bis besser,
T0B1A5

[bond]

Unfortunately the muxing bug from the last version still remains. My file is 5:30 instead of 11:38.

This doesn't seem to be an mp4box problem, since ELDER4XVID doesn't have this issue.is everything in the created .mp4? are no parts missing? is the framerate correct?

when muxing raw .264 to .mp4 with mp4box you always need to set a framerate, otherwise mp4box will use 25fps

[Revgen]

Half of the "chunks" aren't muxed properly for some reason, so the file is cut in half. The video plays fine except for the missing parts, so it isn't a framerate issue.

[bond]

what do you mean by "chunks"? are frames missing?

[Revgen]

ELDER divides the video into "chunks" while encoding, which allows the encoding to be more balanced between the 2 cores when using multiple threads. Once these chunks are finished they are merged into one video. For some reason these chunks aren't being muxed properly. So, yes, frames are missing from the video.

[bond]

so elder propably feeds mp4box with a wrong or incomplete muxing command so it doesnt mux all of the chunks

btw are there more than one video stream in the mp4 created by elder?

[708145]

so elder propably feeds mp4box with a wrong or incomplete muxing command so it doesnt mux all of the chunks

btw are there more than one video stream in the mp4 created by elder?

Gonna print the mux command in the next beta for your convenience. Then you can tell me if the command line is wrong or mp4box is misbehaving.

bis besser,
T0B1A5

[Revgen]

Wow you're a genius!

There is indeed 2 video streams in the file. I didn't check for that.

Both streams look good. Here is how they are separated:

Sream 1 - Beginning of video, end of video

Stream 2 - First middle part of video, second middle part of video

I used 4 chunks when encoding so this is how they are separated.

[bond]

thats because for some reason mp4box recognized those streams to not use the same settings
its only allowed to merge streams which use the same settings, i think mp4box checks the SPS/PPS for that

maybe you can extract the raw streams from the mp4 and run the h264_parse tool from mpeg4ip over them
this will show you the structure of the headers (SPS/PPS) and if they are identical

[Revgen]

I ran h264_parse over both of the the streams. Stream 1 took a while and I was only able to copy the last of what was done. Stream 2 took a very short time amd ended with a message saying "couldn't find start code in buffer from 0"

[ChrisBensch]

After trying to get ELDER to work for quite some time, I've had no luck:

"perl ELDER4X264_start.pl -a D:\$VIDEO_WORK$\WORKING\test-elder.avs -m 2 -b 4000 -c 4 -o c:\temp\test-elder.mp4 -q "--bframes 3 --b-pyramid --filter -2,-1 --subme 1 --analyse none --me dia --progress --no-psnr""

I did create some batch files from the CLI provided by MeGUI and tried two simultaneous instances of x264 and I got about a 50% increase in speed (~8.5fps with MeGUI with two threads to 13fps with the batch tests). I'm really interested in ELDER because of this increase since most of my encodes are HD sources.

I've tried a whole bunch of times with various settings. I copied the CLI output from MeGUI and nothing seems to work. I've attached a picture of my command window in hopes that maybe someone can see something I'm doing wrong

[Revgen]

Try this and see if it works. Please don't ask why.;)

"perl ELDER4X264_start.pl -a D:\$VIDEO~1\WORKIN~1\test-~1.avs -m 2 -b 4000 -c 4 -o c:\temp\test-e~1.mp4 -q "--bframes 3 --b-pyramid --filter -2,-1 --subme 1 --analyse none --me dia --no-psnr"

[ChrisBensch]

Unfortunately, it didn't help, I get stuck at the same point. It appears to be stopping right when the "b" perl script is being called:

ex: perl ELDER4XVID_b.pl 776519

Something else to note, it's not detecting my FPS from the AVS script:

detected fps = format
detected frames = 2001

[Revgen]

Are there actually 2001 frames in your video, or is that a misread by ELDER?

2001 frames are too short for using 4 chunks. Try increasing it to about 8000 or even 16000 as "708145" suggests.

If it's a misread than use the -f numberofframesinyourclip before the -q "" parameter starts.

EG: your clip is 16000 frames then use:

"perl ELDER4X264_start.pl -a D:\$VIDEO~1\WORKIN~1\test-~1.avs -m 2 f- 16000 -b 4000 -c 4 -o c:\temp\test-e~1.mp4 -q "--bframes 3 --b-pyramid --filter -2,-1 --subme 1 --analyse none --me dia --no-psnr"

[ChrisBensch]

Yes, the 2001 frames is correct, but a mistake on my part. I had shrunk the frame amount for other testing. I'll increase the frame count to 16000 and try again. By the way, my setup is that on one hard drive I had elder then on another I had all of my video tools. I had to move the AVS script into the same dir and make an absolute reference to my .d2v within that script as things are on separate drives. By the way, the output from the earlier posted setup was terribly blocky and you could almost not even tell what the video was.

Another try coming...

[Revgen]

By the way, the output from the earlier posted setup was terribly blocky and you could almost not even tell what the video was.

I had this problem too but I solved it by removing --pass2 --stats and --mixed-refs

Perhaps there are other parameters that cause problems for ELDER. The only parameters that you are using that I didn't use was "--no-psnr" and "--filter". Perhaps these are causing problems.

[ChrisBensch]

Another try in progress, instead of moving and editing the AVS to have an absolute path to the .d2v, it just needed the absolute path (mpeg2source("d:\videowork\test.d2v") not (mpeg2source("test.d2v"). Seems to be working fine so far.

Edit: The job has finished. Using MeGUI on the HQ-Slow preset, I would get about 8.5fps on the first pass and then about 3.5fps on the seconde with my HD sources on my 2.8GHz dual core. With the same source and AVS, I'm getting about 14.3fps (with 2001 frames, redoing with 16K frames) with ELDER for the one pass. I would however, like to use two-pass like MeGUI but am unsure as to exactly how I would do that. I am just copying the cli from MeGUI for testing so that I have at least a somewhat reasonable comparison between the two.

[ChrisBensch]

One thing that would be nice is if you were somehow able to capture the average FPS for the whole process without having to sit and watch the output windows. Maybe write it to a file for info purposes?

[Revgen]

Is your encode the right length?

[ChrisBensch]

Before I go any further, I'm gonna sleep on it. My eyes are crossing as I've been up way to long to be doing this tonight. All I can say is ELDER is very interesting and I'll be back at it tomorrow. Thanks to all.

[bond]

I ran h264_parse over both of the the streams. Stream 1 took a while and I was only able to copy the last of what was done. Stream 2 took a very short time amd ended with a message saying "couldn't find start code in buffer from 0"i need the sequence and picture parameter at the beginning of h264_parses output

[Revgen]

i need the sequence and picture parameter at the beginning of h264_parses output

Okay I have the beginning sequence for Stream1.

I had to go into safemode command prompt form of windows to do it. My machine is way too fast in normal mode, hence their was no chance to pause the operation and copy anything.

[bond]

thx but we would need both for comparing if the streams use the same settings

btw passing the output of h264_parse to a txt is very easy via the following cmdl:
h264_parse raw.264 > h264_log.txt

[Revgen]

The stream2.txt in my previous post (post #65) has the beginning info for second stream.

Only stream1.txt (in the previous post) was cut short.

[bond]

hm they are identical, except the bitrate in the SEI, tough i dont think that mp4box checks the SEI

actually i have no idea what could be wrong on the mp4box side

maybe elder uses not -cat everywhere but -add maybe?

[foxyshadis]

No, everything is -cat, it's just one tiny loop over all pieces.

708145, is it really necessary to have so many "$pathtoencraw =~ s/\//\\/g;" type lines? Windows understands forward slashes fine.

[708145]

No, everything is -cat, it's just one tiny loop over all pieces.
708145, is it really necessary to have so many "$pathtoencraw =~ s/\//\\/g;" type lines? Windows understands forward slashes fine.

I never tested :shame:
I'll delete them then :)

[foxyshadis]

So I was scoping out the perl and found a few things. You might want to change:

$tmpdir=int(rand(1000000));

to

$tmpnum=int(rand(1000000));
$tmpdir="/var/tmp/".$tmpnum; # /tmp? /home/tmp? dunno.
#WINDOWS#
$tmpdir="%temp%\\".$tmpnum;
#WINDOWS#

(mkdir is one instance where / doesn't work). That way there's no write-permission issues. And also check for existence of the avs before going on to encode nothing (it hangs, I'm not quite sure where though). I'm tinkering with it to see if I can replicate revgen's issues.

[708145]

So I was scoping out the perl and found a few things. You might want to change:

$tmpdir=int(rand(1000000));

to

$tmpnum=int(rand(1000000));
$tmpdir="/var/tmp/".$tmpnum; # /tmp? /home/tmp? dunno.
#WINDOWS#
$tmpdir="%temp%\\".$tmpnum;
#WINDOWS#

(mkdir is one instance where / doesn't work). That way there's no write-permission issues. And also check for existence of the avs before going on to encode nothing (it hangs, I'm not quite sure where though). I'm tinkering with it to see if I can replicate revgen's issues.

good idea. but I have to change the pathto.. variables then as well to absolute paths.

maybe setting up a svn would be wise as well. this way motivated people like you can contribute directly :)

[Revgen]

Okay, I've tried ELDER on a different movie than what I was using earlier. And the encode turned out fine this time with no muxing issues. Very strange:confused:

I'm going to try to see if I can replicate the issue with some other sources.

[ChrisBensch]

Edit: After this was finished, I checked the video and it is extremely blocky and I'd say 100% unwatchable...

I've done a very small scale test to see the speed increase I can come to expect and I'm pretty impressed. I've posted the AVS and CLI I used below. The HDTV source is 1920x1080 that was capped from a DVR. I'm seeing a speed of about 25.12fps for the first pass and 14.10fps for the final pass with 100% cpu util with ELDER. As I stated in earlier posts, I was seeing a first pass of 8.5fps and second pass of 3.5fps @ 80% cpu util with MeGUI/x264.

Here is my AVS:

LoadPlugin("D:\$VIDEO_WORK$\bin\AviSynth Plugins\DGDecode.dll")
LoadPlugin("D:\$VIDEO_WORK$\bin\AviSynth Plugins\RemoveGrainSSE3.dll")
MPEG2Source("D:\$VIDEO_WORK$\WORKING\The Matrix Reloaded-HDTV.d2v")
Crop(2,2,-2,-6)
LanczosResize(1280,608)
RemoveGrain(mode=2)
Trim(20000,23999)

Here is the command line I used:

perl ELDER4X264_start.pl -a D:\$VIDEO_WORK$\WORKING\test-elder1.avs -m 2 -b 4000 -c 4 -o test-elder.mp4 -q "--keyint 240 --ref 5 --bframes 3 --b-pyramid --weightb --subme 6 --trellis 1 --analyse all --8x8dct --progress"

[Revgen]

Try removing --progress and see what happens

[ChrisBensch]

Just for the hell of it, I'm trying a different build of x264. The version included with ELDER is build 400 svn, I'm sub'ing it with build 397 from sharktooth.

Also, I noticed that during the encoding stages, the cli looks like this:

..\x264.exe --progress -B 0 -o test3.264 stageC-3.avs

I don't see any of the special options I passed to it on the cli???

[ChrisBensch]

The only difference of note when changing x264 builds is that the file was 80KB larger...

I'll try removing "--progress" and see what happens

[Revgen]

The "..\x264.exe --progress -B 0 -o test3.264 stageC-3.avs" is just a temporary stage. It is used in my encodes too.

It doesn't hurt to try another version of CLI, but IMHO the problem lies with ELDER.

[708145]

..\x264.exe --progress -B 0 -o test3.264 stageC-3.avs
I don't see any of the special options I passed to it on the cli???

My bad. I just noticed I did not rename the variable holding the passed settings everywhere. Thus the 2nd pass options don't get used :/
Expect a bugfix release in a few days, there is a lot to fix :p

edit: this also explains the extreme blockyness some of you experienced.

[ChrisBensch]

The "--progress" option was killing the video. Now my 4000 frame clip is ~80MB...and the quality is what I expect.

Edit: The "--no-psnr" option works fine too...as long as it's not the last option on the cli..
--no-psnr --8x8dct" worked where --8x8dct --no-psnr" did not work.

[ChrisBensch]

I was wondering what was going on...

My bad. I just noticed I did not rename the variable holding the passed settings everywhere. Thus the 2nd pass options don't get used :/
Expect a bugfix release in a few days, there is a lot to fix :p

edit: this also explains the extreme blockyness some of you experienced.

[ChrisBensch]

When I run my 23.976fps file through VLC, the Stream and Media Info window shows that my Frame rate is 25.000000...is that a muxing issue?

[Revgen]

It might be.

See bond's explanation (post #57) for more info.

[ChrisBensch]

gotcha!

[blubberbirne]

-q " --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --weightb --subme 7 --analyse all --me esa --no-psnr --8x8dct "

this option does not work for me. If i removed the --no-psnr parameter, everythink works fine ?!?!

[708145]

-q " --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --weightb --subme 7 --analyse all --me esa --no-psnr --8x8dct "

this option does not work for me. If i removed the --no-psnr parameter, everythink works fine ?!?!

did you try a newer x264 build. the one shipped is svn400.

[Revgen]

-q " --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --weightb --subme 7 --analyse all --me esa --no-psnr --8x8dct "

this option does not work for me. If i removed the --no-psnr parameter, everythink works fine ?!?!

I would suggest removing --mixed-refs. It gave me problems as detailed in post #52. Also remove the --no-psnr parameter, just to see what happens. If that doesn't work than maybe a different x264 build will work.

[Revgen]

I've decided to do a test and comparison using the movie I described in post #85. I used the exact same settings for both.

Test Settings:

For ELDER: perl ELDER4X264_start.pl -a H:\BLAZING_SADDLES\VIDEO_TS\saddles.avs -m 2 -b 1500 -c 4 -o H:\saddle.mp4 -q "--keyint 240 --ref 5 --bframes 3 --b-pyramid --b-rdo --bime --weightb --subme 6 --analyse all --8x8dct --me umh --merange 20"

For X264.exe: x264.exe --bitrate 1500 --keyint 240 --ref 5 --bframes 3 --b-pyramid --b-rdo --bime --weightb --subme 6 --analyse all --8x8dct --me umh --merange 20 --threads 2 --progress --output "H:\BLAZING_SADDLES\VIDEO_TS\saddles.mp4" "H:\BLAZING_SADDLES\VIDEO_TS\saddles.avs"


Test Results:


12.11 FPS W/ ELDER -m1 -c4
22.36 FPS W/ ELDER -m2 -c4 - About an 85% improvement over -m1 -c4

4.80 FPS W/ x264.exe ABR 1 thread
8.37 FPS W/ x264.exe ABR 2 threads - 74% improvement over ABR 1 thread


Quick Summary:

1) ELDER doesn't use the options that were given in the test settings. See the .txt attachments for more details.
2) The ELDER videos play back at 25 FPS while the x264.exe encodes playback at 23.976. This is probably the MP4Box issue that bond talked about in post #57.
3) The ELDER videos are 11min 0 secs while the X264.exe videos are 11:28 secs. The ELDER videos are shorter than they should be. This may be because of the 25 FPS playback issue.
4)The quality of the x264.exe encodes are better the ELDER videos. Possibly having to do with summary item #1 above.
5)The ELDER encodes are 128mb while the x264.exe encodes are 121mb.
6) It seems that ELDER scales better than x264.exe. But we can't be sure since it isn't using all of the options.


Extended Summary:

ELDER is pretty fast but that's because it isn't using all of the options. For example if you look at the .txt attachments created by h264_parse you'll see that ELDER used "subme 5" instead of the "subme 6" that I orginally specified. There are other options that aren't used either. Consequently the videos doesn't look bad, but just not as good as the x264.exe encodes. I have no idea why the ELDER videos are larger than x264.exe's encodes. It could possibly be due to summary item #1 above.

Conclusion:

It seems that ELDER scales well (compared to -m 1), but still has bugs and issues to iron out. Once these issues are ironed out I think we'll still see 80% to 90% scaling improvements or possibily more.

[foxyshadis]

I don't think it makes any sense to do these tests on it with several known killer bugs, when a bugfix release should be coming shortly. Otherwise, of course results are going to be thrown way off.

[blubberbirne]

did you try a newer x264 build. the one shipped is svn400.

yes, i tried lastet one pack which come with megui, but it doesnt work

[Revgen]

I don't think it makes any sense to do these tests on it with several known killer bugs, when a bugfix release should be coming shortly. Otherwise, of course results are going to be thrown way off.

I'm testing it so that it can be improved. This program is a beta after all.

On top of that, 708145 asked us to do these tests in his first post.

[ChrisBensch]

Edit: I noticed that I didn't include the stats file, when trying to use --stats "c:\temp\elder\154754\stageC-0.stats" in the CLI, x264 complains giving an error:

x264 [error]: ratecontrol_init: can't open stats file
x264_encoder_open failed

I'm not sure where to go next, I guess it's just time to wait for the next beta.



Original:

After messing around a bit, I decided to stop ELDER just when it starts to encode the first two chunks. Then I went into the stageC batch files and changed the x264 command line from the broken one to:

..\x264.exe --progress --keyint 240 --ref 5 --bframes 3 --b-pyramid --weightb --filter -2,-1 --subme 6 --trellis 1 --analyse all --8x8dct --no-psnr -B 4368 -o test0.264 stageC-0.avs

I then launched the batch files two at a time (selected two in Windows explorer and then hit the <ENTER> key, launched them both). I have to say I am a bit disappointed, both encodes ran fine but the speed was exactly the same as if it were using MeGUI. The first passes ran mighty quick, but the second pass was the same. Overall it's a speed up, but the second pass was the one I was hoping would have the most improvement.

[blubberbirne]

@708145

Hope you can get all x264 parameter working. i see big picture differents, because your script doesn't like all settings :(

[708145]

@708145

Hope you can get all x264 parameter working. i see big picture differents, because your script doesn't like all settings :(

beta4c is out :D

please check if parameters work and if the muxing goes all right.
if the quality is constant and it hits target size in the ELDER encodes then I can move along to
* the linux port
* better scaling with more CPUs
* improved rate control for xvid
* GUI

[Revgen]

I just tried it.

There seems to be a problem with Stage C. It seems to take about 5 secs while stage A takes about 15 min.

And sure enough, after the encode is done, there is no MP4 file in the directory where I specified with the -o command.

Here's my script

perl ELDER4X264_start.pl -a H:\BLAZING_SADDLES\VIDEO_TS\saddles.avs -m 2 -b 1500 -c 4 -o H:\saddle.mp4 -q "--keyint 240 --ref 5 --bframes 3 --b-pyramid --b-rdo --bime --weightb --subme 6 --analyse all --8x8dct --me umh --merange 20"

[708145]

There seems to be a problem with Stage C. It seems to take about 5 secs while stage A takes about 15 min.


after Stage A completes there should be some debug output. Could you paste this please. I think there are some wrong values (maybe zeroes) in the size array.

edit: I figured it out. It's my stupidity when reading the command line. I will fix it tomorrow.

[Revgen]

Debug Output?

Can you give me a hint? :confused:

EDIT:

Is this a new command that isn't in the Readme.txt?

[blubberbirne]

beta4c is out :D

please check if parameters work and if the muxing goes all right.
if the quality is constant and it hits target size in the ELDER encodes then I can move along to
* the linux port
* better scaling with more CPUs
* improved rate control for xvid
* GUI

sounds good :thanks:

i'll test later when i'm back from work :p

[708145]

Debug Output?
Can you give me a hint? :confused:

EDIT:
Is this a new command that isn't in the Readme.txt?

there are output lines starting with "debug:" in beta4c. But those values seem to be correct. The fault is in the command line parsing :rolleyes:

I'll update this. Hang on for beta4d. :)

bis besser,
T0B1A5

[blubberbirne]

perl ELDER4X264_start.pl -a G:\cut\x264test.avs -m 2 -b 3400 -c 6 -o G:\cut\x264test.mp4 -q " --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --weightb --filter -2,-1 --subme 7 --trellis 2 --analyse all --8x8dct --me umh --progress --no-psnr "

This one doesn' work

[708145]

This one doesn' work

As a quick workaround I suggest to change the defaults in ELDER4X264_acrf.pl to your settings and don't specify -p and -q on the command line.

If this is too much hassle for you then I suggest to wait for 4d.

[blubberbirne]

If this is too much hassle for you then I suggest to wait for 4d.

Ok, waiting for 4d :)

[Revgen]

Okay, I changed the default settings in the ELDER4X264_acrf.pl file to the ones I posted earlier that didn't work.

The video turned out fine. No muxing issues. Quality is much better. Video plays back at 23.98FPS in VLC player instead of 25.000FPS like last time. The length of the video is also correct.


However, I will wait to do speed tests and quality comparisons once the 4d version is released.

[708145]

The video turned out fine. No muxing issues. Quality is much better. Video plays back at 23.98FPS in VLC player instead of 25.000FPS like last time. The length of the video is also correct.

However, I will wait to do speed tests and quality comparisons once the 4d version is released.

Great! :)

I'm cleaning up the code a bit and reimplement parameter passing from scratch.

The quality tests are of prime importance since the scaling I can do myself since for the same -c value the output is bit identical independent of CPU count.

bis besser,
T0B1A5

[Revgen]

Okay I tried my goofs video clip, and this one still does have muxing issues. I'm not sure why though.

The video quality, framerate, etc are fine though.

[Revgen]

I decided to take a closer look at how my clips are being encoded to see why one muxes correctly and the other muxes incorrectly, and I found something interesting:

Goofs (unsucessfull mux):

Finishes 1st: Stage A-0
Finishes 2nd: Stage A-1
Finishes 3rd: Stage A-3 *
Finishes 4th: Stage A-2 *

Finishes 5th: Stage C-0
Finishes 6th: Stage C-1
Finishes 7th: Stage C-3 *
Finishes 8th: Stage C-2 *

Saddles (sucessfull mux):

Finishes 1st: Stage A-0
Finishes 2nd: Stage A-1
Finishes 3rd: Stage A-2
Finishes 4th: Stage A-3

Finishes 5th: Stage C-0
Finishes 6th: Stage C-1
Finishes 7th: Stage C-2
Finishes 8th: Stage C-3

*It seems to me what differentiates the Saddles clip from Goofs clip is that Saddles finishes all of it's stages sequentially while Goofs is insequential on the 3rd and 4th stage. Although ELDER does wait for the 3rd stage to finish before muxing or passing on to stage C, there might be a issue with the way the stages are stored that may be causing problems with muxing.

[708145]

*It seems to me what differentiates the Saddles clip from Goofs clip is that Saddles finishes all of it's stages sequentially while Goofs is insequential on the 3rd and 4th stage. Although ELDER does wait for the 3rd stage to finish before muxing or passing on to stage C, there might be a issue with the way the stages are stored that may be causing problems with muxing.

I'll try to reproduce this. I'm quite sure the muxing command line still is correct. But mp4box might use file timestamps to determine muxing order instead of parameter order? :confused:

[Revgen]

Any luck yet?

[708145]

Any luck yet?

Sorry I was held back by real life and work. Gonna try tonight :)

[elfurbe]

Following the thread. Just wondering, you mention a Linux software for DVD converting on your site, but most of the development seems to be on this Windows project. I'm currently putting together a Linux video encoding cluster for academic purposes, and I'm very interested in your project for doing parallel x264 encodes from high-def MPEG2. Is there a current Linux version I can try out or is that coming after the Windows version works?

[708145]

Following the thread. Just wondering, you mention a Linux software for DVD converting on your site, but most of the development seems to be on this Windows project. I'm currently putting together a Linux video encoding cluster for academic purposes, and I'm very interested in your project for doing parallel x264 encodes from high-def MPEG2. Is there a current Linux version I can try out or is that coming after the Windows version works?

Good to hear that :)
The original linux project will not be developed further. But the port of the current ELDER code to linux happens via wine at the moment. When avisynth is available native for linux then the wine dependency will fall :D

I'll release the linux port when I think it works good enough to bother others with testing :p

What you could do now is set up an NFS so that all clients see a common directory. Further you'll need wine with avisynth 2.5.6 installed. Add all the avs filters you need as well. All the rest will ship in the linux package. :)

I'm really looking forward to testing on a cluster myself and tweaking scalability there since we have a very heterogeneous cluster wrt. speed (slowest node is ~10x slower than fastest node). But this speedup tweaking is a hobby of mine :)

bis besser,
T0B1A5

[Revgen]

How's the project going so far?

Were you able to detect the problem?

[708145]

How's the project going so far?
Were you able to detect the problem?

I located the muxing problem and the fix is underway... but no more ELDER today...

[Revgen]

Thanks. I won't bother you anymore about it.:D

[Revgen]

@708145

Have you looked into possibly using mp4creator from the mpeg4iptools suite instead of mp4box. It's harder to find, but it could be more stable than mp4box. I have the April 23, 2005 build and can send it to you if you're interested.

[Revgen]

Here's a quote from forum post http://forum.doom9.org/showthread.php?p=829027#post829027


A not so short note first:
ELDER is capable of just using 4 chunks, just use the -c option to specify number of chunks :p
More chunks is there to improve load balancing on several cores since the segments differ in encoding speed your total time would otherwise depend on the slowest part! More segments help with resume as well, aka shutdown during encode and continue later (maybe after some gaming session or TV capture or holidays or...)



A complete 1st pass was my pick which is the most accurate and reasonably fast.
But even when you got relative difficulty of the chunks how would you encode them without a stats file? I hope CBR is out of the question.

About muxing: I solved the muxing issue I had which had to do that mp4box used the timestamps of the chunks instead of the order given in the command line! :S

One open issue:
If someone wants to contribute a GUI for ELDER I can spend my time on the advanced rate control modes I plan like an CRF like mode for xvid and the high fps mode called xvid50p which is optimized for 50p and 60p encodes more and more people out there are doing. At same quality 50p consumes just ~20% more than 25p using xvid50p! So it's time for fluid motion!

bis besser,
708145

Sounds good. I like to encode my 720p (60p) HDTV caps alot since I've gotten it. This should make it easier to encode.

I'm awaiting the new release.;)

[havix]

Is anymore work being done on this? Also why was it decided that only Linux would have cluster support? This seems rather shortsighted since most people that have multiple machines will be Windows based.

[708145]

Is anymore work being done on this? Also why was it decided that only Linux would have cluster support? This seems rather shortsighted since most people that have multiple machines will be Windows based.

I plan to get back on this project... when time permits.

I have nothing against a windows cluster mode. I just stated that I won't implement it but I'm open to contributions :)

bis besser,
708145

[Sharktooth]

it's getting its way into megui... :)

[ChrisBensch]

I can't wait...I have 4 machines at home waiting to crunch through my HD captures!

[tomos]

it's getting its way into megui... :)

yay :thanks:

will be so much faster encoding with this :)

[Revgen]

I plan to get back on this project... when time permits.

I have nothing against a windows cluster mode. I just stated that I won't implement it but I'm open to contributions :)

bis besser,
708145

I'm still here to test if you need me.:D I still have the same movie that didn't mux right the last time.

[708145]

I'm still here to test if you need me.:D I still have the same movie that didn't mux right the last time.

Thanks for all the testing.

I'm not sure how to proceed with ELDER4X264. It might go commercial.
I'll get the last release bugfixed nevertheless.

ELDER4XVID will remain in GPL-land and I will add SNOW and Theora encoding shortly :D Maybe someone can convince me to support Dirac. :p
Per hour I work on a commercial product I'll spend some time on the GPL side.

bis besser,
T0B1A5

[Sharktooth]

well... x264 is not a commercial product though...

[708145]

well... x264 is not a commercial product though...

The good thing about GPL is that the current state of the code will stay GPL. And I might even add some encoding modes later, but the ELDER fork has priority for me.

I'm not sure how to proceed with ELDER4X264. It might go commercial.
I'll get the last release bugfixed nevertheless.


So here it is: the bugfixed mux-enabled version of ELDER4X264.
Grab your copy of beta4d here:
http://www.funknmary.de/bergdichter/projekte/index.php?page=ELDER

changelog for beta4d:
* fixed muxing to mp4
* split the project from ELDER4XVID

bis besser,
T0B1A5

[bkman]

Tobias, do you think you can support single-pass (crf) parallelisation in Elderx264?

[708145]

Tobias, do you think you can support single-pass (crf) parallelisation in Elderx264?

Yes. That is one of the encoding modes I was speaking about here. But I can't tell when I'll get around to that as other ELDER parts (xvid, audio, live streaming) have higher priority for me.
OTOH, it is not much effort to hack it in.

And I might even add some encoding modes later, but the ELDER fork has priority for me.

bis besser,
T0B1A5

[Revgen]

The good thing about GPL is that the current state of the code will stay GPL. And I might even add some encoding modes later, but the ELDER fork has priority for me.



So here it is: the bugfixed mux-enabled version of ELDER4X264.
Grab your copy of beta4d here:
http://www.funknmary.de/bergdichter/projekte/index.php?page=ELDER

changelog for beta4d:
* fixed muxing to mp4
* split the project from ELDER4XVID

bis besser,
T0B1A5

Great! I'll test it eventually. Perhaps this weekend.

[708145]

Great! I'll test it eventually. Perhaps this weekend.

There is even more to test: The latest beta5 series of ELDER contains x264 modes. I'll integrate a keyframe accurate 2pass mode soon.
Oh, and there is a fancy GUI as well :)

bis besser,
T0B1A5

[woah!]

i have an issue with the import() function within the made up avs scripts :(

import("default.avs")

is all it loads in and not the path to the avs? so it doesnt find the script and exits.

so my stageA-0.avs has:

import("default.avs")
trim(0,15370)


my cmd log is below:



C:\ELDER>perl ELDER4X264_start.pl -a default.avs -o ELDER.mp4 -p "-b 2 -r 1 -m 4
" -q "-b 2 -r 5 -m 4"
The system cannot find the file specified.
### encode job 363586 set up successfully
### starting encode now

C:\ELDER>perl -w ELDER4X264_acrf.pl -t 363586 -a default.avs -o ELDER.mp4 -p -b
2 -r 1 -m 4 -q -b 2 -r 5 -m 4
detected fps = 59.94
detected frames = 122958
using 4 threads to encode 8 chunks with about 15370 frames each.
Total size will be 525146 bytes.
### starting process 0
### starting process 1
### starting process 2
### starting process 3
### stage A issued

C:\ELDER>perl -w ELDER4X264_c2.pl 363586
####### Stage A Progress: 1/8 chunks done (12%) #######
####### Stage A Progress: 2/8 chunks done (25%) #######
####### Stage A Progress: 3/8 chunks done (37%) #######
####### Stage A Progress: 4/8 chunks done (50%) #######
####### Stage A Progress: 5/8 chunks done (62%) #######
####### Stage A Progress: 6/8 chunks done (75%) #######
####### Stage A Progress: 7/8 chunks done (87%) #######
####### Stage A Progress: 8/8 chunks done (100%) #######
### Stage A completed
### Stage C generating jobs
Illegal division by zero at ELDER4X264_c2.pl line 52.

C:\ELDER>perl -w ELDER4X264_d.pl 363586




i looked at that line and it has this:

$chunktarget = ($totalbytes * $chunksizes[$i]) / $total_chunksize;

then my head exploded...

[woah!]

hmmm its now working but i dont really know what i did to solve it heh...

flat lined quadcore:

http://images.dr3vil.com/files2/default/100QX.jpg

[708145]

Wow! Nice.

I decided to include x264 2pass into beta5 in one of the next releases.

bis besser,
T0B1A5

[woah!]

its doing 2pass ok right now for me?

i added some default x264 settings to your ELDER4X264_acrf.pl and it is works great with all the settings i need at 210fps total on the 1pass for this setup at 448x336 reizing.

seems i have found my tool of choice thank you :)


http://images.dr3vil.com/files2/default/210fps.jpg


heres the 2pass result of 80fps total:

http://images.dr3vil.com/files2/default/80fps.jpg

rate control seems good aswell.

[708145]

its doing 2pass ok right now for me?
seems i have found my tool of choice thank you :)
rate control seems good aswell.

Yes it was working in beta4 for a long time. But the fact is I abandoned that branch. beta4 is/was only usable to hard core users like you. :p
OTOH there were vague plans to include it into the beta5 and 6 branch again (and even better :D) with a GUI, better rate control, pause/resume feature.
I decided a week ago to actually _do_ that. So you could already try to get beta5 working on your setup since I'll add x264 2pass real soon :)

bis besser,
T0B1A5

[Al79]

Yes it was working in beta4 for a long time. But the fact is I abandoned that branch. beta4 is/was only usable to hard core users like you. :p
OTOH there were vague plans to include it into the beta5 and 6 branch again (and even better :D) with a GUI, better rate control, pause/resume feature.
I decided a week ago to actually _do_ that. So you could already try to get beta5 working on your setup since I'll add x264 2pass real soon :)

bis besser,
T0B1A5

:cool: :o :thanks: Im waiting for that beta, really, thank you very muchhhhhhhhh tobias, ur great :sly:

[PatchWorKs]

Any chance to have a theora/dirac(and, why not, snow)-enabled version ?

Those codecs needs this power !

[708145]

Any chance to have a theora/dirac(and, why not, snow)-enabled version ?

Those codecs needs this power !

I experimented with theora already. The missing part is the muxing.

[708145]

beta5f reunites x264 2pass with the main branch. See first post for a download link.
Enjoy the GUI :D

PS: x264 3pass and theora encoding will come soon (in beta6).

bis besser,
T0B1A5

[PatchWorKs]

Just a question is it open source ?

If so, i hope to see some kind of collaboration with MediaCoder (http://mediacoder.sourceforge.net/) that is planning (http://mediacoder.sourceforge.net/forum/viewtopic.php?t=1132) a distributed encoding support.

[708145]

Just a question is it open source ?

If so, i hope to see some kind of collaboration with MediaCoder (http://mediacoder.sourceforge.net/) that is planning (http://mediacoder.sourceforge.net/forum/viewtopic.php?t=1132) a distributed encoding support.

ELDER is GPL and all of ELDER has been integrated to a single package that is distributed in the announce thread (http://forum.doom9.org/showthread.php?p=718465#post718465). Current version is beta6a and does support cluster encoding to up to 128 nodes :)

I've been busy inventing my very own codec lately so I didn't spend much time on ELDER. Of course it would be good to see ELDER integrated into GUIs/frameworks since the GUI I provide is rudimentary at best. :p

bis besser,
T0B1A5

[rig99]

I'm having trouble getting this to work. All I get is this:

using 2 threads to encode 1 chunks of about 0 frames
warning: less chunks than available threads!
Illegal division by zero at functions.pl line 334.
collecting stage A results ( chunks in total)
stage A import done
stage B jobs issued
collecting stage B results (-1 chunks in total)
stage B completed
stage B imported
1st pass fps = 0
Modification of non-creatable array value attempted, subscript -1 at functions.p
l line 565.
collecting Stage C1 results ( chunks in total)
Stage C generating jobs
stage C jobs issued
collecting stage C results ( chunks in total)
stage C completed
2nd pass fps = 0
total time for encoding: 1171876680 seconds
overall fps = 0
now merging the chunks...
invalid suffix. using .mp4 instead.
"2>" ist syntaktisch an dieser Stelle nicht verarbeitbar.
chunks merged
Illegal division by zero at functions.pl line 937.


I don't understand why it thinks my video has 0 frames.

[andrewone]

Dear ELDER gurus,
sorry but I do have the same problem.

This is the default.avs script:

AviSource("e:\elder\e.avi")
ConvertToYV12()

And this is the command prompt used to star encoding:

perl functions.pl xvid -a e:\elder\default.avs -m 4 -b 800 -c 64 -o e:\elder\blafu.mp4

Then start with control.bat

The result is:

using 4 threads to encode 1 chunks of about 0 frames
warning: less chunks than available threads!
Illegal division by zero at functions.pl line 334.
collecting stage A results ( chunks in total)
stage A import done
stage B jobs issued
collecting stage B results (-1 chunks in total)
stage B completed
stage B imported
1st pass fps = 0
Modification of non-creatable array value attempted, subscript -1 at functions.p
l line 458.
collecting stage C results ( chunks in total)
stage C completed
2nd pass fps = 0
total time for encoding: 1177524182 seconds
overall fps = 0
now merging the chunks...
invalid suffix. using .mp4 instead.
"2>" ist syntaktisch an dieser Stelle nicht verarbeitbar.
chunks merged
Illegal division by zero at functions.pl line 937.

What am I missing?

Thanks a lot and best regards
Andrea

[708145]

Dear ELDER gurus,
sorry but I do have the same problem.

using 4 threads to encode 1 chunks of about 0 frames
warning: less chunks than available threads!
Illegal division by zero at functions.pl line 334.
...


Yes I noticed that there are a few cases where ELDER does not detect the number of frames correctly (and the detection returns 0) => It fails to do anything useful.
I did not do anything on ELDER the past half a year but I do have an unfinished version of a new detection algorithm. I'll see what I can do but the weather is just too good :D

bis besser,
T0B1A5

[708145]

I don't understand why it thinks my video has 0 frames.

I somehow missed that post. But the reply is the same as to the other poster.

[paranoid87]

hey i get the same problem as andrews!!..why??

[708145]

hey i get the same problem as andrews!!..why??

Still the same reason as most of you guys have: The video length is detected as 0 frames! To help you guys I will package my incomplete rewrite of the detection. ETA is about a week so hang on :)

bis besser,
T0B1A5

[708145]

Still the same reason as most of you guys have: The video length is detected as 0 frames! To help you guys I will package my incomplete rewrite of the detection. ETA is about a week so hang on :)


Well, it's been a long "week" but I finally run the last tests of beta6b and notice that there is a lot that broke :(
Here is a intermediate version where -f should be handled well but -s and -b are broken! It's not a trade-off, just a bug that happens when you look at code just once in several months for a couple of minutes :(
I'll announce the link in the first post of this thread when I put it online. :D
Have phun. :D

bis besser,
T0B1A5

[Jdot]

708145, your tool is great. ver.5 worked for me for xivd 2 pass and x264 crf pass (no gui selectable 2pass). I couldn't get 6 to work at all via gui or cmd line. So i hacked ver 6's additional passes into 5 . I've only tested xvid2pass, x264crf & x264 2pass they all work well for me. Are you still active on development? How many cores can elder support before gains become irrelevant.

I not a coder so I don't understand why you changed the frame detection portion of you code if it already works... :confused:

Thanks again for the great tool :cool:

J