x264 multi-core (4+) threading optimization

[morph166955]

EDIT: To make it easier for people to find the thread_pool04 patch is available at http://www.benswebs.com/public/x264/patches/x264_thread_pool.04c.r680.diff


I've made the disturbing realization that as of right now every method I have tested on sending video to x264 is faster on xp then it is in freebsd...which really should not be! I'm looking for some options (or what other people are doing) to try out on this system to get x264 to run at its peak speed wize. This machine is an octacore xeon which is pulling well over 100fps in xp when doing the megui benchmark using avisynth/mt.dll yet it barely hits 100 if at all in bsd...even if i pipe it raw y4m files! I really want to get this machine running as fast as humanly possible (which for this thing should be obsurdly fast) and I'd rather not run it in XP. x264 is not running anywhere near 100% (infact its closer to 35% on all of the cores) when in first pass and only maybe 75% in the second pass so I know it can definitely speed up. So please...any ideas on how to pipe the video into x264 to run this thing to its max would be greatly appreciated.

(note for mods: im not looking for "bests" here...im looking for options to test and opinions on what may or may not be the ideal settings on this setup)

[Dark Shikari]

Tried using more threads if its not using all the CPU power?

[AGDenton]

Raw Y4M == HDD bottleneck. It is imperative that you decode the input in another program, then pipe it to x264 (using | or a fifo). However, x264 tends to assume its standard input to be raw YUV, not Y4M, so you'll have to apply the patch that adds a --y4m-input option (or use a fifo with name **.y4m).

Have you tried ffmpeg -f yuv4mpegpipe | ...?

Anyhow, there's always the option of using Avisynth under wine with avs2yuv, which was done for that exact purpose. It will be slower than under Windows, but not by much.

AG

[morph166955]

forgot to mention...the source was loaded in a ramdisk. the ram is 4 gig worth of 240Pin DDR2 667 FB-DIMM's...so shall we say...no slow down on the source in terms of accessing it :D

[akupenguin]

No problems here. x264 reading from a rawyuv file uses all the cpu I can give it. (not a ramdisk, just relying on Linux's file-chache to keep it in ram)

[morph166955]

At this point I've found that (at least freebsd) has some issues with scheduling for more then 4 cores (it wont even consider letting x264, ffmpeg or mencoder have more then 9 threads which i assume is 1 for input and 8 for x264).

I'm (saddly) going to end up running xp as the primary os on this box for now and possibly run some form of linux in a vmware workstation to do testing even though this will slow down the operation of the vmware slightly (although with that many cores its probably negligable). I'm moving things around on the drives now so that I can change my partition types w/o having to export my files off of the system onto externals or w/e. I already have windows loaded on this box on another parition so I'm going to use acronis to redo the paritions and just make it one huge one on this disk. Once I get something working stably in vmware I'll probably migrate over to that OS but I at least want something to tinker with until then seeing as it may take a good while to do that.

I'll let ya all know how it goes in a few hours once i do some more tweaking.

[giandrea]

At this point I've found that (at least freebsd) has some issues with scheduling for more then 4 cores (it wont even consider letting x264, ffmpeg or mencoder have more then 9 threads which i assume is 1 for input and 8 for x264).

I'm (saddly) going to end up running xp as the primary os on this box for now and possibly run some form of linux in a vmware workstation to do testing even though this will slow down the operation of the vmware slightly (although with that many cores its probably negligable). I'm moving things around on the drives now so that I can change my partition types w/o having to export my files off of the system onto externals or w/e. I already have windows loaded on this box on another parition so I'm going to use acronis to redo the paritions and just make it one huge one on this disk. Once I get something working stably in vmware I'll probably migrate over to that OS but I at least want something to tinker with until then seeing as it may take a good while to do that.

I'll let ya all know how it goes in a few hours once i do some more tweaking.

That's what I tought. Try running Linux, it's better at multithreading. Try Ubuntu server and build x264, MPlayer, ffmpeg and what else with optimizations for your processor.

[morph166955]

I'm going to run vmware on the xp install to test different distro's for speed (assuming that vmware degrades each speedwize approximately equally) as well as stability and ease of use. I was goign to try debian, fedora and ubuntu first...then maybe a few of the other ones depending how those do. I think its going to be a lot of trial and error to get the most optimal setup for me.

[akupenguin]

test different distro's for speed
What kind of speed could you test?

Responsiveness of their default desktop environment? (choice of window manager doesn't have much to do with distro)
Speed at which they run x264? (should depend only on the kernel's smp scheduler and the amount of crap running in the background, not anything distro-specific)
Boot time? (the amount of time I spend waiting for a reboot over the lifespan of a computer is probably less than the time it would take to install one alternate distro)

[morph166955]

yea i was going to shoot for encoding speed basically...most of it was going to be ease of installing/compiling software and such (and just how i liked it overall). speedwize the only thing i can think of is if the os has any added software running that can take up cpu time and/or different libraries (such as the threads library) that may affect how it all runs. its mostly going to come down to how i like the system overall and im probably just going to end up playing with it over the course of a few weeks...ill just run everything in xp until then it should do for now.

[morph166955]

and xp bites the dust...it wont let me access more then 2 gig of my ram w/o going to xp64...fedora core 6 here i come!

[morph166955]

ok i loaded up fedora core6, compiled x264, and piped the megui test through mencoder using the x264 options in their job1-2.xml file for a first pass. i'm still only getting 35-40% cpu utilization on the first pass putting me at ~177fps (yea i know im complaining about 177fps on x264...sounds almost too good to be true right).

@akpenguin
what would be the most optimal way for me to create a file from the test mpeg into something that a non-y4m patched x264 can read the fastest? ffmpeg? mplayer/mencoder?

[akupenguin]

ffmpeg, mplayer, and mencoder will all produce identical yuv files.
shortest commandline is `ffmpeg -i test.mpg -f rawvideo -y test.yuv`

I might have reproduced your problem (though not as great a magnitude given only 2 cores). I have two core2 duo systems here, and one gets 105% smp efficiency (i.e. 100% cpu use and 5% less total cpu-time than 1 thread needed) while the other gets 90% efficiency (i.e. 85% cpu use and 5% less total cpu-time). This is true even with intra-only, which should have no synchronization overhead at all. So I don't know why it happens, but at least I can test alternatives.

[morph166955]

Heres some CPU Utilization numbers for you. Source is generated from using the megui test .avs file thru mencoder into a raw yuv file. x264 runs were done using the megui test options (see below). You must divide the cpu percentage by 8 i believe to get the average. Pass 1 is still only around 45% and pass 2 is around 80%. These final #'s were generated after the passes were all run (they were obviously run pass1 then pass2 not all the 1's then all the 2's). I did 10 runs with these settings.


PASS1:
encoded 1488 frames, 189.16 fps, 989.84 kb/s
27.75user 0.42system 0:07.87elapsed 357%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 182.12 fps, 989.84 kb/s
28.00user 0.41system 0:08.17elapsed 347%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 187.77 fps, 989.84 kb/s
27.84user 0.41system 0:07.93elapsed 356%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 188.89 fps, 989.84 kb/s
27.83user 0.46system 0:07.88elapsed 358%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 188.22 fps, 989.84 kb/s
27.87user 0.42system 0:07.91elapsed 357%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 187.57 fps, 989.84 kb/s
27.79user 0.40system 0:07.94elapsed 355%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 185.17 fps, 989.84 kb/s
27.85user 0.41system 0:08.04elapsed 351%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 187.65 fps, 989.84 kb/s
27.78user 0.46system 0:07.93elapsed 356%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 185.20 fps, 989.84 kb/s
27.72user 0.42system 0:08.04elapsed 350%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 188.29 fps, 989.84 kb/s
27.77user 0.44system 0:07.91elapsed 356%CPU (0avgtext+0avgdata 0maxresident)k

PASS2:
encoded 1488 frames, 116.38 fps, 1009.85 kb/s
84.39user 0.90system 0:12.87elapsed 662%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 116.82 fps, 1009.74 kb/s
84.26user 0.91system 0:12.82elapsed 664%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 116.19 fps, 1009.75 kb/s
84.25user 0.90system 0:12.88elapsed 660%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 116.74 fps, 1009.39 kb/s
84.41user 0.91system 0:12.83elapsed 665%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 116.34 fps, 1009.38 kb/s
84.42user 0.84system 0:12.87elapsed 662%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 116.08 fps, 1009.44 kb/s
84.27user 0.93system 0:12.90elapsed 660%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 116.15 fps, 1009.78 kb/s
84.28user 0.89system 0:12.89elapsed 660%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 115.93 fps, 1009.79 kb/s
84.22user 0.92system 0:12.91elapsed 659%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 115.99 fps, 1009.38 kb/s
84.24user 0.87system 0:12.91elapsed 659%CPU (0avgtext+0avgdata 0maxresident)k
encoded 1488 frames, 115.59 fps, 1009.48 kb/s
84.33user 0.87system 0:12.95elapsed 657%CPU (0avgtext+0avgdata 0maxresident)k


x264 options:

x264 --pass 1 --bitrate 1000 --stats test-NEW.stats --bframes 3 --b-pyramid --direct auto --subme 1 --analyse none --vbv-maxrate 25000 --me dia --merange 12 --threads auto --thread-input --progress --no-psnr --no-ssim -o /dev/null stream.yuv 640x480

x264 --pass 2 --bitrate 1000 --stats test-NEW.stats --ref 3 --bframes 3 --b-pyramid --weightb --direct auto --subme 6 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --merange 12 --threads auto --thread-input --progress --no-psnr --no-ssim -o /dev/null stream.yuv 640x480

[AGDenton]

What happens if you lower mvrange-thread manually and increase the number of threads?

[akupenguin]

The extreme case is --keyint 1. I-frames can all be encoded independently without any mvrange-thread.

[morph166955]

@akupenguin
how is mvrange-thread calculated in x264 when its set to auto (i assume thats the default?)

I'm a little confused about the whole mvrange-thread thing all together as far as what it exactly does and there isnt a whole lot of stuff on here minus the random post saying "yes use it its good". Whats a good way for me to calculate it to optimize speed w/o it being overly crazy where it will affect my bitrate?

[AGDenton]

The extreme case is --keyint 1. I-frames can all be encoded independently without any mvrange-thread.

That's the case here? I thought default keyint was 250 or something...

[akupenguin]

That's the case here? I thought default keyint was 250 or something...
Right. But if you're going to change options, keyint 1 is the obvious one to try.

how is mvrange-thread calculated in x264 when its set to auto
Divide the frame height by the number of threads, subtract the buffer needed for encoding and filtering. That's the amount of space available between threads.
Then take half, round up to a multiple of the macroblock height. That's the space allocated to mvrange-thread.
The other half is scheduling leeway, so that threads don't have to wait for eachother after every row.

The derived value of mvrange-thread is printed at the beginning if you run `x264 -v ...`

[morph166955]

ah...cool. thanks for the explanation! its comming up now default at 24. I'm moving some files around right now on those drives so once thats done (should be soon) I'll run a few passes with lower numbers for that (im going to try it first at threads=12 first for different values to see what that does, then ill play around with different #'s of threads)

[akupenguin]

I think thread synchronization benefits from an illustration.
http://akuvian.org/images/x264_threads_pic.png
Assume 3 threads and no B-frames, so the threads are encoding consecutive P-frames. Assume mvrange-thread = 40 (2.5 macroblocks). In the illustration I ignore the temporal offsets and pretend it's one frame, to better show the spatial relationship of the threads.
black = macroblock boundaries.
red = macroblock currently being encoded by the numbered thread.
blue = pixels that have been encoded, but have not been deblocked and hpel filtered yet.
dark green = pixels that have been fully encoded and filtered by the thread below, and are avilable for use (motion search) by the thread above. The height of this band is equal to mvrange-thread.
light green = pixels that have been fully encoded and filtered by the thread below, but the thread above hasn't synchronized yet so it doesn't know they're available.

At the time of this snapshot, threads 1 and 2 are happily encoding, but thread 3 is waiting on thread 2. 3 won't continue until it has another row's worth of pixels available, because if it continued now it would only have a 24 pixel range to search.

mvrange-thread shouldn't directly affect speed, rather it controls the number of threads that can run concurrently. (Which is why there is no single default value of mvrange-thread, and instead it's derived from the number of threads requested.) Each thread takes up some amount of vertical space (mvrange-thread + 1.5 macroblocks (the .5 comes from deblock and hpel, which each require some border)), and all the threads that are running in referenced frames at once must be able to stack vertically within the height of one frame. Non-referenced B-frames are exempt, since nothing is waiting for them.

[morph166955]

im still running tests but I had to post this up...even though keyint=1 is insane I ran it anyway...x264 set @ threads=12, mvrange-thread=24, keyint=1

encoded 1488 frames, 289.30 fps, 1005.11 kb/s
17.49user 0.27system 0:05.14elapsed 345%CPU (0avgtext+0avgdata 0maxresident)k

encoded 1488 frames, 219.21 fps, 1015.43 kb/s
44.20user 0.80system 0:06.86elapsed 656%CPU (0avgtext+0avgdata 0maxresident)k


yea...thats 289fps!!! and it still is only at 43% cpu!

Minus altering keyint, setting mvrange-thread is only affecting me by 3-5fps on each pass and that can be contributed to other processes using the drive for a ms here and there. 24 was the default mvrange-thread i got from -v so i included it here.

[morph166955]

@ak i just saw your post after i posted my last one...that was a very good explanation of it. THANK YOU!

[morph166955]

ok im getting a little stumped here. ive just tested all sort of values of mvrange_thread and mvrange from 2 up thru 48 (all different values since I thought I was going nuts and/or had something really backwards). the most that this does is a 3-5fps difference here or there (186-190 is basically the range, close to 188 most times). other then keyint=1, nothing has gotten this thing to speed up at all. even at keyint=1 the thing cpu ran at like 45%. I tried to access the source off of the 10K RPM to see if there were any speed differences...none.

[morph166955]

holy multiple posts batman...but i think i have a reasoning for the speed cap. Instead of just running on one small video clip like the megui test, i decided to test it on something longer so i picked the dvd of die another day. i imaged my dvd down to an iso so that there was no lag from that. i then used mkfifo to create a pipe. i had mencoder do the rawvideo output and i resized it using scale=854:480,crop=848:352:4:62,pullup. after letting it sit i noticed that when watching it in top (in thread view as opposed to an overall view) the cpus are all running somewhere between 20 and 60 percent however on each refresh theres only one active thread of x264 (occasionally theres a second one running at 2% on one core). shouldn't i see multiple threads running at the same time? is it possible that x264 is either not requesting enough data or that im unable to send the data to it fast enough in some form or something like that?

[akupenguin]

That's just a limitation of top. It only displays threads that live longer than top's refresh rate (default: 1 sec). But x264 spawns a new thread for each frame and kills it when the frame is done, which is much less than 1 sec at 289 fps, so you're only seeing the master thread. The total cpu usage (in the header lines) should still be correct. Another case this limitation shows up is if I'm compiling lots of small c files, each instance of gcc goes by quickly enough that I can be at 100% cpu usage without any individual processes showing as taking any cpu.

[morph166955]

ah...ok. well pass 1 just finished on it using the identical settings of the megui test (while probably not deseriable in all cases i wanted to keep as much as i could the same to avoid speculation on other issues). it finished at 112fps and through out the entire encode top read between 35 and 40% cpu. pass2 is running now and its between 65 and 85 percent hovering in the mid 70's and occasionally peaking to 89 but thats rare.

[morph166955]

aku let me ask you this then since im still kinda new with the whole x264 threading thing (this is really my first threaded box that actually had the power to do anything). if you were running x264 on my box with this source (die another day from dvd/iso @ 848x352) what would your command lines be for these two passes?

[morph166955]

bump...still havin the bottleneck...any other ideas guys?

[AGDenton]

You know, apart from the thing akupenguin's trying to identify with >100% SMP efficiency, the issue here really is that your job (a not-really intensive processig of a DVD source) is just not easily parallelizable. You can already see it here : the second pass, which is more intensive than the first, parallelizes more than the first (>650% CPU vs ~350% CPU). In other words, you're hitting the ceiling of the "sliceless threading" algorithm.

This ceiling will be higher (and you want it to be >800% for your Xeons) if you process higher-def content, or process it in a more intensive way. So try doing HQ-Insane on a 1080p source, and you'll probably find that all your cores are all at work :).

If you want to process this particular job faster, you'll have to ditch x264's threading and use x264farm instead. Try setting up a controller and an agent under your computer in XP...

AG

[morph166955]

I think that inevitably what im going to do is write a script/program similar to x264farm but for a local encode setup to remove any overhead from the network stuff and avoid the issue that x264farm requires a windows controller which I wont have in this setup.

[akupenguin]

But there's no good reason for sliceless threading to have a ceiling. If intra-only doesn't get perfect scaling on an arbitrarily high number of cpus, that's a bug. And it's probably the same bug that's limiting scaling in the real use cases too.

[morph166955]

Well you are the x264 master...and I am ready willing and able to run what ever kind of tests you want me to run on this to assist in diagnosing and repairing any bug or what ever you want to call this issue. I'm still going to attempt to work on some software similar to x264farm but for local use. If nothing else, it should let me pause encodes for large stuff and for now let me peg my cpu. What I figure may work "best" is running 4 x264 processes, each with threads set to 3. That "should" give me the 12 threads that I would have originally run, and alleviate the issue I'm seeing here. I'll let you know how it goes.

@akupenguin
In regards to my above software, I want to run a pass0 similar to x264farm (at least from what I have heard about it). What would be a good way to run a very fast pass over the source to find my scenecuts so that I can setup the pass1 and pass2 files to make the most efficient use of x264's encodes

[akupenguin]

This has probably been discussed in the x264farm threads.
Short answer: overlap the 1st pass chunks by about 1 gop length. Pick any I-frame in the overlap region, and make that the split point.

[morph166955]

ok so let me make sure im 100% right with this before i go coding something. lets for arguments sake say that there is an I frame every 250 frames (so 0, 250, 500, etc). I could do the encodes so that lets say the first chunk is 0-999, the second chunk so its 750-1999, the third chunk at 1750-2999, the fourth at 2750-3999, and so on and so forth (obviously those lengths are arbitrary and involves a whole lot of unnecessary encoding but lets just go with it for the theory/understanding of it and I'll tinker with good values for distance of the chunks).

[akupenguin]

yes.
You can reduce the amount of redundant work if you have some monitoring process to terminate each encoder when it finds a split point, but the simple way is as you said.

[morph166955]

awesome. what options would you recommend to do a super fast pass to generate an accurate stats file that I can parse to get those I files? (im still kinda getting used to using x264 straight as opposed to through mencoder so please bare with me until i get all the syntax down)

[akupenguin]

While I suppose I do have some favourite settings, I try to leave recommendations to 3rd parties :) See whatever megui uses for "fast 1st pass".

[morph166955]

fair enough...ill check it out. I'm also running some passes using mencoder to pipe a raw yuv stream and then doing a SAD comparison of the Y parts (thanks omion for the idea!). assuming I can tweak that to settings comparable to x264 that should allow me to compare the frames w/o ever encoding them. Once I have that data then deciding where I want to make my splits should be a matter of personal pref (ill prob put a cli in so i or anyone else using this can run it at what they want)

[akupenguin]

I added some timing constructs to find out where the idleness was happening. afaict most of it is due to a delay between when one frame finishes and when the master thread notices it. If the frame is already done when the master calls pthread_join then it returns immediately, but if it has to wait then there's an additional delay (2.5ms) beyond the time needed to finish the frame.
This might be fixed by switching to a thread-pool. It's a shame, I liked the "call a subroutine in a new thread" model better, but I guess there's a reason most threaded apps use thread-pool.

[morph166955]

ah...makes sense. I'm going to be running some tests using my memory pipe software and x264 later on to see where it maxes out. as of last night, it looked like it was between 3-6 threads. more info to come later!

oh and just to throw something else in the pot...when doing a straight SAD comparison of the frames when the source is in the ram...it was working at close to 950fps (that was using 8 sad comparison pthreads). while i realize that an sad comparison is no where near the amount of work encoding takes, it definitely shows that the system is capable of doing it.

[Mutant_Fruit]

If the frame is already done when the master calls pthread_join then it returns immediately, but if it has to wait then there's an additional delay (2.5ms) beyond the time needed to finish the frame.
2.5ms is a fairly steep cost if it's a fixed delay. The faster your machine is, the worse that delay is as compared to the time needed to encode a frame. At 100fps, that'd represent a 25% increase in time needed to encode a frame (if the thread was forced to wait).

[akupenguin]

I'm sure it's not a fixed cost, and it's not even constant (2.5 is just the average), but the settings I measured that on were running at 140fps.

[morph166955]

@aku...do you want me to run that code on this system to see what I average time wize?

[akupenguin]

apply patch (http://akuvian.org/src/x264/x264_thread_delay.diff).
run: x264 $options --quiet &>log
run: x264_thread_delay.pl (http://akuvian.org/src/x264/x264_thread_delay.pl) log
and/or send me log.

x264_thread_delay.pl should print something like
avg delay (join first): 2459 us (1535 samples)
avg delay (stop first): 11 us (693 samples)
avg idle before join: 591 us (693 samples)

"avg delay (join first)" is what I referred to above.
"avg idle before join" is only valid if # threads <= # cpus. Otherwise, it's just the time that a given thread spends idle, which doesn't necessarily mean the cpu was idle.

[morph166955]

OK here are your results. CPU was running ~35-40%, pass averaged 137fps. run using:
mencoder dvd://1 -dvd-device DIEANOTHERDAY_DISC1.ISO -ovc raw -of rawvideo -nosound -o stream.yuv -vf harddup,scale=854:480,crop=848:352:4:62,format=i420 -fps 24000/1001 -mc 0 -noskip -frames 5000

x264 --threads 8 --progress --bitrate 1025 --stats DIEANOTHERDAY_DISC1.stats --bframes 3 --b-pyramid --subme 1 --analyse none --me dia --pass 1 --fps 23.976 --quiet -o /dev/null stream.yuv 848x352 > joinlog.txt 2>&1

Your pearl script gave me:

avg wallclock per frame: 7271 us (5000 samples)
avg delay (join first): 297 us (4 samples)
avg delay (stop first): 23 us (4996 samples)
avg idle before join: 34955 us (4996 samples)


If you want the log itself let me know and ill send it to you. That average idle before join seems abnormally high...

That was the first 5000 frames, I'm going to run it full through now to see what that does.

[akupenguin]

updated patch (http://akuvian.org/src/x264/x264_thread_delay.01.diff).
This time I want the following option sets, with the log itself:
--subme 1 --analyse none --qp 24 --bframes 3 --b-pyramid
--subme 1 --analyse none --qp 24 --scenecut -1 --bframes 3 --no-b-adapt
--subme 1 --analyse none --qp 24 --scenecut -1 --keyint 1

5000 frames each is fine, unless your longer run gives wildly different numbers. (And of course still make sure the input is on a ramdisk or equivalent)

[morph166955]

The full length run just ran and it spit this out:

avg wallclock per frame: 7428 us (190375 samples)
avg delay (join first): 182 us (7 samples)
avg delay (stop first): 25 us (190368 samples)
avg idle before join: 35815 us (190368 samples)

I'm applying your patch and running it now. I'll let you know shortly.

[morph166955]

aku am i applying this new patch in addition to the other or should i put it on a clean x264?

EDIT: nevermind, i see that the new on includes the old one...

[morph166955]

ok heres the output & cli's from the runs. I ran this from my ram buffer that i wrote. 5000 frames each. the last run managed to get the cpu up to around 60%.


./x264_thread_delay.pl job1.txt
avg wallclock per frame: 5705 us (5000 samples)
avg delay (join first): 995 us (4 samples)
avg delay (stop first): 20 us (4996 samples)
avg idle before join: 22907 us (4996 samples)
FPS: 175.67

./x264_thread_delay.pl job2.txt
avg wallclock per frame: 3615 us (5000 samples)
avg delay (join first): 879 us (127 samples)
avg delay (stop first): 13 us (4873 samples)
avg idle before join: 10603 us (4873 samples)
FPS: 273.70

./x264_thread_delay.pl job3.txt
avg wallclock per frame: 3808 us (5000 samples)
avg delay (join first): 369 us (57 samples)
avg delay (stop first): 10 us (4943 samples)
avg idle before join: 9930 us (4943 samples)
FPS:259.60


run with:

x264 --subme 1 --analyse none --qp 24 --bframes 3 --b-pyramid --threads 8 -o /dev/null stream2.yuv 848x352 > job1.txt 2>&1

x264 --subme 1 --analyse none --qp 24 --scenecut -1 --bframes 3 --no-b-adapt --threads 8 -o /dev/null stream2.yuv 848x352 > job2.txt 2>&1

x264 --subme 1 --analyse none --qp 24 --scenecut -1 --keyint 1 --threads 8 -o /dev/null stream2.yuv 848x352 > job3.txt 2>&1


as far as the ram cache thing goes...the way that I've been running it is:

start mencoder & set to write to stream1.yuv
start x264 & set to read from stream2.yuv
start ram cache...it reads all 5000 frames into memory from mencoder & then writes them out to stream2 as fast as it can. I think the only way that I can make it faster is to not freeup the ram until its written all the frames out but I dont really think thats causing any sort of a bottle neck since I can dump it to /dev/null

I assume thats the fastest way to run it.

EDIT: oops forgot to tack on your logs...here you go... had to 7z them so they would be small enough