x264 multi-core (4+) threading optimization

[morph166955]

so i was running a few more benchmarks. if i can figure out some way to run mencoder's x264 in two threads that would optimize how fast I can pull data off the drive and decode it. while i realize this isnt a foolproof method for benchmarking, its the best i could do for now. i forked 2 identical mencoder threads and wrote a small program that would read frames from each. when i ran it on more then 2 threads, the mencoder threads seemed choked (couldnt get over 65-75% cpu) and ran slower. at 2 threads, the two mencoders ran near 100% and it gave me a hair under 400fps (~396fps).

I'm trying to figure out if I can run framestep=2 or something like that and get even frames out of one and odd from the other or something to that effect. assuming i can then that should theoretically maintain a stable enough memory cache for the speeds im looking for since i doubt ill surpass that (im averaging like 200fps when im trying to simultaniously fill the back of the cache and work on the frames in the front of it.) So if that works...then that hopefully removes the last bottleneck that i'm should find when writing my x264 parallel encoder.

p.s. ill just note now before anyone says "well theres your problem" that the 5000 frame tests above were not done with that program, it was done with one that does all the reading into memory and then dumps that out.

EDIT:
I love how i come up with an idea, then the moment after i type it all out and post it i have a brain storm on how to do it. I'm officially decoding H264 in linux multithreaded. it was rather simple and all thanks to the tinterlace filter in mencoder. i have one set to tinterlace=1 and the other to 2. i wrote a small program to read both and then spit both back in order and if you weren told that It was running that way you wouldnt be able to see a difference. Its late and i have a test tomorrow morning then class till late tomorrow night so I doubt that I'll be able to implement this how I want to tomorrow but hopefully i'll be able to get this working on friday and if that happens then I shouldnt have any issues writing some parallelizing software over the weekend/next week/and on and on and on since thats how software is.

[akupenguin]

updated patch (http://akuvian.org/src/x264/x264_thread_delay.04.diff) and updated parser (http://akuvian.org/src/x264/x264_thread_delay.04.pl). same option sets.

[morph166955]

All three had the following first 3 lines from the parser:


Use of uninitialized value in subtraction (-) at ./x264_thread_delay.pl line 22, <> line 8.
Use of uninitialized value in subtraction (-) at ./x264_thread_delay.pl line 22, <> line 13.
Use of uninitialized value in subtraction (-) at ./x264_thread_delay.pl line 22, <> line 29.



Job1:

avg wallclock per frame: 5670 us (5000 samples)
avg cpu per slice: 15130 us (5000 samples)
avg prolog_cp: 1531 us (5000 samples)
avg prolog_rc: 2645 us (5000 samples)
avg prolog_etc: 706225484933 us (5000 samples)
avg prolog: 21188 us (5000 samples)
avg thread create: 1815 us (5000 samples)
avg delay (join first): 58 us (4 samples)
avg delay (stop first): 9 us (4996 samples)
avg idle before join: 22775 us (4996 samples)
avg epilog: 914 us (5000 samples)
avg between frames: 497 us (4989 samples)



Job2:

avg wallclock per frame: 3651 us (5000 samples)
avg cpu per slice: 14124 us (5000 samples)
avg prolog_cp: 766 us (5000 samples)
avg prolog_rc: 6 us (5000 samples)
avg prolog_etc: 706225510474 us (5000 samples)
avg prolog: 11716 us (5000 samples)
avg thread create: 948 us (5000 samples)
avg delay (join first): 710 us (118 samples)
avg delay (stop first): 8 us (4882 samples)
avg idle before join: 10912 us (4882 samples)
avg epilog: 948 us (5000 samples)
avg between frames: 1690 us (4989 samples)



Job3:

avg wallclock per frame: 5894 us (5000 samples)
avg cpu per slice: 36904 us (5000 samples)
avg prolog_cp: 1524 us (5000 samples)
avg prolog_rc: 1605 us (5000 samples)
avg prolog_etc: 12 us (5000 samples)
avg prolog: 3142 us (5000 samples)
avg thread create: 1181 us (5000 samples)
avg delay (join first): 607 us (2132 samples)
avg delay (stop first): 10 us (2868 samples)
avg idle before join: 7445 us (2868 samples)
avg epilog: 818 us (5000 samples)
avg between frames: 511 us (4992 samples)


you can pull the logs from http://www.benswebs.com/x264/ they were too big to upload even when zipped so i just put them on my webspace.

i should be around all day tomorrow and the whole weekend to run as much code as you want so just lemme know.

[akupenguin]

thread pool patch (http://akuvian.org/src/x264/x264_thread_pool.00.diff). benchmark patch (http://akuvian.org/src/x264/x264_thread_pool.00.bench.diff) to be applied on top of thread pool. parser (http://akuvian.org/src/x264/x264_thread_delay.pl). I'm interested in both the logs (as before) and in the performance of thread pool compared to svn (without the benchmark part of the patch, since that might affect performance).

thread pool should eliminate "avg thread create", and might or might not affect "avg delay (join first)". I also know how to eliminate epilog, though I didn't do it in this patch.

weirdness: a quick test (on a core2 duo) says that this patch is 3% faster than svn when using 2 threads, but 3% slower than svn when using 3 threads.

[morph166955]

im having a heat issue right now with the box...prob wont be run until tomorrow morning. i just got some zalman cpu grease but im sure you can imagine how fun its going to be to remove 2 xeon heatsyncs, clean them and the cpu's, reapply, and reassemble. ill let ya know.

[morph166955]

aku...im getting ready to run your test. do you want me to apply that patch to r650 or r651?

[akupenguin]

Shouldn't matter, as long as you use the same base version for all tests. So 651, just in case I later add something that does interact with 651.

[morph166955]

ok ran it against 650 just now (didnt see your post till it was done). I'll run it against 651 shortly.

run 1 got the cpu between 39 and 48 percent...occasionally hitting 50-52 but that was rare. it averaged 196fps

run 2 got the cpu in the low 60's. it averaged 348fps (niiiiiiice :D)

run 3 got the cpu between 65 and 75 percent. it averaged 174 fps.

you can get your log files from http://www.benswebs.com/x264/jobs-r650-threadpool.00-bench.00.tar.gz

Here is the output from your benchmark program:


avg wallclock per frame: 5085 us (5000 samples)
avg cpu per slice: 14274 us (5000 samples)
avg prolog_cp: 1410 us (5000 samples)
avg prolog_rc: 2561 us (5000 samples)
avg prolog_etc: 706305303995 us (5000 samples)
avg prolog: 19226 us (5000 samples)
avg thread create: 150 us (5000 samples)
avg delay (join first): 30 us (4 samples)
avg delay (stop first): 4 us (4996 samples)
avg idle before join: 21174 us (4996 samples)
avg epilog: 839 us (5000 samples)
avg between frames: 220 us (4989 samples)

avg wallclock per frame: 2875 us (5000 samples)
avg cpu per slice: 14160 us (5000 samples)
avg prolog_cp: 773 us (5000 samples)
avg prolog_rc: 8 us (5000 samples)
avg prolog_etc: 706305327110 us (5000 samples)
avg prolog: 9400 us (5000 samples)
avg thread create: 444 us (5000 samples)
avg delay (join first): 391 us (587 samples)
avg delay (stop first): 9 us (4413 samples)
avg idle before join: 6888 us (4413 samples)
avg epilog: 866 us (5000 samples)
avg between frames: 580 us (4989 samples)

avg wallclock per frame: 5733 us (5000 samples)
avg cpu per slice: 36137 us (5000 samples)
avg prolog_cp: 1432 us (5000 samples)
avg prolog_rc: 1638 us (5000 samples)
avg prolog_etc: 11 us (5000 samples)
avg prolog: 3083 us (5000 samples)
avg thread create: 519 us (5000 samples)
avg delay (join first): 202 us (2131 samples)
avg delay (stop first): 7 us (2869 samples)
avg idle before join: 8261 us (2869 samples)
avg epilog: 871 us (5000 samples)
avg between frames: 360 us (4992 samples)

[morph166955]

oh crap...i need to appologize for something. the script that I had running was fubaring the third job (there was a typo in one of the variables so in reality it was being called w/o any options). this only effects the last like 2-3 tests since i wrote it recently. I'm rerunning the r651 tests now with the updated script ill post up the results shortly. SORRY!

[morph166955]

this is r651 patched (and now a proper job 3). I'm going to run a test on the performance of the raw svn and then the svn + thread_pool w/o benchmark now.

run 1 is averaging between 40 and 50 percent cpu @ 199fps
run 2 is averaging between 55 and 65 percent cpu @ 360fps
run 3 is averaging between 55 and 65 percent cpu @ 327fps

Heres the output of your perl script:

Starting Run @ Sat Apr 21 13:53:29 EDT 2007

================================================================================

JOB 1: --subme 1 --analyse none --qp 24 --bframes 3 --b-pyramid X264BIN:x264-r651-patched
avg wallclock per frame: 5033 us (5000 samples)
avg cpu per slice: 13556 us (5000 samples)
avg prolog_cp: 1407 us (5000 samples)
avg prolog_rc: 2526 us (5000 samples)
avg prolog_etc: 706306835056 us (5000 samples)
avg prolog: 19033 us (5000 samples)
avg thread create: 108 us (5000 samples)
avg delay (join first): 47 us (3 samples)
avg delay (stop first): 5 us (4997 samples)
avg idle before join: 21553 us (4997 samples)
avg epilog: 847 us (5000 samples)
avg between frames: 193 us (4989 samples)
COMPLETE: DID 5000 FRAMES in 25.042000 seconds @ 199.664564 FPS

================================================================================

JOB 2: --subme 1 --analyse none --qp 24 --scenecut -1 --bframes 3 --no-b-adapt X264BIN:x264-r651-patched
avg wallclock per frame: 2793 us (5000 samples)
avg cpu per slice: 13309 us (5000 samples)
avg prolog_cp: 764 us (5000 samples)
avg prolog_rc: 7 us (5000 samples)
avg prolog_etc: 706306857886 us (5000 samples)
avg prolog: 9151 us (5000 samples)
avg thread create: 484 us (5000 samples)
avg delay (join first): 278 us (525 samples)
avg delay (stop first): 8 us (4475 samples)
avg idle before join: 6996 us (4475 samples)
avg epilog: 858 us (5000 samples)
avg between frames: 568 us (4989 samples)
COMPLETE: DID 5000 FRAMES in 13.855000 seconds @ 360.880549 FPS

================================================================================

JOB 3: --subme 1 --analyse none --qp 24 --scenecut -1 --keyint 1 X264BIN:x264-r651-patched
avg wallclock per frame: 3074 us (5000 samples)
avg cpu per slice: 13261 us (5000 samples)
avg prolog_cp: 703 us (5000 samples)
avg prolog_rc: 5 us (5000 samples)
avg prolog_etc: 11 us (5000 samples)
avg prolog: 721 us (5000 samples)
avg thread create: 233 us (5000 samples)
avg delay (join first): 562 us (152 samples)
avg delay (stop first): 9 us (4848 samples)
avg idle before join: 9222 us (4848 samples)
avg epilog: 757 us (5000 samples)
avg between frames: 615 us (4992 samples)
COMPLETE: DID 5000 FRAMES in 15.264000 seconds @ 327.568134 FPS

================================================================================

End Of Run. Sat Apr 21 13:55:36 EDT 2007


logs @ http://www.benswebs.com/x264/jobs-r651-threadpool.00-bench.00.tar.gz

[morph166955]

this is a raw r651 run:

run1 was between 35 and 40 percent cpu @ 178fps
run2 was between 45 and 55 percent cpu @ 282fps
run3 was between 45 and 50 percent cpu spiking to 56 a few times @ 274fps

this is a run with r651 patched ONLY with the thread_pool patch and not the benchmark patch:

run1 was between 40 and 45 percent cpu @ 201fps
run2 was between 60 and 70 percent cpu @ 368fps (now thats a jump!)
run3 was between 55 and 65 percent cpu @ 331fps

to me thats no appreciable difference in framerate with the benchmark vs non-benchmark runs...its like 5 maybe 10 fps difference and that could be caused by almost anything. it is however a very noticable difference with the thread_pool patch on runs 2 and 3 and a decently noticed one on run1.

[akupenguin]

Nice. Now can you try varying the number of threads (maybe between 8 and 16)? since both should benefit from threads>cpus, but there's no guarantee that the optimal numbers are the same.
Ideally the result of that would be a table of fps for each of (3 option sets) x (2 versions) x (9 thread numbers).

Radon sidenote: instead of looking at `top` and saying "35 to 40 percent", you can run `time x264 ...` and get an exact number. Less work too. Though an exact number for cpu usage doesn't really matter, since it doesn't disinguish between real work and synchronization overhead, so it can't compare two versions, it can only serve as a rough estimate of how much potential optimization is left.

[morph166955]

those were all run at threads=8 (as well as everything else i've been running since you said running at threads=8 earlier was good to get proper numbers. ill run 12 and 16 now...give me a few.

[morph166955]

just when doing some initial runs (im getting time to work properly with the forking and everything its being not nice) im noticing that one of the cores is only what i would call a "dead" core. its user cpu usage is litterally 0.0 for most of the time (it does hop around occasionally to another core but thats maybe once in the entire run). thats on threads 12 btw...

[morph166955]

ok...a table you want a table yee shall have... :D

just to make sure I'm right with this before i have it do a bagilion runs, confirm for me that these are in fact the options you want:


JOBS:
#1: --subme 1 --analyse none --qp 24 --bframes 3 --b-pyramid
#2: --subme 1 --analyse none --qp 24 --scenecut -1 --bframes 3 --no-b-adapt
#3: --subme 1 --analyse none --qp 24 --scenecut -1 --keyint 1



Versions:
#1: raw/vanilla r651
#2: r651 w/ thread_pool_patch.00.diff & thread_pool_patch.00.bench.diff


and threads set at 8,9,10,11,12,13,14,15,16

assuming thats right...ill write a script to run exactly that and have it generate a table (probably a csv file since thats easy) of the encoding time, fps and cpu. shouldnt take too long for the whole thing to run, writing that script is probably going to be the longest part of the whole thing lol.

[morph166955]

r651 + threads_pool + benchmark @ 12 threads:


Starting Run @ Sat Apr 21 15:22:41 EDT 2007

================================================================================

JOB 1: --subme 1 --analyse none --qp 24 --bframes 3 --b-pyramid X264BIN:x264-r651-patched-bench
avg wallclock per frame: 5082 us (5000 samples)
avg cpu per slice: 13578 us (5000 samples)
avg prolog_cp: 1407 us (5000 samples)
avg prolog_rc: 2542 us (5000 samples)
avg prolog_etc: 706310046150 us (5000 samples)
avg prolog: 19197 us (5000 samples)
avg thread create: 105 us (5000 samples)
avg delay (join first): 33 us (3 samples)
avg delay (stop first): 5 us (4997 samples)
avg idle before join: 42185 us (4997 samples)
avg epilog: 848 us (5000 samples)
avg between frames: 210 us (4985 samples)

264BUFFER: COMPLETE: DID 5000 FRAMES in 25.283000 seconds @ 197.761342 FPS
x264: encoded 5000 frames, 196.75 fps, 887.69 kb/s
86.65user 1.12system 0:25.42elapsed 345%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+20687minor)pagefaults 0swaps

================================================================================

JOB 2: --subme 1 --analyse none --qp 24 --scenecut -1 --bframes 3 --no-b-adapt X264BIN:x264-r651-patched-bench
avg wallclock per frame: 2669 us (5000 samples)
avg cpu per slice: 16373 us (5000 samples)
avg prolog_cp: 768 us (5000 samples)
avg prolog_rc: 9 us (5000 samples)
avg prolog_etc: 706310068257 us (5000 samples)
avg prolog: 8784 us (5000 samples)
avg thread create: 518 us (5000 samples)
avg delay (join first): 687 us (234 samples)
avg delay (stop first): 7 us (4766 samples)
avg idle before join: 13456 us (4766 samples)
avg epilog: 889 us (5000 samples)
avg between frames: 534 us (4985 samples)

264BUFFER: COMPLETE: DID 5000 FRAMES in 13.230000 seconds @ 377.928949 FPS
x264: encoded 5000 frames, 374.57 fps, 853.20 kb/s
67.63user 1.55system 0:13.35elapsed 518%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+15838minor)pagefaults 0swaps

================================================================================

JOB 3: --subme 1 --analyse none --qp 24 --scenecut -1 --keyint 1 X264BIN:x264-r651-patched-bench
avg wallclock per frame: 2678 us (5000 samples)
avg cpu per slice: 13459 us (5000 samples)
avg prolog_cp: 691 us (5000 samples)
avg prolog_rc: 6 us (5000 samples)
avg prolog_etc: 10 us (5000 samples)
avg prolog: 708 us (5000 samples)
avg thread create: 777 us (5000 samples)
avg delay (join first): 1264 us (38 samples)
avg delay (stop first): 10 us (4962 samples)
avg idle before join: 15898 us (4962 samples)
avg epilog: 777 us (5000 samples)
avg between frames: 573 us (4988 samples)

264BUFFER: COMPLETE: DID 5000 FRAMES in 13.282000 seconds @ 376.449330 FPS
x264: encoded 5000 frames, 373.29 fps, 3585.97 kb/s
72.64user 1.13system 0:13.39elapsed 550%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+6206minor)pagefaults 0swaps

================================================================================

End Of Run. Sat Apr 21 15:24:43 EDT 2007

[morph166955]

I'm about half way through the tests you wanted run and im finding something interesting. its peaking around threads=10 and then as im increasing over that its falling off. im also getting very weird and non linear results on some of these (increasing threads on one doesnt necessarily mean it will do better then a previous run however it also doesnt mean it will do worse...its kinda a crap shoot). any ideas why?

[morph166955]

you wanted a table...so i give you a table! :D

http://www.benswebs.com/x264/x264%20Runs.jpg

it should be noted...the two red columns are merely the numerical difference of the two data columns.

other then the first test threads set at 9 or 10 seems to make the most gain over the vanilla image.

hope this helps! sorry for the delay i'm still having thermal issues on this setup and the rising temperature outside surely isnt helping at all. im working on getting my landlord to install my a/c asap so i can get this place regulated a little better to prevent me from hitting thermal cut.

also note im changing the topic for this thread since its definitely taken a turn towards another focus.

EDIT: I've uploaded the benchmark outputs for all threads 1-16 to http://www.benswebs.com/x264/ in case you want those

EDIT2: There was something funky with threads=9 especially in job 1 so I reran it and it gave much more linear results to the rest of the test so I have updated the table to reflect that. I also added all of the results for threads 1-16. ENJOY!

[Adub]

It looks to be that your best performance comes from 9 to 11 threads, which is very interesting. I am always interested in maximizing performance and these tests are broadening my knowledge, while also forcing me to reconcile the fact that the best number varies with each application.

[akupenguin]

x264_thread_pool.01.diff (http://akuvian.org/src/x264/x264_thread_pool.01.diff)
Now it no longer explicitly assigns one frame to one thread, but keeps a queue of frames and lets each thread grab the next frame whenever it's done with the previous. Queue size is --thread-queue and defaults to 2*threads.
No benchmark patch this time, as most of the numbers wouldn't mean the same thing in this algorithm.

As before, I'm interested in the table of fps vs threads.

[morph166955]

sounds good...im setting up the job to run now. i'm only going to run it by itself since we already have the numbers for the vanilla 651 build. gimme a few hours ill have some shiny results for you.

do you want me to modify --thread-queue or just leave it to default?

[morph166955]

I'm getting a hell of a lot of errors from x264:


x264 [warning]: specified frame type (5) is not compatible with keyframe interval
x264 [warning]: specified frame type (5) is not compatible with keyframe interval
x264 [warning]: specified frame type (3) is not compatible with keyframe interval
x264 [warning]: specified frame type (3) is not compatible with keyframe interval


also...its not stoping when its running out of frames to process. the buffer shows that its dumped all 5000 frames (its final output before exiting is "COMPLETE: DID 5000 FRAMES in 25.137000 seconds @ 198.909973 FPS") but x264 is frozen at 4996 frames (run with --progress -v). I've uploaded the log to http://www.benswebs.com/x264/x264-threadpool.01-threads8.err.txt so you can see whats going on.

[akupenguin]

x264_thread_pool.02.diff (http://akuvian.org/src/x264/x264_thread_pool.02.diff)

x264 [warning]: specified frame type (5) is not compatible with keyframe intervalThat was a simple typo. (didn't notice because my benchmark clip didn't have any scenes longer than keyint)

And the hang was a race condition whereby I locked the wrong mutex so a variable that changed wasn't protected by anything.

Random note: --thread-input (which is implied by --threads) still uses the create/join method. But that shouldn't matter for rawyuv input.

[morph166955]

I havent gotten completely through yet but there seems to be a very steady drop off of fps & cpu usage above threads=7. However, it should be noted that on job 3 @ threads=7 i hit 741% cpu (800 being the max) and a record of 498.72 fps.

i'm still having my thermal problems so it seems that these tests wont be finished until tonight when I can get this place a little cooler.

[morph166955]

aku im sorry to do this but its unavoidable. i cant run this system in here until the a/c gets installed. the machine is hitting thermal cut before i can even get through 1 full set of jobs for 1 thread setting. i talked to my landlord and there supposed to get the a/c in either tomorrow or tuesdsay at the latest so once that goes in were good to crank this thing 24/7. i just dont want to risk burning this thing out now after all that I have spent to get it going in the first place. I'll let ya know once I run the tests which I hope is tomorrow but no promises. Thanks again for all your help on working this out and don't think for a second that this speed bump is going to stop me once I'm past it.

[Hellworm]

akupenguin: You propably know that, but the patch also slightly increases performance (by 2-3%) on not so extreme situations (dual core): cpu usage increased to around 190% (before:185%) and fps to 37 from 36. Tested with mencoder, x264 revision 654, 3 threads, 5 bframes. With 5 threads it reaches to 195%/38fps.

[Mutant_Fruit]

While i *really* doubt this will have much performance impact, i as wondering if popping more than one chunk from the queue would make an appreciable difference in performance. I suspect the cost of acquiring/releasing the mutex as compared to processing a chunk from the processing queue is relatively small, so this would be a bit pointless, but i don't know the exact cost of synchronisation.

It'd only really make a difference if lock contention was high and there was a large delay in getting the lock as compared to the time needed to process the chunk.

EDIT: The reason i mentioned this is because morph166955 mentioned that there appears to be a steady dropoff in FPS for larger numbers of threads. If increased contention is really the cause of the performance dropoff, it might make the above change worthwhile.

EDIT 2: A good way to check this would be to put some logging code that measures the time between trying to get the lock and actually getting it and running that with a large number of threads (4-5) on a single core. If someone could supply me with a win32 build that does that, i'll run the benchmarks.

[morph166955]

ok the guy is supposedly coming tomorrow with the a/c so I should be able to generate all sorta fun results for you by then. just fyi im not going to be around thursday & friday i'm flying out of town for a job interview I should be back early saturday I can run more tests once i'm home. im kinda crossing my fingers right now that its the temperature of this room and not the computer itself causing the heat problems. i should know more in a few days.

[burfadel]

What kind of room temperature are you talking about? If its above 30C (86F) then lowering it to 22C (72F -standard room temperature) should definately at least help. If the temperature is around 30C and its overheating then better case circulation should help... If you're talking 35C (95F) or above then I say your problem is the room temp... at idle on a non-overclocked system you can run them in 45C heat, but they're really not designed for that! Aren't computers designed to be operated at standard international room temperature (22C)?...

[akupenguin]

i as wondering if popping more than one chunk from the queue would make an appreciable difference in performance.
Not possible. A thread can't take 2 consecutive frames from the queue, because the threading method relies on encoding N consecutive frames at once. If any are skipped, then the following frames can't start either. A thread can't take 2 non-consecutive frames, because it would have to know in advance exactly how long it will take to encode the first frame. If frames don't finish in the planned order, then again some threads are waiting for others.

The reason i mentioned this is because morph166955 mentioned that there appears to be a steady dropoff in FPS for larger numbers of threads. If increased contention is really the cause of the performance dropoff, it might make the above change worthwhile.
It could be lock contention, or it could be scheduling contention (extra context switches).

[morph166955]

the room is around 80F give or take but its a small room and their is barely any circulation so its stagnant. when the machine is not doing anything it runs around 45C...it hovers around 55-60C normally when running however if there really going for it one hits 62-63C and its game over. i'll have a definite answer tomorrow when the a/c goes in. i may yank out the 7 in 1 card reader in front so that theres more inlet space up there (not too much up front but there are ALOT of slits along the side to let heat in and out. I'll figure it out I have a few good ideas its just a matter of eliminating the room as a cause.

[morph166955]

It could be lock contention, or it could be scheduling contention (extra context switches).

have you seen the article (i think i saw it on slashdot yesterday) about the scheduling updates that are due to go into the linux kernel soon? Think that will help with any of that?

[akupenguin]

I read that article, but I can't tell if it affects x264. Our ideal situation is one thread per cpu, no context switches except to grab a new frame. If we need more threads than cpus (to fill in gaps from synchronization within a frame, since the gaps between frames are handled by the queue), then it's scheduling between identical tasks. In the case of identical tasks, the only possible scheduler parameter is the tick size, which is already configurable.

edit: ok, I benchmarked the scheduler patch. it significantly improved (+5%) the speed of x264.thread_pool.0 with >= 3 threads (on 2 cores). It had no significant effect on <= 2 threads nor on x264.thread_pool.2 nor x264.svn.

[morph166955]

yay for it being cool again in here! im running the thread_pool.02 passes now, I'll post an updated table soon. as it stands now though from the runs that are done, threads >= cores slows down as threads increases. threads = 7 seems to be the fastest actually on all 3 runs (wonder why not threads = 8)?

EDIT: I stand corrected, it seems that the plateau is at threads = 5 not threads = 7. I'm putting the rest into the table now but thats what it seems like is going on. I only ran up to threads=12 because of the noticed drop off as i got up that high.

EDIT2: Updated table to show thread_pool.02. Somethings going funky with my webspace at this moment and its timing out for some reason so if its not fixed shortly ill figure something else out. long story short, threads between 5 & 7 depending on the job is the fastest on this patch.

EDIT3: My webspace is running ungodly slow right now for reasons unknown to me, http://www.benswebs.com/x264/x264%20Runs.jpg in case the image fails to load on my other post theres a link directly to it. you may have to hit refresh a few times for it to go but it will go eventually it just did for me. sorry for the annoyance I dont know why its doing it at all.

[burfadel]

EDIT3: My webspace is running ungodly slow right now for reasons unknown to me, http://www.benswebs.com/x264/x264%20Runs.jpg in case the image fails to load on my other post theres a link directly to it. you may have to hit refresh a few times for it to go but it will go eventually it just did for me. sorry for the annoyance I dont know why its doing it at all.

Seems to be fine now... but of course now that I've said that!

[morph166955]

yea the host was probably running something that was bogging the box...possibly a backup or something im not sure its hosted elsewhere.

[legoman666]

I would like to do some testing with the updated x264 build with the thread pool, but have no idea how to apply the patch to the source and compile it for win32. Because as it is, x264 can only keep my 2.6ghz AMD X2 @ ~75-90% utilization with 3 threads and I really would like to get the full potential out of this thing :D

[morph166955]

just cause i'm a nice guy...

http://www.benswebs.com/x264/x264-r654-thread_pool.02-win32.exe

and hers a non patched version of r654 to use as a vanialla build to compare too:

http://www.benswebs.com/x264/x264-r654-vanillasvn-win32.exe

compiled with mingw32. its r654 w/ thread_pool.02 patch applied (benchmark patch NOT applied). compile done using my own build of gcc 4.3.0, no mp4 support (i dont have gpac loaded and most people use raw264 or mkv output anyway), and I only used make (not fprofiled, ill make some source up eventually to do a fprofiled build but I dont think it will do very much here). also note this does NOT have the aq patch loaded into it. If you really really really want one with aq loaded i'll consider making a build with it however I think for the purposes of this test putting anything additional onto the vanilla svn is a bad idea since it may skew the results and give inaccurate data.

EDIT: just remember, while the threadpool patch will probably show you something on a dual core, its much more apparent when you have 4 (or in my case 8) cores. also run tests, do so with a raw vanilla build (posted above) and with the patched build at threads 1 thru 3 if your going to run tests on this so we have a good basis to compare. I'm going to work on compiling my ram buffer for windows so that we eliminate the hard drive as a bottle neck (which i've found it can be).

[legoman666]

Most of the encoding I do is super high quality at 720p, and at that high of a bitrate and high quality, it goes so slow that i could read aloud the 1's and 0's for the encoder and it still wouldn't bottleneck. :P

The tests I'll do will be tomorrow (current encode I'm doing will be done in 25 hours so I have to wait until that is done). To do the comparisons, I'll most likely try converting blu ray -> 720p at various bitrates and quality settings.

Thanks for the binaries :D

[morph166955]

I think I may end up doing the same to see if a larger frame into x264 will fill in the gaps. I'll load up a 1080p video and pipe it into x264 to see if the larger video does anything for me speedwize on this. i'm running a test now to measure how fast i can decode a dvd off of a drive and get that into memory to see what my upper limit is.

also note...i think i was wrong about the cpu heat thermal cut thing...i am now under the belief that its my ram thats reaching a thermal limit. fbdimms get HOT! i'm probably going to order out one of those ram coolers to see if that will make this play nice.

[morph166955]

ok so I just ran completely through x264 both passes without hitting thermal cut so I think im good with the whole heat thing assuming that i can keep this place cool finally so I think were good for running tests so aku...bring it on!

[legoman666]

Hmmmm I'm running into a problem when I try to use more than 1 thread on the thread pool build. Once it encodes the 3rd frame, it just stops.

avis [info]: 1280x688 @ 23.98 fps (501 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [warning]: VBV maxrate specified, but no bufsize.
encoded frames: 3/501 (0.5%), 3.10 fps, eta 0:02:40
I:\hdtv\megui>

no error message or anything. Is there some setting or command I;m doing incorrectly? :(

[morph166955]

whats the exact command your running and how are you passing the source to x264?

[legoman666]

--pass 2 --bitrate 10654 --stats "C:\innocence.stats" --ref 10 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --ratetol 2.0 --me umh --threads 2 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "C:\innocence.mkv" "C:\innocence.avs"

innocence.avs is a fairly simple job:

LoadPlugin("I:\hdtv\megui\tools\dgindex\DGDecode.dll")
LoadPlugin("C:\PROGRA~2\GORDIA~1\AviSynthPlugins\UnDot.dll")

SetMemoryMax(64)

mpeg2source("C:\innocence.d2v")

trim(1000,1500)
crop(0,24,1920,1032)
LanczosResize(1280,688)
Undot()

The same command works just fine with the vanilla x264.

[morph166955]

the only thing i can think of is that i didnt put the avis stuff in properly when i compiled it but i was under the impression that was something built into x264 and not an additional library i had to add. i assume you tested it with both of my builds as well as someone elses build?

[legoman666]

Yea, it worked fine in the vanilla x264 you compiled and it works fine with the r654 x264.exe I grabbed from x264.nl.

[morph166955]

intersting. whats your source?

[legoman666]

its an mpeg2 blu ray file (which the newest version of DGIndex supports, w00t)

[morph166955]

i just used the threadpool.02 on a hddvd source (scaled 1080p -> 720p) and used some really insane settings and i was able to get it to 90-95% cpu usage! wahoo!

[bob0r]

http://x264.nl/x264_654.exe
http://x264.nl/x264_654_thread_pool_02.exe

(difference between this and x264.nl version, this is not compiled with make fprofiled)


$ patch -p0 < x264_thread_pool.02.diff
patching file `encoder/slicetype.c'
patching file `encoder/encoder.c'
Hunk #5 succeeded at 952 (offset -6 lines).
Hunk #7 succeeded at 1310 (offset -6 lines).
Hunk #9 succeeded at 1526 (offset -6 lines).
Hunk #11 succeeded at 1819 (offset -6 lines).
patching file `encoder/ratecontrol.c'
patching file `encoder/analyse.c'
patching file `x264.c'
patching file `common/cpu.c'
patching file `common/cpu.h'
patching file `common/pixel.c'
patching file `common/common.c'
patching file `common/amd64/deblock-a.asm'
patching file `common/common.h'
patching file `x264.h'