Log in

View Full Version : threadpool patch showing weird results. fastest at threads=cores*2


morph166955
9th April 2008, 17:47
I have been running some tests with the threadpool patch (http://stashbox.org/96770/x264_thread_pool.r798.diff) modified to work on r807 (needed one line changed in ratecontrol.c) and I am seeing some very weird results. It seems that I am getting the best speed from passes that have threads=16 (8 core machine). The only other patch that I have added to my build is x264_hrd_pulldown.04_interlace.diff. I fprofiled the build with the same yuv file that I used in this test. The code is what was current in git as of around 10am EST this morning. I can upload a diff of my folder or anything if anyone wants that.

The x264 cli that I'm running is "--keyint 24 --min-keyint 1 --vbv-maxrate 38000 --vbv-bufsize 30000 --nal-hrd --mvrange 511 --level 4.1 --bframe 3 --ref 3 --aud --sar 1:1 --fps 24 --partitions none --no-dct-decimate --me umh --subme 6 --crf 18 --threads x --thread-input". My goal is to create BluRay compliant files now that I got my hands on the new LG BluRay burner. The source is a 930 frame raw yuv clip from the BluRay of xmen3 (test source is at 1920x800). I realize that the 800 pixel height isn't BluRay compliant but its what I use to test.

The best output I have gotten (which is at threads = 16) is:
x264 [info]: file name gives 1920x800
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 SSSE3 Cache64
x264 [info]: slice I:104 Avg QP:14.73 size:237451 PSNR Mean Y:47.71 U:47.85 V:50.11 Avg:48.03 Global:47.67
x264 [info]: slice P:751 Avg QP:17.29 size:149069 PSNR Mean Y:45.15 U:45.95 V:48.76 Avg:45.69 Global:45.31
x264 [info]: slice B:75 Avg QP:18.71 size: 51904 PSNR Mean Y:43.84 U:45.79 V:47.86 Avg:44.58 Global:44.35
x264 [info]: mb I I16..4: 61.2% 0.0% 38.8%
x264 [info]: mb P I16..4: 70.4% 0.0% 0.0% P16..4: 25.8% 0.0% 0.0% 0.0% 0.0% skip: 3.9%
x264 [info]: mb B I16..4: 9.9% 0.0% 0.0% B16..8: 50.5% 0.0% 0.0% direct:28.5% skip:11.1%
x264 [info]: ref P 77.7% 14.2% 8.0%
x264 [info]: ref B 87.8% 12.2%
x264 [info]: SSIM Mean Y:0.9797549
x264 [info]: PSNR Mean Y:45.328 U:46.153 V:48.839 Avg:45.859 Global:45.425 kb/s:29014.39

encoded 930 frames, 25.31 fps, 29019.79 kb/s
256.71user 2.77system 0:36.78elapsed 705%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+105057minor)pagefaults 0swaps
Note that I have run it through /usr/bin/time to get the bottom two lines showing the CPU.

I was under the impression that my best results should be at 7 threads (--thread-input should add one more to that making it 8 threads total). I know its running in "thread pool mode" because if I watch the threads in top I get the time on each thread adding up as it should. Also to note (which I think is expected), the PSNR/SSIM/bitrate are different between different amounts of threads.

Any input is greatly appreciated. Thanks!

EDIT: just to note, the clip is from the final dark phoenix fight scene at the end as shes destroying the facility and everything is blowing up. Its crazy high bitrate at times (45mb/s in its raw h.264 format on the BluRay) so I use it since I figure it should max anything out that I run through it.

Sharktooth
9th April 2008, 18:14
https://forum.doom9.org/showthread.php?t=136663
you dont need the thread pool patch

morph166955
9th April 2008, 18:23
If I dont have it I lose 200-300% cpu usage (of the 800 total possible percent). When we ran tests months back we found that my system was fast enough that we were seeing a significant waste of time in the creation and destruction of the threads themselves. I have run tests with the newer versions (r798 was the most recent) and this still holds true for me.

MasterNobody
9th April 2008, 20:24
morph166955
Try also the --thread-queue option (not only --threads) it must be equal or greater than --threads. By default in my variant of thread-pool patch it is equal to --threads (so that patched and not patched versions of x264 give bit-identical results) but in http://mirror05.x264.nl/Cef/force.php?file=./patches/x264_thread_pool.r680.diff (which was the base for my modification) it was equal to 2*threads.

morph166955
9th April 2008, 23:01
ah...I'll modify that to 2*threads to see what happens. Probably the reason that this is happening. I'll run a few passes in the morning and post results. Thanks!