CPU usage percentage with X264 problem

[JonRead]

I was fed up with the low speed of x264 encoding (using Handbrake) with my Phenom x4 940 PC. So I thought buying a 1090T with 8gb of super fast ram would give at least 30 percent better speed when encoding. I have found though that I get only 3 to 4fps difference. With detelecine on I get 13fps on my new PC (was 10fps before) and without it on I get 21fps (was 17fps before).

I notice my CPU usage wont go above 50 percent even when I have put the level up to "above normal" or "high". It is mainly on 32 percent CPU usage which is very low. I bought this PC specifically for encoding so I am really frustrated that I spent a lot of money to get only 3 to 4fps more and finding my CPU isn't even going above half. Handbrake can support 6 cores or more.

What are the possible reasons here ? Even though I have installed Handbrake should I also get x264 from the net ? Is there anything else I could possibly do ? If it stays at this level then I have wasted my money as that is the main software I use for encoding and have for years.

Encoding 720p MP4 from 1080i TS. 1st pass turbo, 2 pass.

Thanks for any advice, my full setup is below (in case it helps).........

1. Windows 7 Ultimate 64bit
2. Handbrake 0.9.4 (latest)
3. 1090T AMD
4. 8gb 1333mhz DDR3 ram
5. GT9500 graphics
6. X-Fi Soundcard
7. USB3 add-on card
8. 8x HDDs, SATA
9. Blu-Ray Drive
10. Ext USB2 drive x2
11. Corsair 550 PSU
12. Artic Cooler Pro
13. Gigabyte GA-890GPA-UD3H Motherboard

[poisondeathray]

you probably have other bottlenecks, (like deinterlacing and resizing which might not be multithreaded) since you are going 1080i => 720p . So the cpu usage isn't optimized

[JonRead]

Sorry to sound a bit thick, but are you saying when I see FPS stats like 200fps from people with as powerful PCs as me, that is because they just use basic settings ? Would this explain why I only get 3fps more than a 940 because it is only using the extra MHZ between the 2 machines and not the extra 2 cores ? Even though handbrake says it uses 6 cores ? I apologise for being so dim, just trying to get my head around it all. So some settings use 6 cores and some don't, is that the crux of it all ? Also because the CPU usage stops around 30 percent, this is because it is using just 1 or 2 cores ? Rather than it not reaching the full 3200mhz ? Also CPU-Z never seems to show 3600mhz, even though I have the CPB enabled in bios.

Thanks for any advice.

[poisondeathray]

FPS stats like 200fps

200fps for a SD size encode like DVD, yes, but not for HD, and certainly not HD with filtering such as you are doing

A porsche 911 turbo can't go very fast if it has a flat tire

I don't use handbreak much, but I suspect the encoder is waiting for for data (=slack idle time) , because the filters are taking too long or poorly multithreaded. A straight encode (no filtering) with 720p material can use 95-100% on a 6core/12thread i7 using x264 , so certainly 1090T should be saturated as well

[J_Darnley]

12. Artic Cooler Pro
Clearly this is your problem! You need complete immersion cooling to get the full performance of your cpu.

[nm]

I notice my CPU usage wont go above 50 percent even when I have put the level up to "above normal" or "high". It is mainly on 32 percent CPU usage which is very low.

So, you have a filtering bottleneck but want to continue using HandBrake. Simple solution: run multiple simultaneous encodings. That way you also get to use some of that expensive RAM. :)

Btw, there may also be a bottleneck in x264 during the fast first pass due to b-adapt=2. Use CRF (or "constant quality" or whatever it's called in HandBrake) if you don't need a specific output filesize.

[JonRead]

Thanks everyone for the advice. Okay, my follow up questions.....

1. Comment about my cooler being the problem. Can this really be true ? I am not overclocking, have stock 3.2ghz. I think that cooler is good enough for that. I did buy a Coolermaster one that was even better but it wouldn't fit in my case, so had to make do.

2. If I run 2 versions of Handbrake I think that would only be of use for multiple files, if I can join them together to make one file encode use 2 versions of the software, then great. But I guess that can't be done ? Sadly I do need file size 2 pass, as I have to make sure I can fit the files on to a DVD disc perfectly otherwise I waste media.

[Dark Shikari]

thanks everyone for the advice. Okay, my follow up questions.....

1. Comment about my cooler being the problem. Can this really be true ? I am not overclocking, have stock 3.2ghz. I think that cooler is good enough for that. I did buy a coolermaster one that was even better but it wouldn't fit in my case, so had to make do.yhbt.

[nm]

2. If I run 2 versions of Handbrake I think that would only be of use for multiple files

Don't you have tons of different movies waiting to be encoded since you wanted to spend that much money on an encoding machine?

if I can join them together to make one file encode use 2 versions of the software, then great. But I guess that can't be done ?

That's also possible, but adds to the manual work unless you write scripts... And then you wouldn't need to use HandBrake and could avoid its bottlenecks in the first place.

Sadly I do need file size 2 pass, as I have to make sure I can fit the files on to a DVD disc perfectly otherwise I waste media.

Well, why do you want to put the files on DVDs? Isn't that a bit old-fashioned and really labour-intensive when we have cheap multi-terabyte HDDs?

[JonRead]

So what software does everyone else use ? I get the feeling I am alone in my use of Handbrake with x264 here. Is it MeGUI that everyone uses ? Would the bottlenecks still exist with other software ? I guess my best bet if so is your solution of running the software twice and doing 2 files at once.

[RunningSkittle]

Use x264 directly+multithreaded avisynth, no need for gui!

x264 input.avs --options here -o output.mkv

[nm]

Would the bottlenecks still exist with other software?

The same bottlenecks may exist in AviSynth, but most of them can be fixed by multithreading the filters.

It's more difficult to optimize HandBrake (and other integrated solutions like MEncoder and transcode) for parallel processing because you'd need to redesign their internal architecture.

[7ekno]

You lay out thousands in hardware without taking an hour to research what is needed to best optimize what you had originally?!? You spend thousands without getting a nice GPU that can offload decoding, resizing, cropping and deinterlacing for a major increase in encoding speed (and would have cost you less than 1 HDD in price)?!?

Given what you currently have , you need to research MultiThreaded Avisynth, De-interlace filters and x264 command line options (or a simple effective GUI to do the same) ...

What is being said here is that by using Handbrake, you are bottlenecked by the single threaded de-interlacer forced upon you by Handbrake ... by using other tools you may get a faster more effective deinterlacer (like QMTC, PureVideo via DGDecNV, etc) which may well speed up encoding drastically ...

Just incase the message hasn't sunk in: your choice of de-interlace filter is what is limiting CPU use of the x264 encoder ...

7ek

[Groucho2004]

You lay out thousands in hardware without taking an hour to research what is needed to best optimize what you had originally?!?

I was thinking the same thing.

Although as a beginner (I assume) it may be difficult to figure out who's blabbering nonsense on various forums and who isn't.

[Didée]

Hey, whats the problem? You got exactly what you wanted!I thought [..] a 1090T [..] would give at least 30 percent better speed when encoding.
I get 13fps on my new PC (was 10fps before)
10 fps before, 13 fps after makes a speedup of exactly +30%. :)


But seriously ... as everyone told, you are limited by the single-threaded filtering. Handbrake uses ffmpeg/libav based filtering, and there's not much threading in there.


What's the difference like? Small test!

Source: 1920x1080i (mpeg-2, "Shuttle" test sequence") - to be deinterlaced & resized to 1280x720
CPU: i7-860 (4Cores/8Threads) / 8GB RAM
encoder: x264 rev1745 (32bit) / settings: --preset veryfast , else default
Avisynth 2.5.8.5 MT (32bit)


Source & filtering singlethreaded:
mpeg2source("Shuttle.d2v")
Yadif()
bicubicresize(1280,720)
"x264 --preset veryfast" ==> 33.19 fps


Source singlethreaded / filtering multithreaded:
SetMTmode(5,4)
mpeg2source("Shuttle.d2v")
SetMTmode(2)
Yadif()
bicubicresize(1280,720)
"x264 --preset veryfast" ==> 65.50 fps


Source multithreaded / filtering multithreaded:
SetMTmode(5,4)
ffmpegsource2("Shuttle.mpg",threads=4)
SetMTmode(2)
Yadif()
bicubicresize(1280,720)
"x264 --preset veryfast" ==> 79.35 fps


You see, x264 is not the only variable. Source filter and video-processing filters are also important factors. And for maximum performance of a single job, they should be multithreaded, too.

[Dirk Diggler]

Turn off the Detelecine unless you really really need it and use Decomb instead

http://trac.handbrake.fr/wiki/Decomb

http://trac.handbrake.fr/wiki/DeinterlacingGuide
HandBrake now offers a new filter, called Decomb, that can be used instead of Deinterlace. It uses yadif in "Slower" mode, but only selectively.
Decomb only deinterlaces frames that are visibly interlaced. This saves time, and makes it safe, usually, to leave it on all the time. It won't destroy the detail in progressive sources the way Deinterlace will, and should be faster than "Slow" or "Slower".

http://trac.handbrake.fr/wiki/Telecine

[nm]

Turn off the Detelecine unless you really really need it and use Decomb instead

Hmm. Decomb and Detelecine filters are for different purposes, they can't be interchanged. Detelecine is to be used when the source is hard-telecined, Decomb when it's interlaced.

[Dirk Diggler]

Hmm. Decomb and Detelecine filters are for different purposes, they can't be interchanged. Detelecine is to be used when the source is hard-telecined, Decomb when it's interlaced.

Understood, I have read most users who think a source is Telecined probably got it wrong anyway. My post shouldve been a little clearer. Unless the OP knows for sure his source is Telecined he can continue to use if but if he is guessing he should probably use decomb instead (not that they are interchangeable but Decomb accomplishes deinterlacing a little faster than the deinterlacer handbrake uses as per my understanding of the WIKI) as the source is probably interlaced. Also IIRC Handbrake shows warnings for Telecined sources when it is scanned.

[JonRead]

Thanks everyone. Just to say though I certainly didn't spend thousands (on this upgrade anyway, over the years perhaps), total cost of upgrade was about £350 UK pounds. Also I can sell my previous hardware to get some of that money back. Okay, so that isn't cheap, but it certainly isn't thousands. I had all the rest of the hardware before the upgrade. Also I did research, I read endless x264 benchmark reviews for that chip, but it is true that I slipped up on not learning that different filters couldn't be multi-threaded. When I read "6 cores is better than 4 cores for x264" in many reviews, and also the stats being shown MUCH higher than my existing 940, I went for it. I didn't know enough about the different filters making a difference.

Didée, haha, yeah, I realised that 30% mistake after I posted it, oops. Thanks for the info. I will have to fiddle with the settings to see what's the best for a quality/speed setting. I guess I am just disappointed it didn't shoot up in speed as hoped, but that is my own fault, no one else's.

Regarding the telecine debate, how do I find out if my source footage is interlaced ? I know these are simple questions to all you experts, but the only way I find out is to ask and learn :) Everyone has to learn to start with.

[nm]

Regarding the telecine debate, how do I find out if my source footage is interlaced ? I know these are simple questions to all you experts, but the only way I find out is to ask and learn :) Everyone has to learn to start with.

If you see combing (http://upload.wikimedia.org/wikipedia/en/1/19/Interlaced_video_frame_%28car_wheel%29.jpg), it's either interlaced, telecined or possibly field-blendend, which is more difficult to salvage (AviSynth SRestore job). To separate pure interlacing from 3:2 telecine, pick a sequence with motion and go through it frame by frame. See if the combing patterns disappear from the frame and appear again in a following frame during steady motion.

More details here:
http://neuron2.net/LVG/telecining1.html
http://neuron2.net/LVG/interlacing.html