x264 Encoding much too slow ?

[LOGiC]

@all

I replaced my old PC system this weekend. Bought it at a retailer two years ago. It consisted of a Core2Duo E6300 with just two gigs of RAM. The crappy mainboard just supported single channel RAM. When using x264 for 1080p re-encoding I received around 1,5-2fps for a CRF18 1 pass with 3 ref frames, 3 bframes, subme 7 and trellis 2.

My new PC has a QuadCore Q9400 which is not overclocked, 4 Gigs of CL4-RAM, Asus P5Q-Pro board and some good cooling. Was curious yesterday evening and did some testing. As my Vista64 hasn't been delivered yet, I went with XP32 to test the functionality.

With the same settings I received about 3,3fps using x264 and to be honest I am a bit disappointed about this.

Any ideas if this really is normal with 2,67Ghz each core ? All Cores have 100% CPU load.

Any hint is appreciated.

Thanks in advance

[Snowknight26]

That seems normal, but it always varies depending on the source complexity.

[gigah72]

@CRF18, is it really worth for you the additional waitingtime to use subme7 & trellis2, maybe subme 6 & trellis 1 or 0 is still good enough?

[shon3i]

trellis2+subme7 is killer for any processor

[LOGiC]

Thanks for your answers yet. I believe that Subme 6 would fit my needs as well. I just thought it would be a bit faster. As I remember it right, Trellis 1 does not work together with Psy-RDO (Subme6) and therefore will be automatically disabled. But I guess in the end, there won't be any noticable difference in PQ.

I was just wondering as I really expected around 5-6fps more then 3-3,5fps with this CPU and I was asking myself, if everything is ok with it.

Someone I know who does 2pass Encoding told me he would get around 24fps at first pass and around 13fps for the second pass with an unclocked Q6600, so I really couldn't believe this.

Do you think I got a problem with my CPU ? What would you guys suggest, using Subme6 and switching Trellis completely off ?

[Wishbringer]

x264 scales very well with additional cores.
Q6600 is around two times faster than E6600 for example.
With the same motherboard I get with an OC Q6600 around 17fps with your settings.

Try to use programs like CPU-Z to find out, if your CPU is really at 2.67GHz and RAM at DDR2-800
Try CoreTemp to find out, if your CPU is too hot and throttles down.
Newest Bios used?

Hm, what resolution do you encode?

[LOGiC]

x264 scales very well with additional cores.
Q6600 is around two times faster than E6600 for example.
With the same motherboard I get with an OC Q6600 around 17fps with your settings.

Try to use programs like CPU-Z to find out, if your CPU is really at 2.67GHz and RAM at DDR2-800
Try CoreTemp to find out, if your CPU is too hot and throttles down.
Newest Bios used?

Hm, what resolution do you encode?


CPU-Z tells me that the CPU is at 2,67 Ghz and DDR2-800 is being shown. As I bought the boxed version of the CPU, the 4 cores are around 60° degrees after 1-2 hours of encoding, but it seemed to be stable. The CPU has TJMax 100° and I think 60-65° shouldn't be a killer, or ? I will replace the boxed cooler soon.

Is it the fault of WinXP32 or is there any other way to measure, if my CPU is running at 2,67 Ghz properly ?

I encode at 1920x1080.

[jeffy]

LoGiC, post your AviSynth script.

EDIT:
trellis 2 & subme 7 slowdown here:
http://forum.doom9.org/showthread.php?t=140687

[kemuri-_9]

As I remember it right, Trellis 1 does not work together with Psy-RDO (Subme6) and therefore will be automatically disabled. But I guess in the end, there won't be any noticable difference in PQ.

Um...
psy-rdo is on for subme >= 6, and psy trellis is on for trellis >=1
these are both automatically enabled in the patched builds as well.

[LOGiC]

@jeffy,

can do this tonight, as I am not at home right now. But just in advance, as I was installing my PC yesterday completely, I was curious about the speed. I used Directshowsource to get a mkv file that was encoded in 1920x1080. My script looked like :

DirectSHowsource("TEstfile.mkv",fps?23,975985......,audio=false)
#deinterlace
#crop
#resize
#denoise

It was very plain, just opened the file via AviSynth Script Editor for testing. Normally I use to index my files with DGAVCINdex, then I load them via AVSSource and have no more filters activated.

Thanks for the link to the other doom9-Thread about the slowdown of Subme7 and Trellis2. Someone still had about 8-9fps then. I don't know which CPU was used, but I just would like to make sure my CPU is alright. Any chances to measure ? I will go with Subme7 and Trellis1 tonight or Subme6 and Trellis1 to check.

[video_magic]

LOGIC: I know this is a non-expert answer, but I'll suggest it anyway. Have you checked for some common software causes?

Check for a virus on the system or some other 'process' taking up a lot of CPU time, or some comflict or lock-up. In services.msc from the Run command, close services that you don't need.

Do you have a lot of little programs running in the system tray (usually near the little clock)? Close ALL other programs that you don't really need to run.

In my opinion you should try to disconnect from the internet and then close any anti-virus and Norton Protection type programs.

Check your MSCONFIG from the Run command and the STARTUP tab. Unload all unnecassery programs.

Update and check your drivers.

I actually have done those steps over the years and I run a clean and tidy system, and mine is often remarked upon to be faster by a lot of people who own better spec hardware! Try all those steps.

[XhmikosR]

Why don't you try to run a benchmark to see how fast your pc is running in comparison with other pc configurations?

http://www.techarp.com/showarticle.aspx?artno=520

http://forum.doom9.org/showthread.php?t=135923

Even though the benchmark doesn't include latest patched builds with psy-rd you can see with numbers how fast is running your pc and compare the results with similar processors.

Also make sure you do what video_magic said. Most of the times there are running useless processes in the background. Make sure you close all of them and in order to have accurate results close any antivirus etc protection.

[audyovydeo]

I actually have done those steps over the years and I run a clean and tidy system, and mine is often remarked upon to be faster by a lot of people who own better spec hardware! Try all those steps.

Ditto !
I have a "before" and "after" batch files which have a list of "net stop" commands stopping a whole bunch of windoze services which really slow your system down. You can really gain fps and encoding time just by stopping your antivirus (most obvious) Windows Themes (no surpise) and (if applicable) NVidia Service (if you got NTune installed).
this is a shortlist of course.

cheers
audyovydeo

[LOGiC]

Thank you guys for all your helpful answers. I could imagine that my system slows the encoding down. I've installed WinXP32 yesterday completely new and all useless services and programs are still running. I didn't have the time to tidy up yet, as my curiosity about the new processor was too big, lol.

[squid_80]

I used Directshowsource to get a mkv file that was encoded in 1920x1080. My script looked like :

DirectSHowsource("TEstfile.mkv",fps?23,975985......,audio=false)
#deinterlace
#crop
#resize
#denoise

It was very plain, just opened the file via AviSynth Script Editor for testing. Normally I use to index my files with DGAVCINdex, then I load them via AVSSource and have no more filters activated.
There's probably half your problem - decoding a 1920x1080 AVC file can take up nearly half the power of a quad core if not more.

[Sharktooth]

As I remember it right, Trellis 1 does not work together with Psy-RDO (Subme6) and therefore will be automatically disabled. But I guess in the end, there won't be any noticable difference in PQ.
Not true if you're using a recent build that has psy-trellis too.

[Rumbah]

Not true if you're using a recent build that has psy-trellis too.

Do the MeGUI builds have psy-trellis? Some time back they only had psy-RDO.

Back to topic: I think that a faster decoder could help your encoding speed. Try ffmpegsource (perhaps a bit faster because it does not need DirectShow) or even better the new DivX decoder or CoreAVC.

[Sharktooth]

coz some time back psy-trellis didnt exist... however, yes, megui has psy-trellis too.

[LOGiC]

Ok Guys, I am at home now. Did some testing with x264 Benchmark HD. All first passes were around 55-57 and all second passes were around 15-16, so it seems to be ok.

What I am really wondering about is the fact that in "live mode" the speeds get down and down and down. I made a test with a 500MB file which I cut off from a bigger file. I did a two pass encode in 1080p with 3 ref frames, 3 b frames, subme7 and trellis1. First pass started with 30 and went down to 24, second pass started with 10fps and went down to 6,5fps. I mean, this isn't that bad, but is it normal ?

Temperatures were around 60° celsius, I think I really need a very powerful CPU cooler instead of the boxed version. Will fix that.

[jeffy]

What are you measuring the temperature with? CoreTemp/RealTemp?

[atreides93]

Although I'm not a big fan of overclocking, if you look at the x264 HD benchmark results for the Q9450 (i couldnt' find any results with your specific processor), if they run it stock, they're getting fps speeds for the 2nd pass of around 16...but if they overclock it from 2.66 to say 3.86 someone showed a 2nd pass speed of 22.92.
of course that's a lot of overclocking :)

My poor little athlon does the 2nd pass at 5.4 fps.

[RickA]

Greets,

Grats on the new comp. Your temps of around 60C do not sound too terribly high for stock air cooling under a full load with normal ambient room temps. Temps closer to 65C would start to concern me. I definitely would not want to venture towards 70C for any extended amount of time. Chances are you will start to throttle back about then. If you like you can seek out the max suggested temps for each CPU on the manufacturer's website.

There is a free program called RightMark CPU Utility you can use to tell if your system is throttling back. ThrottleWatch is another one, but it no longer wishes to work on my Q6600. I've read that TJMax of 100C is the maximum temp at the junction where the CPU chip meets / connects to the motherboard. Very bad things will happen if you flirt with those kinds of temps. Stars will begin to implode and galaxies collide - that sort of stuff. At the very least, your own personal universe will shaken to the core with the smell of molten silicon. As your new cpu / computer turns into a pile of smoldering goo before you very eyes. ;-)

You are crunching 1080p, so that is going to be a good deal slower than say 480i for SD DVD content. Try to take that into account as you look at your fps numbers. If you do some sample tests you can see encode times slow with Trellis 1 and get even slower with Trellis 2. I did not notice as large a speed hit between subme 6 and 7. At least that was the case in my tests several x264 versions ago. A personal choice to decide on the time / quality trade-off.

Cheers,
Rick

[LOGiC]

Greets,

Grats on the new comp. Your temps of around 60C do not sound too terribly high for stock air cooling under a full load with normal ambient room temps. Temps closer to 65C would start to concern me. I definitely would not want to venture towards 70C for any extended amount of time. Chances are you will start to throttle back about then. If you like you can seek out the max suggested temps for each CPU on the manufacturer's website.

There is a free program called RightMark CPU Utility you can use to tell if your system is throttling back. ThrottleWatch is another one, but it no longer wishes to work on my Q6600. I've read that TJMax of 100C is the maximum temp at the junction where the CPU chip meets / connects to the motherboard. Very bad things will happen if you flirt with those kinds of temps. Stars will begin to implode and galaxies collide - that sort of stuff. At the very least, your own personal universe will shaken to the core with the smell of molten silicon. As your new cpu / computer turns into a pile of smoldering goo before you very eyes. ;-)

You are crunching 1080p, so that is going to be a good deal slower than say 480i for SD DVD content. Try to take that into account as you look at your fps numbers. If you do some sample tests you can see encode times slow with Trellis 1 and get even slower with Trellis 2. I did not notice as large a speed hit between subme 6 and 7. At least that was the case in my tests several x264 versions ago. A personal choice to decide on the time / quality trade-off.

Cheers,
Rick

@Jeffy

I measure the temperatures with CoreTemp


@RickA

Thanks for your help. First of all I thought that I could definately go with the boxed cooler, but I think I will go for a real CPU cooler in the next days. At least summer will be passing by soon, so the overall temperatures in the room (about 22°-23° celsius) will go down as well, but I really don't want to get the CPU overheated. After 1 hour of testing I am above 60° each core, so what might be the value after 10 hours of encoding ? Don't want to get risky about this.

[gav1577]

@Jeffy

I measure the temperatures with CoreTemp


@RickA

Thanks for your help. First of all I thought that I could definately go with the boxed cooler, but I think I will go for a real CPU cooler in the next days. At least summer will be passing by soon, so the overall temperatures in the room (about 22°-23° celsius) will go down as well, but I really don't want to get the CPU overheated. After 1 hour of testing I am above 60° each core, so what might be the value after 10 hours of encoding ? Don't want to get risky about this.

Temps should never be that high at stock even with stock cooler
you should reseat your heatsink and use some decent thermal paste

[Audionut]

60 with a stock cooler aint that bad. While we're talking about temps, look at this little prog.
http://downloads.guru3d.com/IntelBurnTest-v1.6-download-2047.html

See some details here.
http://www.xtremesystems.org/forums/showthread.php?t=175729


I get about 13 degrees higher temps with this than orthos.

[Blue_MiSfit]

Yeah, 60c is no biggie.

Our Xeon servers used to run at 75c for extended periods of time under full load, no problems whatsoever.

My Q6600 loads at ~65c. No problems.

This isn't Athlon / Pentium 3 days anymore!

~MiSfit

[LOGiC]

Yeah, same opinion here. Max TJ is given with 100° celsius and cover temp with 70° or 71°, I really doubt that 65° celsius are a big issue. In my old PC, my old very very crappy PC without a good cooling concept, my Core2Duo got 3 years old (and it would get even a few years older if I wouldn't have changed the computer).

This Core2 E6300 was very often under load and surely above 70° celsius and really nothing happened. I think those things are given too much importance. My mainboard often was above 40° celsius in those three years. A no name mainboard. I never got a blue screen or any crashes.

[gav1577]

Yeah, 60c is no biggie.

Our Xeon servers used to run at 75c for extended periods of time under full load, no problems whatsoever.

My Q6600 loads at ~65c. No problems.

This isn't Athlon / Pentium 3 days anymore!

~MiSfit

Yeah but for a stock non overclocked Q9400 45 nm running at 60 degrees is high infact its just borderline safe zone and so is your Q6600 assuming its the go stepping model. If its the B3 stepping its way too hot.
http://www.tomshardware.co.uk/forum/221745-11-core-quad-temperature-guide

http://img28.picoodle.com/data/img28/3/9/3/f_123m_b11da34.png (http://www.picoodle.com/view.php?img=/3/9/3/f_123m_b11da34.png&srv=img28)

Obviously i doubt it will cause any immediate damage to his cpu
going over the recommended safe zone but prolonged use will affect the life expectancy of the cpu for sure. Imho unless you have money to burn i would suggest staying at as low temps as possible

[F J Walter]

I think that the type of person that uses subme 7 and trellis 2, and the type of person that is concerned about encoding speed, need to be two different types of people. To be both types at once is bound to end in frustration. Doubling your number of processor cores and speeding up a bit is not going to speed up a 1.5fps encode to 25fps, however relaxing some settings will. You may need to decide your priorities. But I personally would identify subme 7 as being not worth the hard to measure gain in quality for the significant drop in encoding speed.

[florinandrei]

I made some 1080p videos, I'm encoding them on an AMD Phenom quad core 9850 (2500 MHz clock), using MeGUI with the Standalone AVC-HD profile (x264), no rescaling, 2-pass, around 8000...12000 kb/s.

I usually get 7...8 fps during the second pass, a lot more during first pass (looks like 1st pass is limited by the single-threaded decoder that feeds the video to x264).

[Ranguvar]

Could be, or the fact that frametype decision is not multi-threaded yet, which can easily be a bottleneck for fast first pass :) You could test by checking the AviSynth script's playback speed... there was a thread a while back in the AviSynth forum I think about this.

[LOGiC]

@all,

thanks again for all your help on this thread. I just installed my new Noctua CPU cooler yesterday and even after 15 minutes Prime95 the CPU temps don't exceed 47-49° celsius anymore, this is great.

I will do a more-hour encoding test next weekend to see how temperatures will develop, but everything seems to be fine now.

By the way, does anyone know why CoreTemp and Everest report very different temperatures on all cores ? I really don't know which ones to trust. Everest reports about 5-6° higher temperatures all the time. While CoreTemp reports i.e. 32° in IDLE for one core, Everest reports 37°-38° for the same core.

[jeffy]

Try RealTemp.

[Ranguvar]

SpeedFan beta, IMO, is the best ;)