Question about CPU and encoding performance

[Bleck]

Hello.
Im not expert with this, but I bought Core i7 3820 3.60GHz to use x264, avc, and other codecs at high speed. But for unknow reason, Im having ridiculous speed using placebo or veryslow presets. I tried to use 64 bits builds of ffmpeg and x264.exe but the same. I get max. 30 fps at first pass and 1-5 fps second pass. My last resource was trying to use VP8 but I have very slow speeds too.
Any idea please? I think I wasted my money because this processor is performing to me same as my old burned i5 650.
Thanks

This is the log for the last VP8 encode:
Pass 1/2 frame 9851/9852 1418688B 1152b/f 28774b/s 211299 ms (46.62 fps)←[
Pass 2/2 frame 9851/9842 296152258B ←[K 1240F 9514F 2686F 9632F 1350F
Pass 2/2 frame 9851/9857 296227923B 240566b/f 6008161b/s 1216844 ms (8.10 fps)←

im using:
vpxenc.exe input.avi -o output.vp8 --i420 -w 1280 -h 720 -p 2 -t 8 --good --cpu-used=0 --target-bitrate=2000 --end-usage=vbr --auto-alt-ref=1 --fps=25000/1001 -v --minsection-pct=5 --maxsection-pct=800 --lag-in-frames=16 --kf-min-dist=0 --kf-max-dist=360 --token-parts=2 --static-thresh=0 --drop-frame=0 --min-q=0 --max-q=60

x264 benchmark results (i found that this is not using the best settings for the bench test):

--- Now Starting 64-bit Run 1 : Pass 1 of 2 ---
encoded 11812 frames, 61.01 fps, 7753.87 kb/s
--- Now Starting 64-bit Run 1 : Pass 2 of 2 ---
encoded 11812 frames, 12.79 fps, 8002.25 kb/s

Detailed CPU: http://valid.canardpc.com/cache/screenshot/2601000.png
http://valid.canardpc.com/2601000
Whole computer: http://speccy.piriform.com/results/PEkdHaX2jRPPrli5gTFKWWk

what Im saying is im getting lower speed than with my old i5 650.

[littlepox]

Do you really think your CPU is good enough to handle --preset veryslow or --preset placebo?
Your CPU has only 4 cores; I know it is expensive but that does not mean it can deliver the speed you wish to see.
btw, 3820 is probably the worst choice for high-end encoding CPU —— it's as fast as a much cheaper i7 2600.

[Bleck]

Do you really think your CPU is good enough to handle --tune veryslow or --tune placebo?
Your CPU has only 4 cores; I know it is expensive but that does not mean it can deliver the speed you wish to see.
btw, 3820 is probably the worst choice for high-end encoding CPU —— it's as fast as a much cheaper i7 2600.

http://static.techspot.com/articles-info/492/bench/Encoding_01.png
http://images.anandtech.com/graphs/graph5276/43305.png
http://www.guru3d.com/miraserver/images/2012/i7-3820/Untitled-18.png

[littlepox]

Is a bit more fast over 2600k in some reviews.

not exceed 8% for x264. if you are using X79, the best buy would be 3930K, 50% better than 2600.

Your 3820K is just doing what it can for you. It's better not to under-estimate the last two preset in x264.

actually, I don't even dare to use placebo when cooperating with my friend for a BDRip project, when he was providing me with a system of dual E5: 16 cores, 32 threads. We tried but it was too slow and then we decided to use a modified setting based on --preset veryslow.

[littlepox]

handbreak test is not accurate enough to reflect x264 performance. they are different softwares.
The only reliable test is x264-HD, see how much a unoverclocked 3820 outperform 2600K. And remember 2600K is also over overclockable.

[Bleck]

thanks for your information

[Bleck]

so what settings do you reccomend for my system?

[littlepox]

Normally the --preset slow gives a good trade off between speed and quality, and slower should be more applicable to you since you have a reasonably strong CPU.
The highest I would recommend is veryslow, only when you wish to encode something you really care about the coding efficiency.
placebo should never be used as the benefit for quality is too marginal, and the speed is just unbearable for normal human beings.
Remember to choose proper --tune settings. For example, if you are encoding animation, use --tune animation.

[littlepox]

furthermore, unless you wish to control file size precisely, or you have plenty of time to waste, it is never optimal to use 2pass. crf is always preferred.

[littlepox]

If you are troubling with slower speed than i5, what I can recommend is to use the task manager to see whether x264 is running with full occupation of CPU, and use CPU-Z to monitor the frequency.
If there is other software taking up your cpu resources, that's it;
If during the encoding you see the frequency drop to very low level, probably you don't have your CPU fan well-installed.

[Bleck]

Huge thanks for the info. So the fan would be a possible issue? Im used to get my core overheated but not more than a maximun of 70 celsius degrees, and this only at encoding / benchmarks.

I will try to make video at 960x540, 25 fps, veryslow 1 pass settings, following your advice.:cool:
The apps i have in background are Steam (idle), the gpu fan utility, fraps (idle) and not much more (nvidia panel and shit). I dont even have antivirus.

[littlepox]

Huge thanks for the info. So the fan would be a possible issue? Im used to get my core overheated but not more than a maximun of 70 celsius degrees, and this only at encoding / benchmarks.

I will try to make video at 960x540, 25 fps, veryslow 1 pass settings, following your advice.:cool:
The apps i have in background are Steam (idle), the gpu fan utility, fraps (idle) and not much more (nvidia panel and shit). I dont even have antivirus.

The concern is that if you got your cpu overheated, it will automatically down-clock itself, which should be detectable by CPU-Z or other softwares.(a better recommend will be Intel Turbo Boost Technology Monitor). 70 degree is not something terrible, but there is still the chance that your BIOS settings demand a down-clocking at that temperature. If you did see a down-clocking with a low temperature like 70 degrees(normally 85 degree is something dangerous, and 90+ is definitely telling you to look into the CPU fan), check your BIOS settings.

don't mix crf mode vs 1pass mode(if you do), I was saying you should use Const Quality Mode (--crf x where x is commonly chosen between 16-26)

And unless you have good reasons, do not modify the frame rate of your video source. something like 23.976->25 or 30->25 is really a bad idea.

[nhakobian]

Another thing to check is to see if you are decode limited. x264 (or any encoder) can only encode as fast as it is given frames. A good way to look for this is to see if your CPU usage is at 100% while running any encoding job. If its not, then there is a bottleneck somewhere.

I have a 2600K, and when I was encoding some of my personal DVDs, I had to run 4 parallel encodes since I was performing some intensive processing in avisynth (with filters that are not multi-threaded). In the end, with 4 parallel encodes, I was running about 18-20fps (so 4-5 fps in each encode) and was only hitting about 80-85% cpu usage.

With HD material its much more variable (complexity of source, settings, etc.). You may also want to post a detailed x264 command line and if how you are decoding the material as people may have run into your issue. If you think you are CPU decode limited, you may also want to look at some of the hardware H.264 decoders that support piping output into x264.

[Bleck]

Now Im using this:

@echo off
x264.exe --pass 1 --bitrate 2000 --threads 8 --profile high --preset veryslow --tune film -o output.264 input.avi
x264.exe --pass 2 --bitrate 2000 --threads 8 --profile high --preset veryslow --tune film -o output.264 input.avi
ffmpeg.exe -y -i input.avi -vn -acodec libvorbis -ar 44100 -ab 64k audio.ogg
mp4box.exe -add output.264 -add audio.ogg final.mp4
del /f /q output.264
del /f /q audio.mp3
pause
exit

http://s15.postimage.org/d6x5zylgr/1pass.jpg

I used other commands, in ffmpeg, and Bencos, all of them slow as hell (under 20 fps). I will keep testing with your tips up.
And the reason I use2pass is because I need the smallest possible filesize. I got only 64 kbps upload speed and its painful when uploading to youtube.

[littlepox]

Now Im using this:

@echo off
x264.exe --pass 1 --bitrate 2000 --threads 8 --profile high --preset veryslow --tune film -o output.264 input.avi
x264.exe --pass 2 --bitrate 2000 --threads 8 --profile high --preset veryslow --tune film -o output.264 input.avi
ffmpeg.exe -y -i input.avi -vn -acodec libvorbis -ar 44100 -ab 64k audio.ogg
mp4box.exe -add output.264 -add audio.ogg final.mp4
del /f /q output.264
del /f /q audio.mp3
pause
exit

I used other commands, in ffmpeg, and Bencos, all of them slow as hell (under 20 fps). I will keep testing with your tips up.
And the reason I use2pass is because I need the smallest possible filesize. I got only 64 kbps upload speed and its painful when uploading to youtube.

You should have shown us this earlier...
--threads should never be manually set unless you are limiting the threads x264 can use. and notice in order to maximize the multi-threading efficiency, x264 set the threads to be the number of logical cores * 1.5. that is, threads = 12 may be better for your case. simply delete this parameter and see what happens.

[Bleck]

Ok I removed the thread command:

Pass 1:

http://s9.postimage.org/sogfy6rvj/pass1.jpg

Pass 2:

http://s7.postimage.org/gusjyg917/pass2.jpg

[littlepox]

btw, 20fps is really not considered slow.
different sources may require different amount of computation during encoding. imagine a video with still images and another with very complex scenes like explosion, do you think they will be processed at the same speed, holding others equal?
so your worry that your new system is slower than your previous one may be just a illusion. Are you testing the two with the same set of scheme, or simply saying: now I can only encode 720p @ 20fps, while my old computer used to encode @ 25...

[Bleck]

Yes I think Im a bit paranoid, but its just that I expected something like 60 fps @second pass.
And not, Im not testing the same files.
I will take a look at the bios later ...

Thanks for your tips. I will stand my processor and keep testing stuff.. 1 pass...
(I encode uncompressed avi files from sony vegas, that were renderized at the sony yuv codec. i had problems with some x264 builds that crashed with that sources or took ages to index).

[littlepox]

You may try to use some image bam websites to make your picture readable...

[littlepox]

60fps @ second pass......Are you serious?
try --preset ultrafast/superfast/veryfast/faster/fast if you really wish to achieve that.
Otherwise, upgrade your computer with Quad E5, 8C16T x 4.

[Bleck]

Now with a similar file (960x540 25 fps yuv) and same commandline I get more speed:
http://s13.postimage.org/q0514und3/insane.jpg
http://s13.postimage.org/buz831ebr/insane2.jpg

My PC is haunted. :scared:

[nhakobian]

Of course you will encode at a higher speed when using input that is a lower resolution!

Your other encodes are 1280x720. (960 * 540) / (1280 * 720) ~= .56. When you have roughly have the visual information in a frame, it can process more frames per second.

You still havent said if your CPU usage is hitting close to 100% when encoding. What format is your source in? You hinted in your last post that the source is YUV (possibly raw yuv)? In that case, your hard drive might be an input bottleneck at 720p, but not with your sample at 960x540.

Just some ideas.

[Bloax]

Different sources of different complexity encode at different speeds, and by the looks of it - the second source is easier to encode (assumed from much lower bitrate).

Not to mention that lower bitrates are usually easier to encode either way. (Though that might all be due to CABAC.)

Though if you really want to clarify the "HDD bottleneck" theory going off for some reason, try reading the source off a RAM drive and doing -o NULL for outputting.
I'd expect it to be faster too, yeah - but on the bright side, at least you don't get 4 fps on a 640x480 source with --preset veryslow like I do.

[06_taro]

Try some sources with yuv420p pixel format and see what speed it can archive.
swscale may sometimes become the bottleneck in fast encoding.

And if your sources were raw yuv (as you mentioned before, but I'm not sure whether it referred to rawvideo or other codecs with yuv pixel format) then ignore this, but if not, internally ffms uses only single thread for decoding, which may also become another bottleneck.

[Bleck]

When raw:
http://s10.postimage.org/3sjp7yvd5/codec1.jpg

When yuv:
http://s10.postimage.org/j2jkf5qvd/codec2.jpg

All my videos are from Sony Vegas. Using the avi uncompressed option and sometimes the Sony Yuv option.

http://s15.postimage.org/hq56fuipn/vegas1.jpg

http://s15.postimage.org/3kzde19ob/vegas2.jpg

Now I will make a test to see the Cpu usage.

[Atak_Snajpera]

Good advice. Don't use intermediate uncompressed .avi.
Just install Frameserver DebugMode
http://www.debugmode.com/frameserver/

[Bleck]

Thanks, Im downloading it.

Also I recorded a quick file of my encoding environment:

http://www.mediafire.com/file/baxx7rdgiwgid4w/final.mp4

EDIT: Installed the frameserver but cant find it anywhere on vegas? i fllow this too: http://www.debugmode.com/frameserver/usage.htm

[Atak_Snajpera]

http://www.youtube.com/watch?v=hntNbK_vpK0

[Bleck]

Im using Vegas Pro 12, totally different from your video GUI.
Another problem: Using the x264.exe to encode instead of ffmpeg.exe gives me this long extra process:
http://s16.postimage.org/jg3kx7ead/fafsd.jpg
What is it? Do I need that really long long check before the encoding process?

[nixo]

Do I need that really long long check before the encoding process?

try --demuxer lavf

--
Nikolaj

[Warperus]

Bleck
Quick note about Vegas. Sony YUV expects studio levels on timeline, avi rgb uncompressed/frameserver RGB expect computer levels on timeline (in studio gamma projects).

[Atak_Snajpera]

you need 32bit version of vegas. unfortunatelly sony abandoned x86 in latest version.

[Bleck]

Thanks for all of your help.
Now, what I have learned from you and online investigation:
-I have a good processor but not a server-like processor, so I will still getting slow speeds at "veryslow" and "placebo".
-"Placebo" is a useless preset, it only give 1% more quality from "veryslow".
-The best presets for daily encoding are "slower", "slow" and "medium". The quality is much alike with a great speed.

Maybe I will make my own balanced command line to keep more control over it.

Greetings :o

[Bleck]

--> Update.

I made a custom commandline:

x264.exe --demuxer avs --input-csp bgr24 --output-csp i422 --crf 25 --b-adapt 2 --rc-lookahead 50 --ref 8 --bframes 8 --b-pyramid 1 --me esa --direct auto --subme 10 --analyse all --trellis 2 --partitions "p8x8,b8x8,i8x8,i4x4" --weightb --mixed-refs --8x8dct --output video.264 input.avi

The source is uncompressed 960x540 25 fps avi. The average speed:

http://s12.postimage.org/8586jx1gt/weird.jpg

I still thinking is a really slow speed for my processor, and for that weak configuration...

[J_Darnley]

Why have you set --analyze all then overridden it with --partitions? Do you expect --me esa to be fast? Use the presets!

[Bleck]

Why have you set --analyze all then overridden it with --partitions? Do you expect --me esa to be fast? Use the presets!

Yes, but that commandline is similar to "slow" preset. With my processor it is weird anyway to have that speed... (with placebo, I get 0.5 fps in a 540p source )...

[Asmodian]

Switch to --me umh if you care about speed at all, --me esa is not used in any preset and really isn't worth the speed cost. (--me tesa is used in placebo)

Your command line is similar to the "slower" preset but with b frames set to 8 and --subme at 10, so somewhere between slower and veryslow. I assume "slower" is too fast and you want to slow it down for a little extra quality?

--crf 25? ouch, does it look ok?

[Soichiro]

As said above, --me esa is probably slowing you down (it doesn't provide much quality benefit above --me umh), and also, you shouldn't be specifying both --analyze and --partitions because they both do the same thing and one will override the other. But either way, when you get into veryslow/placebo presets, you shouldn't except to see speeds much above 20 fps for most content on any current processor. I use custom settings somewhere between slower and veryslow and I average about 12 fps on one-pass live-action content (Phenom II quad-core). Although animated content does encode much faster (20-30 fps) due to the lower complexity.