Low CPU usage in 1-pass

[JnZ]

Hello all,
first of all, I try to search forum but nothing found about this problem, so write new thread.

I have Core i7 and making some tests, but when try to encode, 1-pass give me only about 18% CPU usage. 2nd pass almost 100%.

Using simple command:
x264.exe --pass 1 --bitrate 1500 --deblock -2:-2 --stats x.stats --preset veryslow --output NUL "a.avs"

a.avs is simple too:

LoadPlugin("DGDecode.dll")
mpeg2source("a.d2v")
crop(0,42,720,488)
LanczosResize(720,336)


Source is PAL DVD.

I've try many settings, but still same problem. Same problem on second encoding machine with quad Opteron.

Any explanation for this?:thanks:

[nurbs]

Has been discussed often. Try this (http://forum.doom9.org/showthread.php?t=147018) and :search:

[LoRd_MuldeR]

Do you use x264 r1246 or later? The "threaded lookahead" introduced in r1246 should significantly reduce the lookahead bottleneck.

But it doesn't completely remove the bottleneck, because lookahead only runs in its own separate thread now. The lookahead itself still is not multi-threaded.

Anyway, the "veryslow" preset uses a maximum of eight b-frames. You could try appending "--bframe 4" and see if that helps...

(And of course you could always use "--slow-firstpass" to make the first pass slower, which should hide the lookahead bottleneck and increase CPU load)

[JnZ]

Do you use x264 r1246 or later?
I use the latest one. I noticed this behavior since commandline of x264 was changed (I don't know exact version, I think some around 1050). Before this change 1st pass gived me always 100% usage.


Anyway, the "veryslow" preset uses a maximum of eight b-frames. You could try appending "--bframe 4" and see if that helps...
It happens at any preset. I try "--bframes 4" and report then.


(And of course you could always use "--slow-firstpass" to make the first pass slower, which should hide the lookahead bottleneck and increase CPU load)
I tried it. but still only about 60% and fps ai almost same like without --slow-firstpass.

Someone mentioned --b-adapt 2 is slow and no multithreaded well. So I tried --b-adapt 1, but with no influence.

Strange thing, older x264 versions runs like hell (always 100%), but now I can't reach such usage.

[LoRd_MuldeR]

I use the latest one. I noticed this behavior since commandline of x264 was changed (I don't know exact version, I think some around 1050). Before this change 1st pass gived me always 100% usage.

This makes no sense. If at all, the "Threaded Lookahead" should improve the performance. Not make it worse...

Someone mentioned --b-adapt 2 is slow and no multithreaded well. So I tried --b-adapt 1, but with no influence.

Not exactly. Frame-type decision is part of the Lookahead.

And Lookahead is what is not multi-threaded yet. Also "--b-adapt 2" makes the Frame-type decision (and thus Lookahead) slow, especially with high b-frame count.

However since r1246 the Lookahead runs in its own separate thread, which should mostly remove the Lookahead bottleneck...

[kemuri-_9]

I use the latest one. I noticed this behavior since commandline of x264 was changed (I don't know exact version, I think some around 1050). Before this change 1st pass gived me always 100% usage.

latest from where? latest means little as different places can have different revisions to be referred to as latest
so latest specifies no useful information.
provide the buid's revision number, attainable by going
x264 --version

[JnZ]

latest from where? latest means little as different places can have different revisions to be referred to as latest
so latest specifies no useful information.
provide the buid's revision number, attainable by going
x264 --version

Ok. Official latest at x264.nl, exactly rev. 1251.

[nurbs]

Since the core i7 has hyperthreading do you see 4 or 8 cores in the task manager? If it's 8 then the 18% could be OK. One core is maxed out with the lookahead (8 b-frames and --b-adapt 2) and another is doing the actual encoding which should be pretty fast considering the low resolution and fast first pass settings.

[JnZ]

Since the core i7 has hyperthreading do you see 4 or 8 cores in the task manager? If it's 8 then the 18% could be OK. One core is maxed out with the lookahead (8 b-frames and --b-adapt 2) and another is doing the actual encoding which should be pretty fast considering the low resolution and fast first pass settings.
I see 8 cores in task manager.
Without --slow-firstpass it gives me about 18% and 56fps.
When I turn on --slow-firstpass, cpu is at 60% and still about 56fps.
Ok. lets say, that i7 is very powerfull, but I experiencing same problems on my old quad opteron (only with about half FPS)...but speed with and without --slow-firstpass is almost same.

[kemuri-_9]

x264 generally only manages to consume 1 core of many when a couple conditions can happen:
A. your input source is too slow to allow x264 to process quickly (avisynth for avs scripts or I/O accesses for y4m/yuv is taking too long)
B. the lookahead-specific settings are way beyond that of the encoding-specific ones. (encoding threads use practically no cpu and all time is spent in the lookahead)

from what you've stated it looks like you're in the position where A is the culprit.

[GZZ]

I had the same 'issue' and still have. I dont think x264 is the only bottleneck. I change to DGDecodeNV that uses the Nvidia GPU for decoding and I went from 10-15 fps on first pass (60% CPU) to 50-70 fps on the same pass (CPU usage is now 60-80%, never 100%). Also my 2pass is much faster. I'm getting around 20-25 fps, its around 10 fps+ faster then without the GPU. So I think the issue is the decoding filter (not x264) that limits the amount of data parsed to x264 and in that way the cpu usage goes down.

My sciprt can be seen here: http://forum.doom9.org/showthread.php?t=149426

[Chengbin]

I think it is not the input that's bottlenecking (I don't see how an i7 cannot decode a SD MPEG 2 video at more than 56fps with 1 core). I think it is x264's bottleneck.

I'm sure x264 will get support for multithreaded lookahead. For now, it is only 1 thread.

BTW, 56fps is very fast.

[foxyshadis]

With i7, at least that single core is probably boosted to be much faster than the others, so it's not so bad. Consider using crf if you just can't live with the single-core slowdown, or use --b-adapt 1.

[Selur]

or use --b-adapt 1.
didn't he already try that?
Someone mentioned --b-adapt 2 is slow and no multithreaded well. So I tried --b-adapt 1, but with no influence.

[roozhou]

I see 8 cores in task manager.
Without --slow-firstpass it gives me about 18% and 56fps.
When I turn on --slow-firstpass, cpu is at 60% and still about 56fps.
Ok. lets say, that i7 is very powerfull, but I experiencing same problems on my old quad opteron (only with about half FPS)...but speed with and without --slow-firstpass is almost same.

Well it seems decoding(avs) is the bottleneck. Try piping from mencoder or ffmpeg.

[Rumbah]

LanczosResize(720,336) can slow down your encoding significantly.

Try opening your avs file in Virtualdub and choose Run video analysis pass in file menu. Click off the preview and then look at the fps and your cpu usage. This should give you the maximum script speed and cpu usage for this script.

[roozhou]

On my E8400 software decoding of SD MPEG2 content can run at 300+fps using ffdshow's libmpeg2 decoder, with only a single core.
And decoding + downscaling using ffdshow runs at 100+ fps.

DGDecode and avisynth's internal resizer are significant slower than ffdshow.

[JnZ]

After long retesting, trying on different DVD source, I found some new information, so I must take this thing to the "new light".

Using --slower profile and 64bit x264:

1) --slow-firstpass with/without speeds and CPU
Without --slow-firstpass i have more fps than with this switch (112 vs 36 fps ), where CPU usage is ~25% vs ~90%.

2) switching --b-adapt 1 will give: 169 fps/~20% vs 30fps???/~90%.

Strange thing, --b-adapt 1 slows x264 with --slow-firstpass.

Considering all this things, I think everything is OK on x264 at i7. Probably bad first source?

LanczosResize(720,336) can slow down your encoding significantly.
Try opening your avs file in Virtualdub and choose Run video analysis pass in file menu. Click off the preview and then look at the fps and your cpu usage. This should give you the maximum script speed and cpu usage for this script.
AVS script speed is about 200fps.