x264 running only at 60% on dual core?

[bur]

x264 only runs at 60% max mostly around 50% on my X2. Priority is set to normal and setting to higher doesn't help. No other processes are running that might interfere.

I have this avisynth:

LoadPlugin("d:\video\DGMPGDec\DGDecode.dll")
mpeg2source("four_rooms.d2v")
Crop(0,16,-0,-16)

Setting --threads to 2 manually doesn't help. Also removing crop from the script doesn't do anything.

[Dark Shikari]

How about you post your commandline? ;)

[bur]

E:\RS-Downloads\Four Rooms DVD>d:\video\x264\x264 --ref 1 --deblock -2:-1 --b-adapt 2 --bframes 16 --b-pyramid --direct auto --me dia --subme 2
--psy-rd 1.0:1.0 --trellis 0 --partitions none --pass 1 --bitrate 2000 --progress --no-ssim --no-psnr --stats "four_rooms_x264.stats" --output NUL --threads 2 "four_rooms.avs"

[Dark Shikari]

--b-adapt 2 --bframes 16 --threads 2:scared:

Well obviously it's not going to multithread very well; if you would read the documentation:

--b-adapt Adaptive B-frame decision method [1]
Higher values may lower threading efficiency.
- 0: Disabled
- 1: Fast
- 2: Optimal (slow with high --bframes)Not only does b-adapt 2 limit threading, you've made it incredibly slow by using an absurd --bframes value, making things even worse.

[LoRd_MuldeR]

x264 only runs at 60% max mostly around 50% on my X2. Priority is set to normal and setting to higher doesn't help. No other processes are running that might interfere.

I have this avisynth:

LoadPlugin("d:\video\DGMPGDec\DGDecode.dll")
mpeg2source("four_rooms.d2v")
Crop(0,16,-0,-16)

Setting --threads to 2 manually doesn't help. Also removing crop from the script doesn't do anything.

Try "--threads auto". This will actually use three threads on a Dual Core machine ;)

[bur]

Not only does b-adapt 2 limit threading, you've made it incredibly slow by using an absurd --bframes value, making things even worse.
Okay, I set up x264 with Brother John's site but made a mistake. He suggests to use --bframes 16 with --b-adapt 1 or --bframes 4 to 5 with --b-adapt 2. I somehow skipped the last part of the sentence and thought 16 would be advisable with any --b-adapt setting. :o

What would you suggest? b-adapt 1 with a higher bframes setting or b-adapt 2 with a lower one?


Try "--threads auto". This will actually use three threads on a Dual Core machineIt was set to auto at first, didn't help. Still good to know that auto defaults to 3 on a dual core machine.



edit: It works now but only when I specifically set --threads to auto or 3. I thought auto was the default setting?

[Forteen88]

I'd use --b-adapt 2 and --bframes 3 for Blu-ray compatibility but also for very good quality. You don't gain any quality using 16 bframes, even animated film most usually don't need that many bframes.
--psy-rd 1.0:1.0 is a bit strong (causes artifacts) psy-trellis, use --psy-rd 1.2:0.2 or --psy-rd 1.2:0.0.

[LoRd_MuldeR]

edit: It works now but only when I specifically set --threads to auto or 3. I thought auto was the default setting?

No! The default settings is "--threads 1" :scared:

Specifying "--threads auto" (aka "--threads -1") will use number_of_cores * 3/2 threads, which should give 100% CPU load...

[nm]

"--threads auto" (aka "--threads -1")You mean (aka "--threads 0").

[bur]

I used psy-rd 1.0:1.0 because I found a thread here where someone did some tests and though the samples weren't available anymore most posts said 1.0:0.8 or 1.0:1.0 or even 1.0:1.2 was liked most. It also said 1.0:1.0 was the default before. Maybe I'll encode a sample again with 1.0:0.2 as you suggested and see if I like that more.

[LoRd_MuldeR]

You mean (aka "--threads 0").

eh, yes :D

I used psy-rd 1.0:1.0 because I found a thread here where someone did some tests and though the samples weren't available anymore most posts said 1.0:0.8 or 1.0:1.0 or even 1.0:1.2 was liked most. It also said 1.0:1.0 was the default before. Maybe I'll encode a sample again with 1.0:0.2 as you suggested and see if I like that more.

Generally Psy-Trellis 1.0 is a bit too strong. You may be better off with something like "--psy-rd 1.0:0.2". But you'll need to do your own tests, of course...

[burfadel]

--b-adapt 2 with a lower b-frame is suggested over --b-adapt 1 with a higher b-frame. --b-adapt 2 is more efficient at finding where b-frames should be inserted, and actually typically uses more b-frames than --b-adapt 1. Around 5 b-frames is a good value. Isn't psy-trellis disabled anyway if you have trellis set to 0 as he does?

I'd use a higher subme too, I thought it was generally suggested a subme of less than 5 isn't recommended unless speed is a critical issue or for testing purposes. It probably wouldn't hurt the encoding performance to much, just raise the cpu usage by setting it to 5 or 6 for example!

[Forteen88]

Isn't psy-trellis disabled anyway if you have trellis set to 0 as he does?Yeah, you're right (I didn't notice that first), so there's no point in having that line there (as default psy_rdo is 1.0:0.0, which is the same).
I assume he's deliberately doing a turbo-pass1, so that's why subme is so low.

[burfadel]

Ah ok! I missed that too :)

[bur]

Yes, it's for 1st-pass.

It seems most people here prefer low settings for psy-trellis. I got the 1.0 setting from this thread (http://forum.doom9.org/showthread.php?t=141249). It was suggested in regard to a specific video sample though, so on the whole you'd say values from 0 to around 0.2 are generally better? Why was 1.0 the default then earlier?

[Forteen88]

It seems most people here prefer low settings for psy-trellis. I got the 1.0 setting from this thread (http://forum.doom9.org/showthread.php?t=141249). It was suggested in regard to a specific video sample though, so on the whole you'd say values from 0 to around 0.2 are generally better? Why was 1.0 the default then earlier?Because high (over ~0.4) psy-trellis value causes artifacts. More info about psy-trellis here: http://forum.doom9.org/showthread.php?t=146552

[infinityloop]

i have a simmilar issue.
my core i7 has a overall cpuload of ~25% when encoding a PAL-DV (or mpg2) to x264 using xvid4psp, handbrake, or others. :confused:
other apps like the elecard encoder studio (h.264) or Adobe Premiere have no troubles at all to max out all 8 cores and therefore encode a lot quicker.

264 [info]: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0 me=dia subme=1 psy_rd=0.0:0.0 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0
cqm=0 deadzone=21,11 chroma_qp_offset=0 threads=8 thread_queue=8 nr=0 decimate=1 mbaff=1 bframes=3
b_pyramid=1 b_adapt=1 b_bias=0 direct=1 wpredb=1 keyint=250 keyint_min=25 scenecut=40(pre) rc=abr bitrate=1656 ratetol=1.0 qcomp=0.60
qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00

[burfadel]

Going by the settings listed, its probably the avisynth path thats causing the slowness! I presume thats first pass settings too! Shouldn't threads be set to 12 on an '8 core' (I know its 4 hyperthreaded cores, but still).

[kemuri-_9]

i have a simmilar issue.
my core i7 has a overall cpuload of ~25% when encoding a PAL-DV (or mpg2) to x264 using xvid4psp, handbrake, or others. :confused:

you stated no information on your input source to x264, this is also necessary information to help you.

but from your speedy settings, the possibilities are
A. you're being limited by the current x264 threading model in which slicetype decisions are not yet threaded.
B. you're being limited by your source (either its own processing (AVS) or size/locality (I/O))

some misc points:
C. you're using a customized build of x264, it could be broken.
D. you're using an old build of x264, update to the latest.

[Forteen88]

Going by the settings listed, its probably the avisynth path thats causing the slowness! I presume thats first pass settings too! Shouldn't threads be set to 12 on an '8 core' (I know its 4 hyperthreaded cores, but still).That's probably better (for more usage of the cores), but the videoquality suffers the more threads you use.

[burfadel]

I was forgetting that, its not really an issue even with a quad core! The main thing is, the settings listed are very fast (assuming its first pass, otherwise the settings are crap :D). The avisynth chain is unable to feed x264 the frames quick enough, therefore slowness occurs. This is of course, assuming avisynth is used. I found that every filter I use can be threaded except for the decode thread (DGDecode most typically), which is unfortunate. Threading of it can cause crashes. For all other filters, I can use (such that its the second line of the script):

SetMTMode(1,4)

Which is handy!

If this is done, you are only restricted by the decode thread. The decode thread being single threaded will still slow it down, but by no where near as much.

[infinityloop]

you stated no information on your input source to x264, this is also necessary information to help you.

but from your speedy settings, the possibilities are
A. you're being limited by the current x264 threading model in which slicetype decisions are not yet threaded.
B. you're being limited by your source (either its own processing (AVS) or size/locality (I/O))

some misc points:
C. you're using a customized build of x264, it could be broken.
D. you're using an old build of x264, update to the latest.
Thank you for your reply. :)

i have to admit that i do not really know what is going on in the background of xvid4psp or handbrake. :(

all i did was to let them load a PAL-DV (avi) or a PAL mpeg2 (dvdrip/dvb-recording), which are stored on my 2x 1TB Seagate - Raid 0 (yes stripeset, not mirrow. only used as temp-drive. :) ).

in xvid4psp there is no setting to define the number of threads x.264 uses.
it sets the 8 threads automatically.

i've been reading through many other forums, and i heard many times that avisynth could be the bottleneck.
but i've not read any solutions for this. :(

[frank]

dgmpgdec/dgDecode only uses ONE cpu core!! No multithread.

[infinityloop]

dgmpgdec/dgDecode only uses ONE cpu core!! No multithread.

Shouldnt it then max out at least one of the cores then?

[Manao]

Shouldnt it then max out at least one of the cores then?Not necessarily. The OS can schedule a single thread application to run on both cores, alternating between both cores quickly (every 100ms or so). A single thread program doesn't care if there's a change of core.

[LoRd_MuldeR]

If you really want to execute a program on one single core, you can set the "processor affinity" with ProcessExplorer. But there usually is no reason to do that.

[Sharktooth]

Not necessarily. The OS can schedule a single thread application to run on both cores, alternating between both cores quickly (every 100ms or so). A single thread program doesn't care if there's a change of core.
bad idea... amd dual core optimizer was made to prevent that and it worked pretty well.
i dont know what microsoft engineers smoked when coding the windows kernel...

[LoRd_MuldeR]

bad idea... amd dual core optimizer was made to prevent that and it worked pretty well.
i dont know what microsoft engineers smoked when coding the windows kernel...

Probably the idea was that distributing the load is better than putting all load (heat) on one single core while the other core is completely idle. So it may be a "stability" thing.

If not before, this way of thinking is wrong with Core i7 now. That's because it will automatically "overclock" one core and put the other cores to sleep when there is only one thread running.

So switching a single thread between different cores may prevent this optimization form being enabled and thus hurt the performance significantly...

(I don't know if the latest Windows versions have special support for Core i7 though)

[Arshad07]

Yea, i'm on a quad extreme and its not using all the cores :confused:

Commandline:

program --pass 2 --bitrate 5562 --stats ".stats" --level 4.1 --ref 4 --mixed-refs --bframes 3 --weightb --direct auto --deblock -1:-1 --subme 7 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh --threads 4 --thread-input --progress --no-psnr --no-ssim --output "output" "input"

Screen:-

http://i40.tinypic.com/2reo1l3.png

[J_Darnley]

1 - Why --threads 4? x264 would choose 6 or 12 (if that thing has HT) threads
2 - What is your source? Post the full script if you are using Avisynth.

[Arshad07]

I thought threads 4 meant 4 cores, isnt that right?

Script:

LoadPlugin("mt_masktools-26.dll")
LoadPlugin("mvtools.dll")
LoadPlugin("mvtools2.dll")
DirectShowSource("bluray", fps=23.976, audio=false)
Crop(2,134,2,134)
Spline64Resize(1280,544)
backward_vec3 = last.MVAnalyse(isb = true, delta = 3, pel = 2, overlap=2, sharp=1, idx = 1)
backward_vec2 = last.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=2, sharp=1, idx = 1)
backward_vec1 = last.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=2, sharp=1, idx = 1)
forward_vec1 = last.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=2, sharp=1, idx = 1)
forward_vec2 = last.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=2, sharp=1, idx = 1)
forward_vec3 = last.MVAnalyse(isb = false, delta = 3, pel = 2, overlap=2, sharp=1, idx = 1)
last.MVDegrain3(backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=300,idx=1)
LimitedSharpenFaster(ss_x=1.5, ss_y=1.5, dest_x=last.width, dest_y=last.height, Smode=3, strength=80, radius=2, Lmode=1, wide=false, overshoot=1, edgemode=0, special=false, exborder=0)

[J_Darnley]

I thought threads 4 meant 4 cores, isnt that right?
It has been mentioned previously in this thread.

And your script shows why CPU usage is low, x264 is being limited by the slow input. If you want to speed it up, use MVTools2 and avisynth MT. If you want more CPU usage, turn up the x264 settings. Also, are you really running avisynth 2.6?

[Arshad07]

It has been mentioned previously in this thread.

any chance u can explain please :o


And your script shows why CPU usage is low, x264 is being limited by the slow input. If you want to speed it up, use MVTools2 and avisynth MT. If you want more CPU usage, turn up the x264 settings. Also, are you really running avisynth 2.6?

ok, i'll figure out how to use avisynth MT.

I'm using 2.5.8 btw.

[Manao]

bad idea... amd dual core optimizer was made to prevent that and it worked pretty well.
i dont know what microsoft engineers smoked when coding the windows kernel... At least on Core 2s, a single threaded program which affinity was forced to a specific core (I tested both) is slower than a single threaded program which affinity isn't restricted, and which uses both cores. It shouldn't be the case, yet that's what I observed.
If not before, this way of thinking is wrong with Core i7 now. That's because it will automatically "overclock" one core and put the other cores to sleep when there is only one thread running.

So switching a single thread between different cores may prevent this optimization form being enabled and thus hurt the performance significantly...I don't think switching on a sub second basis will enable that on the fly over/under clocking.

[LoRd_MuldeR]

I don't think switching on a sub second basis will enable that on the fly over/under clocking.

Yeah, that's why I thought that switching cores for a single thread may be a bad idea on those new CPU's.

If it was not a bad idea before, then it is now. But maybe they already have a workaround implemented for this kind of CPU. I have no info...

I thought threads 4 meant 4 cores, isnt that right?
It has been mentioned previously in this thread.

any chance u can explain please :o

4 threads would only keep a Quadcore busy at 100%, if all threads always have work to do and if none of these threads ever has to synchronize (wait for results from another thread).

In reality threads running in parallel are never absolutely independent of each other. So each thread will become idle now and then.

For example: If you run n threads on a n core machine and only one of these threads has to synchronize, then the CPU usage will drop from 100% to 75% while that thread is waiting.

That's why creating more threads than cores does make sense indeed. If one thread becomes idle, then another (previously inactive) thread can become active immediately.

But you can't blindly generate a HUGE number of threads, because the overhead for managing all these threads will grow. You need to find a good trade-off, like "3/2 * cores" for x264.

So use "--threads auto" and let x264 decide the optimal number of threads for your system...