x264/Me-GUI not utilizing all CPU speed / cores ?

[jonathonsunshine]

Howdy yall, I am running a intel q6600 quad core processor and I am using it to encode 1920x1080 mpeg2 to x264 at 640x352 at about 1000kbit/s

On the 1st pass it encodes using only 33% of my total cpu speed at about 22fps.

On the second pass thou it uses 90% at about 20fps.

I am using the "x264 - HQ Slowest" profile in Me-GUI (v0.2.6.1046) which is using x264.exe v859-3.

I have found that if I disable all cropping and resizing in the AVISynth script (1920x1080) it goes at %60 cpu utilization (with a warning saying that since resolution is not divisible by 16 i will take a speed hit) so if i crop 8 pixels off the bottom (1920x1072) it will use 90% again.

I have also noticed that when encoding standard definition mpeg2 (720x567) to x264 (again, at 640x352) I get another performance hit on the 1st pass, althou this time not as much, is uses +/- 60% of the cpu.


So yeah ? How can I can I get faster encoding on the 1st pass ? IE 60fps'ish ?


Thanking you in advance.



==============================
additional info that might be pertinent.

the command line parameters for x264,
===================
--pass 2 --bitrate 1000 --stats ".stats" --ref 10 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "output" "input"
===================


The AVS script (avisynth 2.5.7) *edit, This is the script generated by Me-GUI, the only tweak being that I added the trim function just so I wouldn't have to wait 40minutes to see how fast the 2nd pass would go
=============
DGDecode_mpeg2source("T:\TV - Ready to Encode\10HD Veronica Mars - 20080325 -.d2v",info=3)
ColorMatrix(hints=true)
#deinterlace
crop( 4, 2, -4, -2)

LanczosResize(640,352) # Lanczos (Sharp)
#denoise
trim(2000,3000)
=============

DGIndex version 1.50b10-1

[Dark Shikari]

(with a warning saying that since resolution is not divisible by 16 i will take a speed hit) so if i crop 8 pixels off the bottom (1920x1072) it will use 90% again.That warning says nothing about a speed hit; it talks about a compression hit (a very very very minimal one at that).

I suspect the primary reason it isn't using all your CPU is the processing in the avisynth script bottlenecking the encoder.

There actually is a known issue involving very fast first passes and B-adapt on machines with 4 or more cores which limits the first pass to about 3 cores. A fix is in the works for this.

[jonathonsunshine]

I suspect the primary reason it isn't using all your CPU is the processing in the avisynth script bottlenecking the encoder.


Yeah, thats sounds reasonable, I threw the avisynth script through Media Player Classic and its using almost spot on 25% of cpu time but doesn't quite manage to render it "on the fly". Only manages about 23fps (from 25).


There actually is a known issue involving very fast first passes and B-adapt on machines with 4 or more cores which limits the first pass to about 3 cores. A fix is in the works for this.

When I disable resizing I get close to 100% cpu utilization thou. Would this confirm that the problem is with the Avisynth bottleneck?

And if so, is there a fix in the works for that?

(I realise that if there is a fix in the works for this, its not your responsibility, it would be upto the avisynth folks (most likely involving multithreading the resizing routines ?).

thankyou for your time/consideration


PS, since i see that we joined this forum at the same time (sept 2005) can I assume that you will be making this issue your top priority ? ;)

[Dark Shikari]

When I disable resizing I get close to 100% cpu utilization thou. Would this confirm that the problem is with the Avisynth bottleneck?Yes
And if so, is there a fix in the works for that?

(I realise that if there is a fix in the works for this, its not your responsibility, it would be upto the avisynth folks (most likely involving multithreading the resizing routines ?).Have you tried the existing multithreaded Avisynth?

[saint-francis]

When I disable resizing I get close to 100% cpu utilization thou. Would this confirm that the problem is with the Avisynth bottleneck?

And if so, is there a fix in the works for that?


The problem, as you have noticed, is with avisynth and not x264 so DS probably won't be involved in any development which will change your issue. What you need to do is get an avisynth plugin called MT. It has a .dll plugin which you put in you avisynth plugin folder and a new copy of avisynth which you replace the original with. This will solve your problem. There's a lot of talk about it in the avisynth forum. You can check it out there. I have 4 cores and I happily max them out all the time because of this nice tool. :)

EDIT: Sorry for posting this. I didn't see the post by Dark Shikari for some stupid reason.

[jonathonsunshine]

Howdy yall, I have been trying to work out this MT filter, but I'm ot having a lot of luck, my best effort has yielded me just a 15% increase in CPU utilization (50% from 35%)

I must confess that the majority of my scripts have simply been manipulations of the ones created by AutoGK.

Basically, I am hoping someone can just post what they would do this script to optimize it for a 4 core processor.

===============================


loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mt.dll")

loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")

mpeg2source("VeronicaMars.d2v")

lanczosresize(640,352)


===============================


Thanking you in advance.

[saint-francis]

Basically, I am hoping someone can just post what they would do this script to optimize it for a 4 core processor.

===============================
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mt.dll")

loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")

mpeg2source("VeronicaMars.d2v")

lanczosresize(640,352)
===============================


You don't need to load the plugins because they are in the avisynth plugins folder and are therefore automatically loaded.

You aren't calling MT in your script. You probably want it to look like this:

setMTmode (2,0)
mpeg2source("VeronicaMars.d2v")
lanczosresize(640,352)

[Kurth]

You can force the resize to work with more cores with this command.

MT("Spline36Resize(640,last.height())",splitvertical=false)MT("Spline36Resize(last.width(),352)",splitvertical=true)

Well at last this works better on my dual core processor.