Having x264 max out all i7 980x cores

[asarian]

I just bought a new i7 980x (Windows 7, 64-bit), especially to go the extra mile with x264.

I know a CPU load of ca. 50% is good (with HT enabled); but, at around 30-40%, x264 is not even using all logical cores. Consistently it appears to not be using 3 cores or so. I use an AVS script like this:

FFVideoSource("M:\jobs\gits.mkv")
Crop(32, 38, -32, -38)
LanczosResize(1920, 1068)
MCTemporalDenoise(settings="medium")
GradFun2DBmod(thr=1.8,thrC=1.0,mode=0,str=2.4,strC=0.0,temp=50,adapt=64,mask=false,show=false)
SupTitle("M:\jobs\gits_exp.sup", forcedOnly=false, swapCbCr=false, relocate=true, relocOffset="0,0,0,0")

Why is x264 not maxing out all my cores? I've heard people say in the past: "Just start another x264 process!" But I don't want 2 jobs running: just 1 job very fast. :)

Thanks.

[Atak_Snajpera]

FFVideoSource("M:\jobs\gits.mkv") <- possible bottleneck (decoder uses one thread) Switch to dds() and make sure that ffmpeg-mt is selected for avc
MCTemporalDenoise(settings="medium") <- 99 % bottleneck

[asarian]

FFVideoSource("M:\jobs\gits.mkv") <- possible bottleneck (decoder uses one thread) Switch to dds() and make sure that ffmpeg-mt is selected for avc
MCTemporalDenoise(settings="medium") <- 99 % bottleneck
Yeah, you're probably right; especially since I don't use a multi-threaded AviSynth install yet.

I don't think you can actually run scripts like MCTemporalDenoise multi-threaded, can you? I should give it a try, though.

[Atak_Snajpera]

1) Install AviSynth-MT (http://forum.doom9.org/showthread.php?p=1412274#post1412274)

2) Change ffdshow decoder settings

http://img695.imageshack.us/img695/2677/83921465.png

http://img697.imageshack.us/img697/4402/17189462.png

3) Modify script:

SetMTMode(5,6)
LoadPlugin("C:\Program Files (x86)\Haali\MatroskaSplitter\avss.dll")
DSS2("M:\jobs\gits.mkv")
SetMTMode(2)
Crop(32, 38, -32, -38)
LanczosResize(1920, 1068)
MCTemporalDenoise(settings="medium")
GradFun2DBmod(thr=1.8,thrC=1.0,mode=0,str=2.4,strC=0.0,temp=50,adapt=64,mask=false,show=false)
SupTitle("M:\jobs\gits_exp.sup", forcedOnly=false, swapCbCr=false, relocate=true, relocOffset="0,0,0,0")

[Groucho2004]

If you have a (not too old) Nvidia card, do yourself a favour and get DG NV tools. That would sort out the decoding bottleneck.

The next step would be Set's Avisynth MT which seems to work pretty well at this stage.

[asarian]

Thanks, Atak! That was most useful information! :)

[kieranrk]

1) Install AviSynth-MT (http://forum.doom9.org/showthread.php?p=1412274#post1412274)

2) Change ffdshow decoder settings
[/CODE]

What's wrong with using a mt version of ffms2?

[Atak_Snajpera]

What's wrong with using a mt version of ffms2?
it likes to crash.

[asarian]

it likes to crash.
That's too bad, because FFVideoSource was nicely frame-accurate. :(

[Atak_Snajpera]

DSS2() is also frame accurate.

[asarian]

Turns out I was mistaken about 50% CPU load (in Task Manager) on the i7 980x with HT on meaning full CPU load, as it really can go to 100%. So I did what I said I didn't want to do: I started a second x264 process (a job with also MCTemporalDenoise as bottleneck); and now all my cores are 100% saturated, all the time (without actually taking away from the first process).

Guess I have to get used to make efficient use of 12 logical cores. :) But with 2 jobs running, each around 2.5 times as fast as my previous Q6700 @ 3.2Ghz, we're looking at 5x faster overall job time here. :) Long live the i7 980x!

[asarian]

DSS2() is also frame accurate.
My bad. I read here (http://docs.aegisub.org/manual/Video): "Warning: DSS is known to have problems with frame-accurate seeking. Do not use it if you can avoid it." But maybe that's not referring to DSS2.

[Guest]

DSS2() is frame accurate only if the timestamps are continuous. Any discontinuities will break it. Also, it is not frame accurate for elementary streams, as they carry no timestamps.

[Groucho2004]

@asarian

Just curious - You paid around 1000$ for the 980, one might assume that you'd have 15 bucks left for DGDecNV.

Is there anything you dislike about it or it won't do for your needs?

It seems to me the logical choice for what you are looking for.

[Atak_Snajpera]

Maybe he has brand new Radeon 5850 and he doesn't like idea of buying extra gfx just for this task. Also the biggest bottleneck in his case is MCTemporalDenoise(). It uses many filters like MVTools and so on. Hardware decoding won't give you huge speed up.

[Groucho2004]

Well, I know from the DG tools thread that he has a GTX260. He's also looking for frame accurate access which is a given with DGDecNV.

[asarian]

Well, I know from the DG tools thread that he has a GTX260. He's also looking for frame accurate access which is a given with DGDecNV.
Actually, for this new system (which I built up seperately) I bought an extra GTX285. Not a very spectacular card, but it should suffice for DGDecNV. Reading the DGDecNV thread, seems there's still a few kinks to work out (all of which seem on CUDA's end, really, not Neuron2's) like memory not being returned to the vid card and such. I'll probably get DGDecNV soon, though anyway; if, for nothing else, just to see what it can do for me. :) And for things other than MCTemporalDenoise, for which I still want x264 to saturate all my cores, extra decoding speed might help.

[Guest]

Reading the DGDecNV thread, seems there's still a few kinks to work out (all of which seem on CUDA's end, really, not Neuron2's) like memory not being returned to the vid card and such. Wrong. The memory leak was fixed several versions ago. And what does "and such" mean. DGDecNV has no known bugs that would prevent it from fulfilling its mission.

[stax76]

I don't understand why there is always a frame accuracy debate, I was thinking it's only relevant when trim is used?

[Atak_Snajpera]

When you use SetMTMode you must also have frame-accurate filter\decoder ...

[asarian]

I don't understand why there is always a frame accuracy debate, I was thinking it's only relevant when trim is used?
See, for instance, Dedee's post here (http://forum.doom9.org/showthread.php?p=1294664#post1294664).

[asarian]

1) Install AviSynth-MT (http://forum.doom9.org/showthread.php?p=1412274#post1412274)

2) Change ffdshow decoder settings

http://img695.imageshack.us/img695/2677/83921465.png

http://img697.imageshack.us/img697/4402/17189462.png


Hmm, I just realize, this is not going to work for VC1 sources, is it? :( I can only choose 'libavcodec' and 'wmv9' in ffdshow. That sucks.

[TheRyuu]

it likes to crash.

Please provide a sample?
I've had few issues using ffmpeg-mt with ffms2 for a long time now.

Anything which gives me trouble is generally not a ffmpeg-mt specific problem.

DSS2() is also frame accurate.

So now we're suggesting dss2 over a probably ok ffms2. I don't follow your logic.
I can understand if it's a BD and you have to use dss2 because ffms2 will fail miserably at it (probably) but that is in now way a ffmpeg-mt specific issue.

[Atak_Snajpera]

Please provide a sample?
Instant crash in MPC-HC with this file.
http://www.mediafire.com/file/xo1uaa45hkg3r6s/video.mkv

I don't follow your logic.
You don't have to. DSS2 + ffdshow always worked better for me. ffms2 incorrectly reports number of frames in interlaced sources. We get number of fields instead of real frames. Also it is not that stable yet (example above)