2 CPUs Encode in x264?

[Hiritsuki]

I got EVGA SR-2 & 2 Xeon L5640 set, that is 12core 24threads.
I want to know how x264 using 24threads supportive? :thanks:

[LoRd_MuldeR]

x264 uses 3/2 * number of CPU cores threads by default (can be overwritten with the "--threads" parameter). The maximum number of threads is 128 currently.

And yes, x264 is actually able to fully utilize a huge number of CPU cores:
http://img15.imageshack.us/img15/7954/x26416cores.th.png (http://img15.imageshack.us/img15/7954/x26416cores.png)

[Hiritsuki]

I see thanks.

[ramicio]

Please let us know what kind of FPS numbers you hit if you do any 1080p HD stuff at a high bit rate. I am planning on 2x E5645's or 2x X5650's, and I want to hear about actual results instead of looking at bench numbers.

[Hiritsuki]

Oh, I look wait a while....

L5640*2@3.8Ghz

1.74fps

General
Complete name : J:\MM\02\BDROM\BDMV\STREAM\MM 03.mp4
Format : MPEG-4
Format profile : JVT
Codec ID : avc1
File size : 847 MiB
Duration : 23mn 40s
Overall bit rate : 5 003 Kbps
Encoded date : UTC 2011-09-16 00:06:47
Tagged date : UTC 2011-09-16 00:06:47

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5.1
Format settings, CABAC : Yes
Format settings, ReFrames : 5 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 23mn 40s
Bit rate : 5 000 Kbps
Maximum bit rate : 14.8 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.101
Stream size : 846 MiB (100%)
Writing library : x264 core 116 r2074 2641b9e
Encoding settings : cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x133 / me=tesa / subme=10 / psy=1 / psy_rd=0.80:0.05 / mixed_ref=1 / me_range=64 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-3 / threads=36 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=5 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=infinite / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=250 / rc=2pass / mbtree=1 / bitrate=5000 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00
Encoded date : UTC 2011-09-16 00:06:47
Tagged date : UTC 2011-09-16 05:30:59

[ramicio]

1.74 FPS?, and that's pretty low bit rate.

[Hiritsuki]

5000 is too low??

Anime can using small bit rate to encode...

[ramicio]

But 1.74 frames per second? Almost 14x as slow as real time. Do you have some crazy kind of filters in the avisynth script or something, that are using old, archaic programming. Can I guess that the slow speed is mainly due to this being 32 bit?

[LoRd_MuldeR]

1.74 FPS?

He is using some super-slow placebo settings, such as ME=TESA and ME-Range=64, so it's not that surprising ;)

Also we have absolouetly no information on how he is decoding his source and which pre-processing filters (if any) he is applying -> might be an input bottleneck.

@Hiritsuki: While you have not posted your command-line, you can probably replace most of it with a simple "--preset veryslow" or even "--preset slower".

and that's pretty low bit rate.

Well, depends on the content...

[ramicio]

I've never understood the point of forcing something to encode slower than it is capable of...treating something digital as if it was to reap some benefits that apply to analog things (e.g. capturing at 1/2 speed.)

I'd be curious on some numbers of live-action content, with a decent CRF value, and no crazy slow settings. Just a basic command like:

x264 --crf 17 --threads 36 --profile high --level 4.1 --output <name> <input>

On an i7-970 I usually hit between 18 FPS and real-time. I'd like to know if I'd be wasting my money on building a dual- or quad-socket server (it will also serve files.)

[Hiritsuki]

no command-line
only one filter is warpsharp

that is my script
LoadPlugin("C:\Program Files (x86)\MeGUI\tools\dgavcindex\DGAVCDecode.dll")
AVCSource("J:\MM\02\BDROM\BDMV\STREAM\00004.dga")
trim(25,34070)
warpsharp(depth=90,bump=90)

[LoRd_MuldeR]

no command-line

Well, there must be some command-line you are using to call the encoder ;)

If you are using a GUI front-end, it may not show the command-line to you. Still the GUI is passing some command-line to x264.

(And still your settings are placebo, so don't be surprised about slow encoding with these settings)

only one filter is warpsharp

that is my script
LoadPlugin("C:\Program Files (x86)\MeGUI\tools\dgavcindex\DGAVCDecode.dll")
AVCSource("J:\MM\02\BDROM\BDMV\STREAM\00004.dga")
trim(25,34070)
warpsharp(depth=90,bump=90)

You could benchmark the script the alone (e.g. with TimeCodec (http://haali.su/mkv/timeCodec.exe)), although I doubt it is the bottleneck here.

BTW: AVCSource has been abandoned. You may want to "upgrade" to DGDeceNV (given you have an NVidia card) or check out FFmpegSource2.

[Hiritsuki]

LoRd_MuldeR thanks, but DGDeceNV is not free, I have no creadit card to pay it.

I'd not set MeGUI's command-line

that is my x264 setting
program --level 5.1 --preset placebo --pass 2 --bitrate 5000 --stats ".stats" --keyint infinite --min-keyint 25 --bframes 5 --ref 5 --rc-lookahead 250 --merange 64 --subme 10 --psy-rd 0.80:0.05 --output "output" "input"

[ramicio]

If you are using 32-bit stuff, you could speed things up a lot with 64-bit avisynth (http://forum.doom9.org/showthread.php?t=152800) and x264. I don't know how these will be treated by MeGUI, though.

[Hiritsuki]

Now I using 32-bit avisynth and MeGUI run vfw4x264 pipe to 64-bit x264 to encode.

[ramicio]

I would try the 64-bit avisynth and get rid of that middle-man pipe. If MeGUI has a problem with it, then just re-install the 32-bit avisynth like normal. You probably will lose any kind of video preview in MeGUI, though.

[madhatter300871]

Hi

I have a P4 quad core 2.8GHz, 4G RAM (3GB used), Windows XP 32 bit. I encode full HD, preset medium, level 4.1 and get speeds of about real time, often just under.

If I use preset very fast I get speeds of about 35fps.

And I have to say, preset very fast produces very nice video indeed.

[ramicio]

I hope those speeds is with cartoon content with a P4. Also, Pentium 4s were not anything but single-core.

[Hiritsuki]

Download my BDrip's screenshot

http://www.megaupload.com/?d=HRCAI6X0

[ramicio]

What is that for? It's just a capture of a frame.

[hello_hello]

I've never understood the point of forcing something to encode slower than it is capable of...treating something digital as if it was to reap some benefits that apply to analog things (e.g. capturing at 1/2 speed.)

Don't the slower presets increase the quality when running 2 pass encoding, and allow for better compression (smaller file sizes) when running CRF encoding?

[ramicio]

Even settings like PLACEBO? I can understand applying those settings that something like --slow-firstpass does, but doesn't --slow-firstpass also slow it down further, just for the hell of it? Why would --slow-firstpass do anything for CRF?

[hello_hello]

You mentioned MeGUI earlier so I assume you use it.
If you open the x264 configuration dialogue, load the defaults, then select "show advance settings", you can change the preset on the Main tab, then switch to the other tabs to see which x264 options change accordingly (well some of them at least). I'm definitely no expert on x264's various options, but I think the ones which change between presets such as Medium, Slower and Placebo are things such as motion estimation algorithms, number of B frames, number of reference frames etc.... stuff which in CRF mode equates to better compression, smaller file sizes. I'm not sure if any of it actually effects the quality as such in CRF mode, but I'd assume it does when running two passes. At least that's my very basic understanding of it.....

I don't really know much about -slow--first pass, but I'd imagine it wouldn't be applicable to CRF encoding.
I've looked at the help files, (http://mewiki.project357.com/wiki/X264_Settings#slow-firstpass) and maybe I'm having a dumb day, but from what's written there I don't seem to be able to get my head around exactly what the setting does, only that it relates to --pass 1 and it's already included in the Placebo preset.

[ramicio]

I don't use it. I use command lines.

[sneaker_ger]

Don't the slower presets increase the quality when running 2 pass encoding, and allow for better compression (smaller file sizes) when running CRF encoding?

Of course. Slower preset = higher quality at the same bitrate.
And there's a reason why the x264 devs called the slowest preset "placebo (http://en.wikipedia.org/wiki/Placebo)".

[ramicio]

But it's not higher quality just because it encodes slower. This is why I think it is dumb.

[hello_hello]

I assume you didn't read any of my previous post then?

[ramicio]

Forcing something to just go slow does not increase its quality. I think the naming is stupid. This isn't CD burning or digitally capturing analog.

[hello_hello]

There's a reason why encoding is slower when using slower presets and why that equates to a higher quality when running 2 pass encoding or smaller file sizes when running CRF encoding. I explained some basics of it a couple of posts ago. The slower presets aren't there solely to annoy you.

Taking your logic a step further, if slower doesn't improve the quality, faster encoding won't decrease it. Why don't you encode everything using the Ultra Fast preset?

[ramicio]

1. I don't define any presets.
2. You explained nothing.
3. There is no reason why literally slowing down encoding will improve quality. I could literally sit there and hit the pause key to pause each frame, and it would sure slow things down, but literally do nothing. If speed itself also mattered then getting a faster CPU would decrease quality, in your logic. There are settings that will improve quality and therefore slow down the process, the goal is not to purposely slow things down. The semantics of slowing things down just sounds stupid, and the preset names should range from speed to quality, not just fast to slow. I'm arguing the naming, nothing else. You seem to always like to put words in my mouth. Go troll somewhere else.

[sneaker_ger]

He's correct, strictly speaking, slower doesn't always equal higher quality. That's also a reason as for why we got the presets. But for x264's preset system slower always means more bang for the buck, i.e. more quality for a given bitrate. A special case, if so to speak. But we all know that, right? No need for more nitpicking.

[ramicio]

And the placebo setting is especially wasteful for cartoons. You should just use CRF and be amazed at how much smaller they will probably be.

[hello_hello]

ramicio, I really can't tell whether you've just got your blinkers on too tight again, or whether you're vainly trying to dig yourself out of a hole. The comment you made which started this discussion was:
"I've never understood the point of forcing something to encode slower than it is capable of...treating something digital as if it was to reap some benefits that apply to analog things"
Now you're trying to claim you were simply referring to the way the settings are named??? What rubbish.
"....but doesn't --slow-firstpass also slow it down further, just for the hell of it?"
You put the words in your mouth, not me.

If speed itself also mattered then getting a faster CPU would decrease quality, in your logic.

Obviously your attitude is based on a complete lack of understanding of anything I've said if you think my logic would allow that conclusion. Speed on it's own doesn't matter. That's not what I've been saying at all.
Once again, slower settings use different algorithms, different compression settings etc which require more CPU time. Hence slower encoding. And you're correct, hitting the pause key to pause each frame would sure slow things down and literally do nothing else, hence anyone with an ounce of common sense would know a slower preset must achieve something aside from simply slowing down the encoding process. And that's what you were originally claiming it'd do.
"I've never understood the point of forcing something to encode slower than it is capable of..."
Which x264 setting only does that???

[hello_hello]

And the placebo setting is especially wasteful for cartoons. You should just use CRF and be amazed at how much smaller they will probably be.

That doesn't even make sense. You can use the placebo setting for both 2 pass and CRF encoding, so you can't claim the placebo setting is wasteful unless you're comparing a placebo CRF encode with a "normal' CRF encode and I think you'll find it's the other way around. A Placebo CRF encode should give you a smaller file size than a "normal" CRF encode, all else being equal.
Maybe rather than just use a command line you should be using a GUI such as MeGUI to help you understand what the various x264 options are for. Then when you do go back to using command line you might have an idea which options to use and what they actually do.

By the way, are you using CRF encoding yourself now? After you recently argued the validity of CRF encoding and accused me of trolling again (as you do when your argument falls over completely) while I tried to explain CRF encoding to you in another thread? After all your pontificating against CRF encoding are you actually using it now and reaping the benefits?

CRF is a complete waste to me, because between two similar-length live action movies, one will be 10 GB and the other will be 20 GB.

Your trouble is, you seem to make up your mind as to how things work long before you have any real understanding of how they work.

[hello_hello]

But for x264's preset system slower always means more bang for the buck, i.e. more quality for a given bitrate.

That's all I've been trying to explain. Nothing else.

[ramicio]

No, you seem to make up your douche-baggish mind up about what someone else means long before you ever try to understand any of it. You should seriously just go kill yourself. And yes, I tried CRF, and it is stupid. It makes sense if you are encoding from a lossless source, but if you are coming from a source that's lossy, you can easily wind up with a file much much bigger. It bases quality off what it sees, not the final result to the human eye. Keep arguing your pseudo-scientific points about perception and blind tests, it only makes you look like an idiot HydrogenAudio fanboy. Seriously, kill yourself or leave me alone with your "blinkered" talk, because the term makes no sense. But anyway, you're on my ignore list now, because I am sick and tired of YOUR viewpoint.

[hello_hello]

Funny stuff!! You want to know what the term "blinkered" means, well.... your last post is pretty much the definition of it.
But just in case you still don't get it: http://www.flickr.com/photos/rockandracehorses/179445447/ ;)

And yes, I tried CRF, and it is stupid. It makes sense if you are encoding from a lossless source, but if you are coming from a source that's lossy, you can easily wind up with a file much much bigger. It bases quality off what it sees, not the final result to the human eye.

So that's what a complete lack of understanding looks like. If the file sizes are too big, use a higher CRF value.
What does a 2 pass encode use to base quality off while the encoder's deciding how to distribute the bits? Aside from what the encoder sees, does it stop at each frame to ask you how it looks? :rolleyes:
So when you use stupid CRF encoding for your cartoons, is your source a lossless format?

[Groucho2004]

You should seriously just go kill yourself.
What is wrong with you?

[Hiritsuki]

Anyway ... today I'll turn BIOS option about [ACPI "T" state] Off to test again.
It's maybe faster.

[Blue_MiSfit]

@ramico: there were actually many dual core Pentium 4 CPUs, and most had Hyperthreading, thus presenting 4 logical CPUs. Not a real quad though ;)

Also, x264 does scale to a lot of threads, though in my opinion you're usually best of using a dual 6 core / 12 thread system (12 cores 24 threads total ) to run many encodes in parallel. I have many of these servers and typically run 3 to 6 encodes in parallel, mainly because my sources (1080p ProRes or 80mbps MPEG-2) decode rather slowly.

Derek