Threading and latency in x264

[akupenguin]

It's possible to have hybrid xvid-threading and frame-threading, with the caveat that both kinds of threads use up the same spatial buffer, so you're only trading off scaling efficiency vs latency, not increasing the max number of threads. And the caveat that xvid-threading is incompatible with accurate RDO (actually I don't know how inaccurate it would be, only that it can't keep track of the bitstream state).
Slice-threading can't mix with frame-threading.
GOP-threading can mix with everything, but that's not what you want for latency
Pipeline-threading can also mix with everything, but it reduces compression efficiency and scaling efficiency.

[Manao]

I wonder what pipeline threading is.

[akupenguin]

Originally Posted by Manao

I wonder what pipeline threading is.

doing different things in different threads; me, mode decision, entropy coding, hpel/deblock, ...

[foxyshadis]

Originally Posted by akupenguin

Slice-threading can't mix with frame-threading.

In that case, what about splitting slices into separate instances of the encoder, then combining and packaging the output as each NAL is output in the control program? Not necessarily x264, but it's an avenue to explore if anyone wants unique ways to lower latency.

Videoconferencing won't really live or die from the efficiency loss of using slices, anyway, it's largely talking heads on static backgrounds.

[Shinigami-Sama]

Originally Posted by akupenguin

doing different things in different threads; me, mode decision, entropy coding, hpel/deblock, ...

I thought thats what was already done?

seems like the best way to do to me
but I'm not a programmer...

[Manao]

Currently, each thread encode a whole frame (ie, does me, mode decision, entropy, hpel, deblock), so each thread have roughly the same amount of work. If you were to give each thread a specific task instead, the thread doing hpel (for example) would do its task faster than the thread doing the ME, so it would unbalanced CPU load, thus generate threading inefficiency.

[akupenguin]

Originally Posted by foxyshadis

what about splitting slices into separate instances of the encoder

Then you can't have mvs span the middle of the frame (nor even near the boundary, i.e. edge mbs can't have small subpel mvs). This is exactly why slice-threading is incompatible with frame-threading.

[Quark.Fusion]

Originally Posted by akupenguin

Pipeline-threading can also mix with everything, but it reduces compression efficiency and scaling efficiency.

Why it reduces compression efficiency?

[Manao]

If you thread decision and entropy, RD costs during the decision will be slightly off, because you won't have the exact cabac context.

[Quark.Fusion]

Maybe then thread everything that not hurt? Thread balancing is already broken with source decoding, b-adapt, 1.5 thread per core and etc. (Don't forget other processes on system that also distrub x264 thread balancing).
If you want perfect threading you must make multitask threads (or many single-task threads for each core) and thread server (with higher priority), that will split source task to smaller parts and distribute them in a smart way to keep all cores busy.

[Sagekilla]

So basically you're saying creating a client/server model where the server assigns the tasks and the clients do the grunt work. If they switched to that threading model, you could probably even span it across multiple systems easier.

Edit: If you could get it to work, you could get close to 35 fps (1080p frames) running at the max of gigabit network, or 85 fps if you're doing 720p frames.

[Manao]

Source decoding isn't part of x264, and will limit any threading scheme, so it can be ignored. b-adapt can be threaded without losing anything. 1.5 thread per core is needed because frames don't take systematically the same time to encode (and because sometimes, motion vectors restriction make a thread wait for another, so in such cases it's nice to have a third thread that can run during that time), it's not quite the same thing as load balancing

Also, don't forget that the more thread, the more overhead from context switching. So a thread server, though it looks nice and paper, can cost a lot.

In any case, without consideration for latency, slice-type decision is currently the only limiting factor, and it is fairly easy to thread it.

[Quark.Fusion]

All that isn't part of x264 can't be ignored as x264 isn't the only thing that running on a computer. It's pointless to split task in equal parts per thread as other processes will break all balance. Threads must be dynamically loaded with small tasks from tasks buffer.
Thread server don't need many context switches. But more that 1 thread per core produces concurrency. There was thread pool patch somewhere — did it provide better efficiency?

[Ranguvar]

It did for large numbers of threads (> 4) and worsened efficiency for less threads. IIRC.

http://komisar.gin.by/x.patch/BugMas...pool.r965.diff