Thread count and quality

[YaBoyShredderson]

I use staxrip to encode and i usually use x264 high profile, crf 18, veryslow, film tune. I dont really mess with the other settings manually, except i set threads to 12. I have a 5900x and set it 12 because thats half my cpu, enabling me to still play games while encoding, or to run 2 encodes simultaneously, usually the latter.

From my understanding, x264 chooses 1.5x your logical thread count by default, i think this is to make sure your cpu is always fed with data, because some threads finish before others? So for me, thag would be 36 threads, which i was told can cause a noticeable degradation of quality. How much so and is it worth going even lower?

No matter how low i go in thread count, it would still take the same amount of time (in theory), because im always encoding many many files. Encoding the files back to back, or all at the same time isnt really relevant. While it would take much longer for any individual file to finish, the whole set would take the same amount of time.

Im not sure exactly about the amount of quality difference, but i would assume its not linear, so the difference between 1 thread and 6 isnt the same as 6 to 12, 12 to 18 etc, but what kind od bitrate savings can i expect? I use constant quality, so will the bitrate reduce at the same crf, or will the quality increase? So i can increase crf from 18 to 19 for example for same quality? Or will the file just be smaller staying with 18?

[Blue_MiSfit]

1 thread is optimal. You inherently lose some compression efficiency when using threading and it's ideal to not use it (from a quality standpoint).

Big cloud streaming providers often run single threaded parallel chunk encoding for exactly this reason. Some encoders have flat-out poor threading, and cloud providers often prefer to work with a large pool of small VMs, so there's no downside to single threading.

[YaBoyShredderson]

Originally Posted by Blue_MiSfit

1 thread is optimal. You inherently lose some compression efficiency when using threading and it's ideal to not use it (from a quality standpoint).

Big cloud streaming providers often run single threaded parallel chunk encoding for exactly this reason. Some encoders have flat-out poor threading, and cloud providers often prefer to work with a large pool of small VMs, so there's no downside to single threading.

So better to run 16 encodes on 1 thread each than a single encode on 16. How much better if u could ballpark it (bitrate savings/crf changes etc)?

[jpsdr]

Don't forget that you'll need a lot more of memory to run 16 encodes on 1 thread than 1 single on 16 threads.

[YaBoyShredderson]

Originally Posted by jpsdr

Don't forget that you'll need a lot more of memory to run 16 encodes on 1 thread than 1 single on 16 threads.

I have 32gb, i should be fine. It usually used about 700-800mb per encode i think.

[benwaggoner]

Originally Posted by Blue_MiSfit

1 thread is optimal. You inherently lose some compression efficiency when using threading and it's ideal to not use it (from a quality standpoint).

Big cloud streaming providers often run single threaded parallel chunk encoding for exactly this reason. Some encoders have flat-out poor threading, and cloud providers often prefer to work with a large pool of small VMs, so there's no downside to single threading.

Also, the overhead of the threads reduces throughput per core some. Multithreading is done to reduce time to encode. If throughput matters a lot more than latency, different threading strategies are optimal.

[imcold]

Originally Posted by Blue_MiSfit

1 thread is optimal. You inherently lose some compression efficiency when using threading and it's ideal to not use it (from a quality standpoint).

Big cloud streaming providers often run single threaded parallel chunk encoding for exactly this reason. Some encoders have flat-out poor threading, and cloud providers often prefer to work with a large pool of small VMs, so there's no downside to single threading.

It's inherently worse quality-wise only when you do slice-based threading, where you lose compression efficiency at slice boundaries, otherwise it depends? I don't remember the specifics for x264 that much, but iirc with default frame-based threading (1 thread per frame), only the vertical motion search for following frame is restricted. With high enough vertical video resolution, that gets irrelevant, because levels give restrictions on the vertical mv range anyway (unless you throw level compliance out of the window, ofc).

[benwaggoner]

Originally Posted by imcold

It's inherently worse quality-wise only when you do slice-based threading, where you lose compression efficiency at slice boundaries, otherwise it depends? I don't remember the specifics for x264 that much, but iirc with default frame-based threading (1 thread per frame), only the vertical motion search for following frame is restricted. With high enough vertical video resolution, that gets irrelevant, because levels give restrictions on the vertical mv range anyway (unless you throw level compliance out of the window, ofc).

Right, there are features that improve efficiency without impacting quality materially, or at all.

For example --b-adapt 1 is now effectively the same quality as --b-adapt 2, although IIRC the defaults haven't been updated accordingly. Compiler optimization like specific CPU targeting or profile driven optimization improve perf some without any quality impact.

x265 has a whole bunch of other features that improve parallelization without impacting quality, some of which could probably be backported to x264. For example, parallel mode estimation (--pmode). It can improve quality a bit AND increases parallelization (although the fps/thread gets a lot worse).