Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th March 2021, 12:01   #1  |  Link
YaBoyShredderson
Registered User
 
Join Date: Jul 2020
Posts: 72
Thread count and quality

I use staxrip to encode and i usually use x264 high profile, crf 18, veryslow, film tune. I dont really mess with the other settings manually, except i set threads to 12. I have a 5900x and set it 12 because thats half my cpu, enabling me to still play games while encoding, or to run 2 encodes simultaneously, usually the latter.

From my understanding, x264 chooses 1.5x your logical thread count by default, i think this is to make sure your cpu is always fed with data, because some threads finish before others? So for me, thag would be 36 threads, which i was told can cause a noticeable degradation of quality. How much so and is it worth going even lower?

No matter how low i go in thread count, it would still take the same amount of time (in theory), because im always encoding many many files. Encoding the files back to back, or all at the same time isnt really relevant. While it would take much longer for any individual file to finish, the whole set would take the same amount of time.

Im not sure exactly about the amount of quality difference, but i would assume its not linear, so the difference between 1 thread and 6 isnt the same as 6 to 12, 12 to 18 etc, but what kind od bitrate savings can i expect? I use constant quality, so will the bitrate reduce at the same crf, or will the quality increase? So i can increase crf from 18 to 19 for example for same quality? Or will the file just be smaller staying with 18?
YaBoyShredderson is offline   Reply With Quote
Old 4th March 2021, 19:44   #2  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,872
1 thread is optimal. You inherently lose some compression efficiency when using threading and it's ideal to not use it (from a quality standpoint).

Big cloud streaming providers often run single threaded parallel chunk encoding for exactly this reason. Some encoders have flat-out poor threading, and cloud providers often prefer to work with a large pool of small VMs, so there's no downside to single threading.
Blue_MiSfit is offline   Reply With Quote
Old 4th March 2021, 19:55   #3  |  Link
YaBoyShredderson
Registered User
 
Join Date: Jul 2020
Posts: 72
Quote:
Originally Posted by Blue_MiSfit View Post
1 thread is optimal. You inherently lose some compression efficiency when using threading and it's ideal to not use it (from a quality standpoint).

Big cloud streaming providers often run single threaded parallel chunk encoding for exactly this reason. Some encoders have flat-out poor threading, and cloud providers often prefer to work with a large pool of small VMs, so there's no downside to single threading.
So better to run 16 encodes on 1 thread each than a single encode on 16. How much better if u could ballpark it (bitrate savings/crf changes etc)?
YaBoyShredderson is offline   Reply With Quote
Old 4th March 2021, 20:30   #4  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,955
Don't forget that you'll need a lot more of memory to run 16 encodes on 1 thread than 1 single on 16 threads.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 4th March 2021, 20:42   #5  |  Link
YaBoyShredderson
Registered User
 
Join Date: Jul 2020
Posts: 72
Quote:
Originally Posted by jpsdr View Post
Don't forget that you'll need a lot more of memory to run 16 encodes on 1 thread than 1 single on 16 threads.
I have 32gb, i should be fine. It usually used about 700-800mb per encode i think.
YaBoyShredderson is offline   Reply With Quote
Old 4th March 2021, 20:45   #6  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,977
Quote:
Originally Posted by Blue_MiSfit View Post
1 thread is optimal. You inherently lose some compression efficiency when using threading and it's ideal to not use it (from a quality standpoint).

Big cloud streaming providers often run single threaded parallel chunk encoding for exactly this reason. Some encoders have flat-out poor threading, and cloud providers often prefer to work with a large pool of small VMs, so there's no downside to single threading.
Also, the overhead of the threads reduces throughput per core some. Multithreading is done to reduce time to encode. If throughput matters a lot more than latency, different threading strategies are optimal.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 5th March 2021, 09:31   #7  |  Link
imcold
pencil artist
 
imcold's Avatar
 
Join Date: Jan 2006
Location: Slovakia
Posts: 201
Quote:
Originally Posted by Blue_MiSfit View Post
1 thread is optimal. You inherently lose some compression efficiency when using threading and it's ideal to not use it (from a quality standpoint).

Big cloud streaming providers often run single threaded parallel chunk encoding for exactly this reason. Some encoders have flat-out poor threading, and cloud providers often prefer to work with a large pool of small VMs, so there's no downside to single threading.
It's inherently worse quality-wise only when you do slice-based threading, where you lose compression efficiency at slice boundaries, otherwise it depends? I don't remember the specifics for x264 that much, but iirc with default frame-based threading (1 thread per frame), only the vertical motion search for following frame is restricted. With high enough vertical video resolution, that gets irrelevant, because levels give restrictions on the vertical mv range anyway (unless you throw level compliance out of the window, ofc).
__________________
fevh264 - open-source baseline h.264 encoder
imcold is offline   Reply With Quote
Old 6th March 2021, 00:05   #8  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,977
Quote:
Originally Posted by imcold View Post
It's inherently worse quality-wise only when you do slice-based threading, where you lose compression efficiency at slice boundaries, otherwise it depends? I don't remember the specifics for x264 that much, but iirc with default frame-based threading (1 thread per frame), only the vertical motion search for following frame is restricted. With high enough vertical video resolution, that gets irrelevant, because levels give restrictions on the vertical mv range anyway (unless you throw level compliance out of the window, ofc).
Right, there are features that improve efficiency without impacting quality materially, or at all.

For example --b-adapt 1 is now effectively the same quality as --b-adapt 2, although IIRC the defaults haven't been updated accordingly. Compiler optimization like specific CPU targeting or profile driven optimization improve perf some without any quality impact.

x265 has a whole bunch of other features that improve parallelization without impacting quality, some of which could probably be backported to x264. For example, parallel mode estimation (--pmode). It can improve quality a bit AND increases parallelization (although the fps/thread gets a lot worse).
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Tags
efficiency, quality, staxrip, threads, x264

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:12.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.