x264 realtime 1080p 60fps encoding

[Cinder]

Hi,

I'll try to explain my "issue" with as much detail as I can, so it's probably going to be a long post. For those not interested in reading the entire thing, the tl;dr version is - "how to improve quality/sharpness and/or reduce pixelation on realtime 1080p 60fps x264 encode"

I'm using a software called OBS to stream my PC gameplay over the internet (twitch.tv). I have a two PC setup - a game PC and a stream (encoding) PC. The game PC feeds the gameplay stream to the stream PC via capture card in 1440p 165Hz resolution. The capture card then feeds a downscaled 1080p 60fps stream to OBS, which encodes it using x264 and sends it to Twitch's ingest servers.

The encoding PC specs are:
2xE5-2697v4 (36 cores, HT disabled)
32GB RAM
GTX 980 (kind of irrelevant)

There isn't much documentation on what x264 settings OBS is running, but you can specify things like max bitrate (vbv-maxrate), bufffer (vbv-bufsize), CBR, CFR, keyint, x264 presets, profile and specify custom x264 settings as well.

I mostly stream a game called Battlefield 1, which is a high motion FPS game. Most of the time the stream PC can handle the slow preset without dropping frames, but on some occasions (high movement scenes, or a camera zoom out effect that occurs when dying/redploying) it will drop frames, so I'm sticking to the medium preset with some options from higher presets.

I've been doing some reading in the past week or so (down to page 50 on this subforum atm) trying to learn as much as I can about x264. I admit I'm far from understanding the details of x264, but I have managed to improve quality a bit with the knowledge gained here. My main problem at the moment is pixelation on high movement scenes and lack of detail on objects that are further away/back. Also trying to increase sharpness a bit, but reducing the pixelation might solve that. The x264 settings I'm currently using, aside from the medium preset and high profile are:

mixed_refs 8x8dct no-fast-pskip b-adapt=2 ref=5 deblock=-1:-1 trellis=2 psy=1 psy_rd=1.00:0.40 direct=auto no-dct-decimate aq-mode=1 aq-strenght=1.2

Things I've noticed to have most impact in terms of quality so far are deblock and trellis. Increasing ref and subme above their current values (5 and 7 respectively) also provides better quality, but at the cost of slowing the encode and occasionally resulting in dropped frames. Current CPU usage with these settings is ~50%. Going with more CPU intensive settings will probably result in the encoder not being able to maintain 60 fps and dropping frames on the stream.

This is what mediainfo reports on local recordings made with these settings:

Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5
Format settings, CABAC : Yes
Format settings, ReFrames : 5 frames
Codec ID : 7
Duration : 44 s 183 ms
Bit rate : 6 100 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.049
Stream size : 32.1 MiB (98%)
Writing library : x264 core 146 r2538 121396c
Encoding settings : cabac=1 / ref=5 / deblock=1:-1:-1 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.40 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-4 / threads=54 / lookahead_threads=8 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=120 / keyint_min=12 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=cbr / mbtree=1 / bitrate=6100 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=6100 / vbv_bufsize=6100 / nal_hrd=none / filler=1 / ip_ratio=1.40 / aq=1:1.00
Color range : Limited
Color primaries : BT.709
Transfer characteristics : sYCC
Matrix coefficients : BT.709

This is a sample recording with the above settings:

https://www.twitch.tv/icinder2/v/105874923

Any suggestions on how to improve quality further are more than welcome :)

P.S. I can't increase the bitrate, as 6100Kbps is already way past what Twitch is advertising as a max bitrate for their ingest servers.

[kuchikirukia]

First off, if you're having performance issues at higher settings, try playing with your thread count with and without HT. I mean, you turned HT off but seem to have left x264 at its default 1.5x logical core count. Queuing those extra threads is exactly what HT is for.
x264 by defualt would have set itself to 108 threads with HT enabled which may have been too much, but I really doubt HT off @ 1.5x is optimal. I'd bench it with HT off @ 36, 45, 54, and HT on at 54, 72, and 90.

Looking through... chroma_qp_offset=-4? Default should be -2, and that's not a small amount of bitrate.

[mariush]

I wouldn't use 54 threads to encode 1080p content. From a certain number of threads, the quality may be ever so slightly affected negatively. 32 should probably be enough.
Are you capturing with color range : limited or are you just setting that flag by accident? The output of your video card should be full range YCbCr or Full RGB
I wouldn't use 6100 kbps as bitrate, if you get a bunch of viewers Twitch would eventually complain. You should probably set it to 3500kbps or whatever the maximum Twitch allows officially, and probably would play with vbv_maxrate and bufsize to allow bursts of higher bitrate when needed.
Could probably raise the keyint to a higher value, but if you have lots of motion in games, you're probably going to get keyframes often anyway.
Motion estimation from hex to something higher (uhm , uneven multi hex) may increase the quality a bit especially with lower bitrates , for little decrease in performance (between 5 and 15% slower compared to hex)
uhm also allows me_range higher than 16, which can be beneficial for hd content and for high motion stuff, but naturally takes a bit longer. It's still multithreaded, so since you have so many threads you may as well put them to work. Try raising it to 20-24 and see if that makes a difference quality wise

Also my advice would be to try your changes with some prerecorded video, capture something and save it to disk compressed losslessly (using a codec like MagicYUV or Huffyuv or Lagarith) and then do your tests by compressing that lossless video to AVC using x264.

ps. what kind of capture card actually captures 165 fps at 1440p ? You sure it's a capture card and you're not really playing the game on that computer and capture it using OBS?

[Cinder]

Hi guys,

Thank you for the replies.


First off, if you're having performance issues at higher settings...

kuchikirukia, x264 isn't NUMA aware as far as I know, so it doesn't play well with HT enabled - it would only load one of the CPUs. With HT disabled both CPUs are equally utilized. Also Hyper threads are not really that great for encoding. I have verified multiple times that I get better performance with HT disabled. (for x264 at least).

I have tried setting threads to 36 manually, but that seemed to lead to more dropped frames using the current settings - as in the encoder wasn't able to keep up. I can try lowering some of the settings and lower the threads as well, but I fear that may lower quality as well.

I wouldn't use 54 threads to encode 1080p content...

mariush, 32 threads would mean I'd leave 4 cores unused, seems ineffective? As I mentioned in my reply to kuchikirukia, lowering threads seems to hurt performance significantly and would require reducing quality settings, so the encoder can keep up with less threads. Either way I can give it a try.

I can encode in full color range, but that doesn't seem to affect quality in any way. I've experimented with that.

Regarding the bitrate, I barely have any viewers right now and I have seen people with 200,400+ viewers streaming at that bitrate with no issues. I don't think you can achieve decent quality at 1080p 60fps at 3500Kbps bitrate. I'll take my chances with using 6100Kbps for now :)

Regarding keyframes, I do indeed have quite a lot of motion in game (as it can be seen in the video).

I've tried using umh instead of hex, but it doesn't seem to provide any noticeable improvement. Using umh with the current settings also results in a 10-15% CPU usage increase, going up to 70% on both cores, which could/would lead to dropped frames eventually. I will probably need to tune down some of the other settings, so I can free up some resources. I guess I could try lowering ref frames to 4 and upping me_range to 20-24 and see if that reduces the pixelation.

I'm using a Datapath VisionSC-DP2 capture card. I'm fairly sure I'm not playing and encoding on the same PC :)

[kuchikirukia]

Hi guys,

Thank you for the replies.

kuchikirukia, x264 isn't NUMA aware as far as I know, so it doesn't play well with HT enabled - it would only load one of the CPUs. With HT disabled both CPUs are equally utilized. Also Hyper threads are not really that great for encoding. I have verified multiple times that I get better performance with HT disabled. (for x264 at least).

Googling around, that seems to be a problem with OBS. Encoding is actually one of the best uses of hyperthreading. It's around a 20% gain. My i7 3770 running at 3.2GHz (turbo disabled and slight underclock because stock cooling) is just as fast as my i5 4670k at 3.9.

I have tried setting threads to 36 manually, but that seemed to lead to more dropped frames using the current settings - as in the encoder wasn't able to keep up. I can try lowering some of the settings and lower the threads as well, but I fear that may lower quality as well.

It's just about speed so find what's fastest. Run the same encode with different thread counts, check the log file for the encoding times, and figure out where you run out of improvement. My i5 is fastest at 6 threads (matching x264's 1.5x rule of thumb) while my HT enabled i7 is 10. (so 1.25 there)
I have my doubts that the 1.5x thing scales to 36 cores, and throwing too many threads at it will slow it down a tad.

But your quality issues are likely mostly due to running CBR. Unless Twitch demands CBR, try upping your vbv_maxrate and bufsize and finding a CRF that gives you the average bitrate you're looking for. As long as you're not pumping 100,000kbps peaks, it's probably average that they care about.
You can just run a 2-pass at 6100 and find the final rate factor in the log to find your CRF. (pretty sure it's in the first pass log, so you only have to run that)

[Cinder]

You're comparing HT on a single processor vs multi-processor. It's not the same. It's x264 that's not NUMA aware, OBS just invokes x264.

I tried setting different values for threads, both lower (as low as 32) and higher (up to 72) and there didn't seem to be a negative effect in terms of quality when increasing the thread count. Reducing the threads did however result in encoder lag in some occasions.

Using me=umh and increasing me_range to 20-24 did slightly improve quality, but at a 15-20% CPU increase, which again results in encoder lag in some situations, especially on higher me_range values.

It's advised to run CBR when live streaming, especially for high motion games, as high bitrate variation usually causes lag (buffering) for viewers.

[sneaker_ger]

Try experimenting with higher vbv buffer.

You said pixelation is an issue. Does OBS allow downscaling to e.g. 720p60? Lower resolutions might reduce the problem. (I assume you are set on keeping 60 fps)

[Cinder]

Increasing the buffer did seem to help a bit with pixelation, thank you. Increasing it too much could lead to high bitrate variation though, which is not nice.

OBS does allow downscaling, but unfortunately with downscaling you always lose quality. I'm set on keeping the resolution as well :)

Does increasing me_range have any effect if motion estimation is hex? Doesn't seem to have, based on a few tests I did. Also, after experimenting with settings a bit more, it doesn't seem like using me=umh provides that much of a gain in visual quality, but it does significantly slow encoding.

[sneaker_ger]

Increasing the buffer did seem to help a bit with pixelation, thank you. Increasing it too much could lead to high bitrate variation though, which is not nice.
Since bitrate and vbv maxrate are equal the filling rate of the buffer should be constant anyways. Maybe also up the keyint. (but I guess you reduced it on purpose)

OBS does allow downscaling, but unfortunately with downscaling you always lose quality.
You alway lose quality, that's why we are having this discussion. Maybe reducing resolution can help. It could also allow you to up other settings further. (on the other hand multi-threading gets more difficult with less lines to split for the cores - maybe it's a bad idea after all?)

Does increasing me_range have any effect if motion estimation is hex?
me range for hex and dia can only be between 4 and 16.

it doesn't seem like using me=umh provides that much of a gain in visual quality, but it does significantly slow encoding.
To get a rough idea where the settings fall in terms of speed lost/quality gained it can be useful to look under which preset they fall. Yours seem to fall roughly into medium, slow or slower presets. subme 7 is medium, trellis 2 is slow, umh and b-adapt 2 are slower preset. So you already have pretty good settings. Maybe try subme 8 and b-adapt 1 to consolidate everything to preset slow. Only thing left to tune would be things like deblock, psy, aq-strength, lookahead and number of bframes and ref frames. (But don't ask me what's best for Battlefield...)

[kuchikirukia]

You alway lose quality, that's why we are having this discussion. Maybe reducing resolution can help.

This. We're talking lossy compression. The resolution only indicates the maximum possible level of detail displayed, it doesn't say you gave it the bitrate to actually keep any of it.
An inconsistent picture doesn't look good. Dropping to 720 can help a low bitrate encode by making the decision to knock everything down to a lower (but still HD) resolution instead of leaving the encoder making tradeoffs in one spot to make room for for a few single-pixel details in another.

I do plenty of 60fps encodes and 6000kbps is in the range of a very high quality 720. (5-8k) It's nowhere in the range of a 1080, which is 10Mbps and up.



It could also allow you to up other settings further. (on the other hand multi-threading gets more difficult with less lines to split for the cores - maybe it's a bad idea after all?)

It's 66.7% of the vertical resolution with half the pixels, so you're dropping overall complexity faster than lines.

[Kisa_AG]

Hi,

"how to improve quality/sharpness and/or reduce pixelation on realtime 1080p 60fps x264 encode"
I'm using a software called OBS to stream my PC gameplay...


Did you try GPU based encoders?
OBS seems to have such possibility for both NVEnc and Intel Quick Sync.
I don't know if your MB has Intel graphics, but at least you have GTX980, so NVEnc should work.

Both technologies are incredebly fast and still can achieve very good quality.

[sneaker_ger]

He is already in the area of preset medium to slower. It's doubtful a hardware encoder will do better.

[Kisa_AG]

He is already in the area of preset medium to slower. It's doubtful a hardware encoder will do better.

He has NVidia card, worth a try.

PS: I tested Intel Quick Sync (Intel HD Graphics 4600) and from my opinion it gives reasonable quality. Let's say at x264 "medium" level.
At 1920x1080 with almost maximum quality settings it gives 2x realtime (60fps) on quite difficult video.
Command line: QSVEncC64 -i Direct.mp4 --la-icq 30 --la-depth 15 --ref 4 -b 5 --quality best -o Direct_recode.mp4

[Cinder]

You alway lose quality, that's why we are having this discussion. Maybe reducing resolution can help. It could also allow you to up other settings further. (on the other hand multi-threading gets more difficult with less lines to split for the cores - maybe it's a bad idea after all?)


720p on the slower preset doesn't look better than 1080p with the settings above, so I don't see any point in downscaling.

To get a rough idea where the settings fall in terms of speed lost/quality gained it can be useful to look under which preset they fall. Yours seem to fall roughly into medium, slow or slower presets. subme 7 is medium, trellis 2 is slow, umh and b-adapt 2 are slower preset. So you already have pretty good settings. Maybe try subme 8 and b-adapt 1 to consolidate everything to preset slow. Only thing left to tune would be things like deblock, psy, aq-strength, lookahead and number of bframes and ref frames. (But don't ask me what's best for Battlefield...)

Unfortunately I can't use both b-adapt 2 and umh. My CPU usage goes up to 70-80% and while on some Battlefield maps that's not an issue (no dropped frames), on others (e.g. lots of foliage/green stuff) it drops a lot of frames. I have to pick one or the other. I'm leaning towards keeping b-adapt 2 and using hex motion estimation. I think I can get further improvement by tweaking deblock, psy and aq-strenght, as you have suggested. I can't go much higher than 5 ref frames without tweaking down other settings and I'm already using 3 b-frames (is there any point in using more than 3 b-frames?)

Is there any place where I can see the possible values of deblock, psy and aq-strenght and what their values might result in?

Did you try GPU based encoders?

I have tried it, quality is nowhere near close x264 at the same bitrate. Not even close to veryfast preset.

[sneaker_ger]

720p on the slower preset doesn't look better than 1080p with the settings above, so I don't see any point in downscaling.
The idea is that you might get less blocking on lower resolutions. Macroblock is 16x16, less blocks on smaller resolution = less blocking. Slower preset would only be a bonus. But if you already tested it and it did nothing: ok.

Unfortunately I can't use both b-adapt 2 and umh.
b-adapt 1 was improved to offer similar quality as b-adapt 2 but with same speed as before. Unfortunately, the x264 revision inside your OBS is too old.

[rwill]

You are using a dynamic GOP pattern with B-Hierarchies and a maximum number of 3 B-frames.
I would try a static pattern like --bframes 3 --b-adapt 0 --b-pyramid strict which is what
you will get most of the time anyway but this time faster.

One problem I see is the amount of threads and the small video buffer ( --vbv-bufsize ).
The way it works is that N frames are done in parallel and special care is taken to not
underflow the buffer, which might happen if the frames all come out too big. So picture
quality is degraded on a macroblock line basis to meet a maximum frame size if one
overshoots a certain bit target determined prior to encoding. So the "wiggle" room a
parallel frame encode has is "something something video buffer / threads". With ~30
threads there is not much wiggle room. This might lead to problems.

Among of what can happen:
x264 underestimated the picture size prior to encoding and has to degrade picture quality
while encoding. x264 overestimated the picture size prior to encoding and then a picture
is encoded not as good as it could be. x264 overestimated the picture size while encoding
and degrades picture quality although it did not have to. x264 underestimated the picture
size while encoding and then has to clamp down HARD towards the end of the picture.

So maybe try a buffer of 5-6 seconds ( --vbv-bufsize (vbv-maxrate*5) ) or reduce the
number of threads ? Maybe lower the --bitrate. And maybe also try a higher --rc-lookahead,
this is what lets x264 see into the future to make better video buffer affecting decisions
in single pass.

Does twitch have a web page which shows what streams they are willing to accept ?

[pandy]

Are you able to confirm this 165Hz later feed to capture board where incoming video is decimated to 60Hz?
IMHO this may be main source for described issues...
I would use HW compression in NVidia (can be better to keep buffer in sync with encoder encoding) and i would limit generated frames to 60Hz. Rapid scene changes may trig problems and disabling scene detection with shorter GOP may reduce perceived issues.
Not sure if some preprocessing can be applied but some form of motion blur can improve overall quality.

[Cinder]

Hi rwill, thanks for chiming in.

You are using a dynamic GOP pattern with B-Hierarchies and a maximum number of 3 B-frames.
I would try a static pattern like --bframes 3 --b-adapt 0 --b-pyramid strict which is what
you will get most of the time anyway but this time faster.

I tried that and CPU usage dropped down a little. I can now use umh motion estimation and me_range 20 (with the encoder occasionally dropping below 60 fps). It did seem to improve quality a bit.

I'm overall happy with the quality achieved so far. The only issue that remains is that the encoder drops below 60fps on some scenes (e.g. when you die the camera zooms out), even though CPU utilization is around 50-60%. I guess it's a matter of how fast frames are being encoded, rather than available CPU resources. Is there anything I can do to avoid that, other than lowering settings to speed things up?

One problem I see is the amount of threads and the small video buffer ( --vbv-bufsize ).
The way it works is that N frames are done in parallel and special care is taken to not
underflow the buffer, which might happen if the frames all come out too big. So picture
quality is degraded on a macroblock line basis to meet a maximum frame size if one
overshoots a certain bit target determined prior to encoding. So the "wiggle" room a
parallel frame encode has is "something something video buffer / threads". With ~30
threads there is not much wiggle room. This might lead to problems.

Reducing the threads has a negative impact (encoder drops below 60 fps more frequently), while it doesn't seem to improve quality. On the other hand increasing threads (>54) didn't seem to have any effect - both positive or negative (as in degrading quality).

So maybe try a buffer of 5-6 seconds ( --vbv-bufsize (vbv-maxrate*5) ) or reduce the
number of threads ? Maybe lower the --bitrate. And maybe also try a higher --rc-lookahead

Increasing the buffer size too much causes high fluctuations on bitrate, which can cause viewers to buffer too much. I've increased it to 15000Kbps and it did seem to have a positive effect - as in reducing pixelation. No one complained with the stream buffering, so I'm guessing the bitrate fluctuations are not that bad.

Does twitch have a web page which shows what streams they are willing to accept ?

Not really. They have a help article (link (https://help.twitch.tv/customer/portal/articles/1253460)) that says the maximum bitrate should be 3500, but there's nothing regarding bitrate in their TOS. There are plenty of people who stream at bitrates way beyond that. I saw some Korean guy streaming at 14000 bitrate yesterday :D

Are you able to confirm this 165Hz later feed to capture board where incoming video is decimated to 60Hz?
IMHO this may be main source for described issues...

I'm not sure I understand the question. Could you elaborate please?

I would use HW compression in NVidia (can be better to keep buffer in sync with encoder encoding) and i would limit generated frames to 60Hz. Rapid scene changes may trig problems and disabling scene detection with shorter GOP may reduce perceived issues.
Not sure if some preprocessing can be applied but some form of motion blur can improve overall quality.

Using NVENC provides substantially lower quality than x264. Limiting frames to 60fps ingame would deteriorate the gameplay experience.

[pandy]

I'm not sure I understand the question. Could you elaborate please?

Well, in fact i'm asking for confirmation of things you wrote earlier - please look bellow:


'm using a software called OBS to stream my PC gameplay over the internet (twitch.tv). I have a two PC setup - a game PC and a stream (encoding) PC. The game PC feeds the gameplay stream to the stream PC via capture card in 1440p 165Hz resolution. The capture card then feeds a downscaled 1080p 60fps stream to OBS, which encodes it using x264 and sends it to Twitch's ingest servers.

1440p @ 165Hz... first it is a bit odd but OK - however for your target you should limit your framerate to something dividable by 60 without fraction (so nearest 120Hz, next 180Hz etc).



Using NVENC provides substantially lower quality than x264. Limiting frames to 60fps ingame would deteriorate the gameplay experience.

Depends on bitrate - if your goal is something like 5 - 10Mbps then perhaps yes but if your are not so restricted by your target then i bet that NVENC will provide lower latency and comparable quality to real time x264 encoding... Most important from game perspective is regular and constant framerate...
Decent antialiasing should also help to improve quality.

CG is extremely difficult to compress - you must be realistic in expectations.

[Cinder]

1440p @ 165Hz... first it is a bit odd but OK - however for your target you should limit your framerate to something dividable by 60 without fraction (so nearest 120Hz, next 180Hz etc).

1440p @ 165Hz is my monitor's resolution/refresh rate. The capture card is matching that, as I'm cloning the monitor to it. My fps ingame doesn't go much higher than 100 (120ish on rare occasions).


Depends on bitrate - if your goal is something like 5 - 10Mbps then perhaps yes but if your are not so restricted by your target then i bet that NVENC will provide lower latency and comparable quality to real time x264 encoding... Most important from game perspective is regular and constant framerate...
Decent antialiasing should also help to improve quality.

CG is extremely difficult to compress - you must be realistic in expectations.

Unfortunately I am limited by bitrate - 6-7k Kbps is the maximum I want to use.

As I said in one of my previous posts, I'm happy with the quality achieved so far, I don't think I'll try to improve any further. I may go for a different OBS version once they upgrade x264 to the current release. I could try and compile OBS from source with the latest x264 version, but that might prove to be challenging :)

Thank you, to everyone who contributed to the thread.

[mariush]

I somewhat agree with pandy, you should try configuring your monitor/output to 120 Hz and capture at 120 fps as well. As you say, games won't go often over 120 fps so it won't affect your gameplay but it may be easier to just drop every other frame instead of dropping random amounts of frames to get it down to 60 fps.

You should be able to create a custom resolution restricted to 120 Hz and your monitor should accept it just fine.
Alternatively, you could try locking the framerate to 150 Hz and stream at 50fps ( divide by a nice round 3). 50 fps should still be fluid and you get a few more bits to increase quality of the stream. Twitch would accept it just fine, and on most computers it would play just fine. It's not perfect as it's a mismatch between the number of frames and the update rate of monitors most people have but shouldn't be an issue for playback.

[Cinder]

Reducing the refresh rate to 120Hz (both monitor and capture card) didn't seem to have any effect. I even capped the ingame fps to 60 and that only made things worse. There was noticeable micro stutter, that was visible in recordings as well.

As I said, I'm happy with the quality I have right now. Only optimizations I'd be looking to do is encoding speed without sacrificing quality.

[pandy]

Reducing the refresh rate to 120Hz (both monitor and capture card) didn't seem to have any effect. I even capped the ingame fps to 60 and that only made things worse. There was noticeable micro stutter, that was visible in recordings as well.

As I said, I'm happy with the quality I have right now. Only optimizations I'd be looking to do is encoding speed without sacrificing quality.

And this is problem especially for bitrate 6 - 7Mbps (for 1080p60 it sounds pretty low) and CG content....
Are you able to try something else - FSAA x2 or 4 (higher better - assumption FSAA=SSAA), lower framerate (120Hz) for example, if possible add motion blur (generally soften video).
To tweak speed you should consider disabling CABAC and perhaps disable deblock loop but this lead directly to increase bitrate and reduce quality...
Would try to set shorter GOP - stream should have more uniform bitrate distribution with shorter GOP at a cost of increased bitrate but perhaps quality loss can be acceptable...
6Mbps for CG is very low bitrate...

Bellow my example (ffmpeg) for real time H.264 HP@L4.0 1080i50 encoding on relatively slow (Core Duo class) PC - for my content (also CG but relatively static) only Intra frames was used:
@set x264opts="qp=%qpval%:vbv_maxrate=20000:qpmin=4:level=4.0:ref=1:bframes=0:b_adapt=0:no_mbtree=1:no_mixed_refs=1:no_8x8dct=0:chroma_me=0:no_chroma_me=1:subme=1:me=dia:me_range=4:no_deblock=1:trellis=0:weightp=0:no_weightb=1:rc_lookahead=0:open_gop=0:keyint=1:min_keyint=1:sliced_threads=0:slices=%slice%:tff=1:interlaced=1:bluray_compat=1:pic_struct=1:aud=1:nal_hrd=vbr:force_cfr=1:cabac=0:threads=auto:no_psnr=1:no_ssim=1:colorprim=bt709:transfer=bt709:colormatrix=bt709:stitchable=1:constrained_intra=0:psy=0:no_psy=1:no_scenecut=1:no_deblock=1:aq_mode=0:ipratio=1.0:chroma_qp_offset=0"

[Wam7]

Hi Cinder, I'm glad you got it sorted. It was an interesting read as I've wrestled with a similar problem doing live capture with x264 on a Dual Xeon setup as well. These are the pertinent settings I've tweaked for best quality without dropping frames; deblock=1:-3:-3 / me=umh / subme=9 / me_range=24 / aq=3:0.80.

I'm using CRF and not a relatively low CBR as you are (though I understand why), which is the crux of your issues but you can go some way to minimising them with the settings you are now using. Just out of curiosity is it not possible to increase the CBR to at least ~ 7000? Those extra bits may probably help. ;)

[Cinder]

And this is problem especially for bitrate 6 - 7Mbps (for 1080p60 it sounds pretty low) and CG content....
Are you able to try something else - FSAA x2 or 4 (higher better - assumption FSAA=SSAA), lower framerate (120Hz) for example, if possible add motion blur (generally soften video).

I could, but motion blur is probably one of the things I hate the most. That's the first thing I would disable in a game :)


Hi Cinder, I'm glad you got it sorted. It was an interesting read as I've wrestled with a similar problem doing live capture with x264 on a Dual Xeon setup as well. These are the pertinent settings I've tweaked for best quality without dropping frames; deblock=1:-3:-3 / me=umh / subme=9 / me_range=24 / aq=3:0.80.

I'm using CRF and not a relatively low CBR as you are (though I understand why), which is the crux of your issues but you can go some way to minimising them with the settings you are now using. Just out of curiosity is it not possible to increase the CBR to at least ~ 7000? Those extra bits may probably help. ;)

Hey Wam, I guess I could increase bitrate, I'm starting to see more and more people streaming at higher bitrates.

[Wam7]

Hey Wam, I guess I could increase bitrate, I'm starting to see more and more people streaming at higher bitrates.Oh yes definitely increase bitrate then. If you can use ~8Mb then the quality should be permanently very good with the settings you're now using. I've been doing this for a long time now and as good as x264 is, you're asking a whole lot of it with 6Mb 1080p60. ;)

The vast majority of people watching on Twitch will have greater than 10Mb download. I watched some of your streams and my DUMeter is showing that I'm averaging at 6.5Mb so give it a try and see what the feedback is.

[Cinder]

Is there any reason to use level 4.0 or 4.1 opposed to level 5 that I'm using (I'm guessing that's auto) right now, other than compatibility with some devices?

[sneaker_ger]

1080p60 requires at least level 4.2. But you'd have to reduce the number of ref frames by 1. Only reason to lower would be to increase compatibility with client devices. If have you haven't heard any complains, then no.

[pandy]

I could, but motion blur is probably one of the things I hate the most. That's the first thing I would disable in a game :)

Motion blur on video before encoding (so in your case 1080p60) - this should reduce peaks in bitrate and smooth overall bitrate distribution. Seem that your problem is classical codec overload.
Side to this x264 (at least in ffmpeg) complain for more than 16 threads...

Full scene anti-aliasing may help to reduce bitrate requirements.

[rwill]

Isnt the death animation in BF1 some sort of fade to grey ? Have you tried
setting --weightp to 0 ? If all reference frames are weighted the list
is populated with duplicates and this might be somewhat much with 5 refs.

Regarding the --vbv-bufsize and high bitrate fluctuations, well you are
already encoding with true CBR ( filler=1 ). This means that the stream
output by x264 is to be interpreted as a constant bitrate stream. So there
is no fluctuation. If a client is able to receive constantly at vbv-maxrate
he only needs to buffer "vbv-bufsize" once and then is able to watch the
stream without the need for additional buffering. This is what the video
buffer model is for, constant rate receiving, an example is satellite TV.
Imagine a bucket at the decoder input ( vbv-bufsize ) that is filled by
a _constant_ stream of water ( vbv-maxrate ) by the encoder. On the
decoder side the decoder empties this bucket by withdrawing more or
less filled cups ( different frame sizes ). CBR means that the bucket
never overflows or underflows ( there is always enough data and never
too much data ). Always enough data is done by filler=1 and never too
much is done by ( sometimes severe ) quality reduction. I do not know
if OBS employs network sending at a constant rate or if it just sends as
fast as possible. At correct CBR it is possible to produce a horizontal
line in a network meter if the application doing the sending is aware
of - and honors - the vbv-maxrate.