the video is blank because it's blank the size is none existent...
x264 has RGB support it's called BGR is relative useless except for lossless encoding. the reason x264 is interesting at all is the speed thanks to it's very good multithreading support.
the encoding doesn't really care in lossless mode if one channel is Y, R, G, B, Cb or Cr they are all saved the same way lossless in lossy mode all channel needs to be treated as Y making it very inefficient.
if nvidia supports it is another story they support 444 lossless for sure.
edit: ok got it working in obs wasn't easy.
it works when the advanced settings are set to RGB full range BT709.
custom ffpeg output is needed with libx264rgb as the encoder.
it usages a different syntax as x264 and libx264 which are -- for x264 - for libx64 and zero - for obs but spaces between option names need to be replace with = as an example qp=0 is lossless.
example file:
https://www.mediafire.com/file/vii46...00-27.mkv/file
other codec should be usable like this too.