Game/Monitor Capture RGB -> YUV (Full or Limited Range?)

[zerowalker]

Been wondering a bit about this, Full and Limited Range.

From what i get, Limited is the TV standard, while PC is Full Range.

Now i may be wrong, hence why i ask,

If the PC is Full, should the RGB -> YUV conversion be done with a Full Range flag then?

As i haven't had any problems at all without this, which makes me skeptical to it, as i know that forcing Full on a Limited causes huge difference in Contrast and Brightness.

So anyone can spread some light on this?

Thanks

[foxyshadis]

Full should be used wherever you have full control over the chain from RGB in to RGB out, and all of your tools can process full-range YUV, since you get that extra 15% fidelity for free. If a custom player or codec is needed, it's a viable option. In other circumstances, including any kind of distribution, limited should be used, since almost all players & editors default to that and many won't even read full-range metadata.

If you're working in 10+ bits that 15% isn't important anymore, then it's much simpler to just stick to limited for everything.

[zerowalker]

So, for Youtube and such it's worthless?

But what does that extra 15% give, that's pretty much color information, and losing that should have quite an impact right?

How come 10bit doesn't matter for that, i thought bit depth and range was 2 different things?

EDIT:

Actually tried capturing with Full Range YUV, it looks the same as it would with Limited forced to Full.
So i guess the Codec(MagicYUV) or Capture Software (Dxtory) does a limited conversion somewhere?

[foxyshadis]

So, for Youtube and such it's worthless?
Youtube will never be able to display more than 8bit limited range, so yeah.

But what does that extra 15% give, that's pretty much color information, and losing that should have quite an impact right?

How come 10bit doesn't matter for that, i thought bit depth and range was 2 different things?
They're closely related, both give you more tones to work with, but high bit depth gives you far more. Full range only gives you slightly less banding/more accurate rounding, but it's compatible with far more 8-bit only processing tools. The accumulated rounding errors of going from full-to-limited-to-rgb are probably more than the little extra headroom is worth.

With 10-bit it doesn't matter because instead of going from a very crowded 224 to 256, you get 896 with limited-10 and don't need a full 1024 as much, either is plenty for further processing and 8-bit RGB conversion.

Actually tried capturing with Full Range YUV, it looks the same as it would with Limited forced to Full.
So i guess the Codec(MagicYUV) or Capture Software (Dxtory) does a limited conversion somewhere?

MagicYUV doesn't, but Dxtory might. Did you also test by capturing to RGB and then converting to YUV full-range manually? (You won't notice a huge difference at a glance, mostly in very smooth gradients the banding will be in different places.)

[zerowalker]

Youtube will never be able to display more than 8bit limited range, so yeah.

Okay, thought Full Range was more common, like Interlaced and Progressive, seems like i was way off.

Full range only gives you slightly less banding/more accurate rounding, but it's compatible with far more 8-bit only processing tools.

True, 10bit isn't available in any normal editing software currently as far as i know, and it's still fairly new in the consumer world. Or at least the first time i heard of it and started using it was with x264 and it's compression efficiency.

With 10-bit it doesn't matter because instead of going from a very crowded 224 to 256, you get 896 with limited-10 and don't need a full 1024 as much, either is plenty for further processing and 8-bit RGB conversion.

ah so both used the same "numbers", thought they were completely different things, but that explains more, and as you say, 224-256 is nothing when you are at about 3-4 times that amount. You got a lot of headroom for rounding errors, guess you could compare it to 16-bit and 32-bit audio editing and noise floor.

MagicYUV doesn't, but Dxtory might. Did you also test by capturing to RGB and then converting to YUV full-range manually? (You won't notice a huge difference at a glance, mostly in very smooth gradients the banding will be in different places.)

That's understandable as the Codec is just handshaking with the sender, haven't tried doing it manually but will now and report back.

EDIT:

Having an RGB recording and Converting to PC.709 will Darken the image, while Rec.709 will make it look normal.

I am guessing that if it's Full, it would be the opposite, PC.709 would look normal, Rec.709 would look brighter (709/601 doesn't really matter here).

EDIT 2:

Or wait, perhaps i have mistaken here, do i have to manually convert it to RGB with the same parameters for it to work?
If so i it works, and i notice a difference, it's appearant at gradients as you said, not breaking, but Limited seems to "cut" it making it less accurate, though i can't say that Full vs RGB is identical as the gradients are quite different from that conversion as well. Though my capture isn't the best example either.

[Asmodian]

Yes, your edit 2 is correct. Your playback chain isn't respecting the full range flag if the image is darkened on playback. (only LAV + madVR respect such flags as far as I know)

[zerowalker]

The MagicYUV flag doesn't seem to be working at all for MadVR, can't say much for the playback for that one, but other than that, learned something new.

Wonder though, can't this be used as 8-10bit to increase compression efficiency, without going to 10-bit (although with much less difference).

[EncodedMango]

Sorry, don't wish to 'hijack' the thread but today I've been wondering about the something very closely related to this...

So I once had this issue (http://forum.doom9.org/showthread.php?t=170078:) and I was wondering what went wrong, that question was answered and the method suggested there worked but I never really understood why Rec709 gives the proper colours but not PC.709 despite using MPC-HC+madVR and source being RGB. I ignored that back then but I tried to replicate it using ffmpeg without any involvement of Avisynth today.

I thought maybe changing pix_fmt yuv420p to yuvj420p would fix the colours because I read somewhere that the J makes it full range but it didn't work. So I kept on searching and trying different things and came across this thread (http://forum.doom9.org/showthread.php?t=170306) and checked every command there and seeing as how in Avisynth earlier I used Rec.709 instead of PC.709, one preset doing the same caught my eye with the option
-vf scale=out_color_matrix=bt709

I tried this and voila! it worked!

Now, can someone explain to me why exactly did this happen?
madVR supposedly reads the correct flags.
The source footage is RGB and converting it to YUV using PC.709 seems like the logical choice for PC playback. [Lossless codec used to capture game footage in this case was Lagarith]
But clearly that gives the wrong colours, Rec.709 gives the correct colours in both the cases of AviSynth and ffmpeg's filter.

[Asmodian]

The step I do not see is the tagging, does your ffmpeg line tag the file with the full range flag? I think ffmpeg needs more options set to tell it to output full range video and I am not sure it does it correctly. Was it the input or output pix_fmt you set to yuvj420p? Maybe give it a shot with +pix_fmt?

-pix_fmt[:stream_specifier] format (input/output,per-stream)’
Set pixel format. Use -pix_fmts to show all the supported pixel formats. If the selected pixel format can not be selected, ffmpeg will print a warning and select the best pixel format supported by the encoder. If pix_fmt is prefixed by a +, ffmpeg will exit with an error if the requested pixel format can not be selected, and automatic conversions inside filtergraphs are disabled. If pix_fmt is a single +, ffmpeg selects the same pixel format as the input (or graph output) and automatic conversions are disabled.

With x264 you would set --input-range PC after using PC.709 in Avisynth.

[EncodedMango]

I'll check the ffmpeg method in a while, I just tried the x264 method using AviSynth PC709 matrix and then --input-range PC in the x264 command line. That did produce the correct colours, now I am even more confused. So what exactly was I doing wrong and what did that lead to?

Also should I use this method? In the original thread this was said when I asked if I should use PC.709 instead of Rec.709:
PC matrix would actually be better quality wise (you're "mapping" 0-255 RGB to 0-255 Y'CbCr, instead of 0-255 to 16-235 Y' , 16-240 CbCr), but the majority of setups will convert back to RGB for display using Rec709 matrix and the levels will look incorrect. But the setups that obey full range flags will decode it correctly if you used a PC matrix for the actual RGB=>YUV conversion
Does this still hold true in my case? I don't really see much point in using this method if it will only work on some setups and not most.

[Asmodian]

As far as I know it only works with LAV + madVR, all other players assume limited range. So I guess you probably want to use Rec.709. ;)

[EncodedMango]

I guess that's what I'll continue using then, but I still don't get why Rec709 works, if that's limited range why does it still show the correct colours for my footage?

[Asmodian]

RGB is not limited or full. You are doing RGB (game) -> YUV (file) -> RGB (screen)

You simply need to use the same matrix (well matrix and its inverse) for both the conversion from RGB and the conversion to RGB.

[EncodedMango]

I see, thanks!

[Reino]

sidspyker, your video is 720p RGB, so only the following 2 apply:

Fraps(>576p,RGB) --> H.264(YV12 TV-Range,BT.709) with FFMpeg's videofilter:
ffmpeg.exe -i "FPS1(bgr24)_720p30_sample.avi" -pix_fmt yuv420p -c:v libx264 -vf scale=out_color_matrix=bt709 ^
-an "FPS1(bgr24)_720p30_sample(ffmpeg,vf,yv12-tv709).mkv"

Fraps(>576p,RGB) --> H.264(YV12 PC-Range,BT.709) with FFMpeg's videofilter:
ffmpeg.exe -i "FPS1(bgr24)_720p30_sample.avi" -pix_fmt yuvj420p -c:v libx264 -vf scale=out_color_matrix=bt709 ^
-an "FPS1(bgr24)_720p30_sample(ffmpeg,vf,yv12-pc709).mkv"

By default x264 uses rec601 (TV-Range,BT.601) for RGB -> YUV conversion, as vivan already mentioned. Your video is >576p, so you need to convert to BT.709. If you don't want to use AviSynth, then you can use FFMpeg's videofilter. As most decoders already use the BT.709-matrix based on the video resolution, you don't have to specify any VUI-commands.

Afaik only MadVR respects pc-range flags. All other video renderers default to tv-range. For compatibility I'd use tv-range, but if you want pc-range, just use -pix_fmt yuvj420p.
Note: Although x264 uses rec601 for RGB -> YUV conversion, ffmpeg doesn't if the input is RGB. You have to specify -pix_fmt yuv420p in that case, otherwise it encodes to bgr24.

[EncodedMango]

Thanks corone.