x264 rgb vs i444 - What's the difference?

[mzso]

Hello!
Probably a noob question. I was wondering what's difference between using "--output-csp rgb" and "--output-csp i444 --range PC"?
Apart from the fact that the latter is about half the size, while all other settings are the same. What does rgb mean? If I understand correctly the h.264 format can't encode into rgb.

[LoRd_MuldeR]

Well, "i444" stands for YUV 4:4:4, I think, while "rgb" stands for, well, RGB.

While the YUV 4:4:4 format is similar, in a way, to the RGB format in that it does not subsample the chroma and takes the same number of bits per pixel (as "raw" data), YUV (YCbCr) (http://en.wikipedia.org/wiki/YCbCr) and RGB (http://en.wikipedia.org/wiki/RGB) still a two different ways of storing "color" information.

Also I think x264 does support encoding in the RGB (or BGR) format for quite a while now:
http://git.videolan.org/?p=x264.git;a=commit;h=ffc41542d73811dffbe5814a23edda65db0e6aea

(I think encoding in RGB might be less efficient than YCbCr, because the latter decorrelates the "chromaticity" and "brightness" information. Maybe also related to what the CABAC contexts are optimized/designed for?)

[vivan]

If I understand correctly the h.264 format can't encode into rgb.It can. There's colormatrix BGR with coefficients like:
Y = B
Cb = G
Cr = R
You can even use subsampling (though it doesn't make any sense), 10+ bit depth and so on...

Technically, for x264 the only difference is --colormatrix (GBR vs undef, which would default to Rec709 for HD, but x264 would use Rec601 for rgb -> yuv conversion. That's why you should perform rgb -> yuv conversion using avisynth).
RGB encoding is very inefficient, at least because you can't increase chroma quantizer (well, you can, but result will be weird). I think the only reason to use it is for lossless encoding. But even in that case 10-bit YCgCo could be more efficient.

So if you want to lossy encode your fraps video I would recommend using 10-bit 4:4:4 YUV.

[mzso]

It can. There's colormatrix BGR with coefficients like:
Y = B
Cb = G
Cr = R
You can even use subsampling (though it doesn't make any sense), 10+ bit depth and so on...

Technically, for x264 the only difference is --colormatrix (GBR vs undef, which would default to Rec709 for HD, but x264 would use Rec601 for rgb -> yuv conversion. That's why you should perform rgb -> yuv conversion using avisynth).
RGB encoding is very inefficient, at least because you can't increase chroma quantizer (well, you can, but result will be weird). I think the only reason to use it is for lossless encoding. But even in that case 10-bit YCgCo could be more efficient.

So if you want to lossy encode your fraps video I would recommend using 10-bit 4:4:4 YUV.

Ok. So from a user viewpoint i444 would be more efficient but I can't use it because the colors will be borked?

While the YUV 4:4:4 format is similar, in a way, to the RGB format in that it does not subsample the chroma and takes the same number of bits per pixel (as "raw" data)

Well, 4:4:4 doesn't subsample either otherwise it wouldn't be 4:4:4. :)


Also I think x264 does support encoding in the RGB (or BGR) format for quite a while now:
http://git.videolan.org/?p=x264.git;a=commit;h=ffc41542d73811dffbe5814a23edda65db0e6aea

I thought the format was pretty much yuv only.

(I think encoding in RGB might be less efficient than YCbCr, because the latter decorrelates the "chromaticity" and "brightness" information. Maybe also related to what the CABAC contexts are optimized/designed for?)

This makes sense then.

[mzso]

One more thing. The x264 help claims that "--range" is not used by the encoder but the end result is different the pc range encode results in a somewhat larger output file and also the color is full range (verified with madvr) while without it it's limited range.

[Dark Shikari]

Ok. So from a user viewpoint i444 would be more efficient but I can't use it because the colors will be borked?I have no idea where this idea came from, but the colors in YUV mode are not "borked" in any way I can think of.

I thought the format was pretty much yuv only.Nope, it's not.

One more thing. The x264 help claims that "--range" is not used by the encoder but the end result is different the pc range encode results in a somewhat larger output file and also the color is full range (verified with madvr) while without it it's limited range.--range affects the output range used by the YUV conversion process. If you're trying to make x264 assume the input is a particular range, you might want to look at --input-range.

[mzso]

I have no idea where this idea came from, but the colors in YUV mode are not "borked" in any way I can think of.
Well, the output is wrong. (maybe it's lightness) How would you say it then? Any way to make the output look "right", without using another program? (People suggested using avisynth)


--range affects the output range used by the YUV conversion process. If you're trying to make x264 assume the input is a particular range, you might want to look at --input-range.

That's cool, but it's in the VUI section in the help, where it's stated: "The VUI settings are not used by the encoder but are merely suggestions to the playback equipment." So this isn't true.

[LoRd_MuldeR]

The "--input-range" is under the input/output options (not VUI) and it's used by x264 to handle colorspace conversions of the input, before the actual encoding.

At the same time "--range" is used to indicate the range that the decoder shall use for any colorspace conversions that may be required after decompressing the video.

As x264 is not involved in the decoding process, all it can do is write the desired "--range" into the VUI and hope that the decoder will take care of that...

(BTW: If you get any "wrong" colors with YUV 4:4:4 then there obviously is at least one "wrong" YUV/RGB conversion somewhere between your original RGB video that you feed into x264 and the final RGB output you get to see on the screen when watching the encoded video. There are many places where this can happen, including the encoder, the decoder and the video renderer)

[mzso]

The "--input-range" is under the input/output options (not VUI) and it's used by x264 to handle colorspace conversions of the input, before the actual encoding.

Really? This wasn't clear at all from the former comments. :)

At the same time "--range" is used to indicate the range that the decoder shall use for any colorspace conversions that may be required after decompressing the video.

As x264 is not involved in the decoding process, all it can do is write the desired "--range" into the VUI and hope that the decoder will take care of that...

(BTW: If you get any "wrong" colors with YUV 4:4:4 then there obviously is at least one "wrong" YUV/RGB conversion somewhere between your original RGB video that you feed into x264 and the final RGB output you get to see on the screen when watching the encoded video. There are many places where this can happen, including the encoder, the decoder and the video renderer)

DS just said that it "affects the output range used by the YUV conversion" and not just specify the range. Also with --range PC the output is like 13% larger.


I guess I'll try forcing "--input-range"

[Dark Shikari]

That's cool, but it's in the VUI section in the help, where it's stated: "The VUI settings are not used by the encoder but are merely suggestions to the playback equipment." So this isn't true.The conversion process is not part of the encoder. The range flag indeed does not affect the encoding process, but it does affect the conversion process if your input needs to be converted first by swscale.

Also, I kind of wrote the code. If the documentation says otherwise, the documentation is probably wrong. That particular line dates to around half a decade before x264cli even had the ability to perform colorspace conversion.

[mzso]

The conversion process is not part of the encoder. The range flag indeed does not affect the encoding process, but it does affect the conversion process if your input needs to be converted first by swscale.

Also, I kind of wrote the code. If the documentation says otherwise, the documentation is probably wrong. That particular line dates to around half a decade before x264cli even had the ability to perform colorspace conversion.

Ok, now I'm getting confused. So I get different file sizes because different color space conversion and so different input to the actual encoding process?

Anyway I just pointed out a conflict with help text, I'm sure what you say is valid.

[Dark Shikari]

Yup: there's four possible outputs of the conversion process:

1. Assume input is full range (--input-range), convert to full range (--range).
2. Assume input is full range, convert to limited range.
3. Assume input is limited range, convert to full range.
4. Assume input is limited range, convert to limited range.

The encoder then encodes the resulting stream and flags it based on --range. So based on which conversion is done, the output might be different. The "doesn't affect output" can be seen with cases 1 and 4: no conversion is done here, so the encoder will get the same input no matter what you do, so the output should be the same.

[LoRd_MuldeR]

Really? This wasn't clear at all from the former comments. :)

DS just said that it "affects the output range used by the YUV conversion" and not just specify the range. Also with --range PC the output is like 13% larger.

I think I was missing one point: The color conversion performed by x264 prior to the encoding is influenced by both, "--input-range" and "--range".

Still the value of "--range" (and only that one) also needs to be written into the VUI and we have to hope the decoder/renderer is going to take care of that properly...

[mzso]

I can't make it right with i444. It never looks right.
If the input range option isn't changed the result is similar, but subtly different. (Also if --range PC the sun seemingly increases). If I force the input range too the result is obviously too light.

Source:
http://abload.de/img/teszt.avi_20130608_13s6pc7.png

Rgb (rather indistinguishable):
http://abload.de/img/rgb.mkv_20130608_1345vgrwn.png

i444 (results in TV range.):
http://abload.de/img/xteszt-tvrange.mkv_20l7rpz.png

i444 --range PC:
http://abload.de/img/xteszt-pcrange.mkv_205yq21.png

i444 --input-range PC --range PC
http://abload.de/img/xteszt-inrange-outran0rqjh.png

[vivan]

As I said earlier this is rgb -> yuv conversion issue.
which would default to Rec709 for HD, but x264 would use Rec601 for rgb -> yuv conversion. That's why you should perform rgb -> yuv conversion using avisynth).
This is your "i444 (results in TV range.):" picture, converted to yv24 using Rec709 matrix and then back to rgb using Rec601 matrix.
http://5.firepic.org/5/images/2013-06/08/y63usdvxlsnv.png
Looks as source, right?

If you don't want to use avisynth for proper rgb -> yuv conversion you have to add --colormatrix bt470bg to x264 encdoing string. However it will work only with madVR + LAV video decoder, other renderers ignore colormatrix flag, and other decoders doesn't send it to the renderer.

As for i444 --input-range PC --range PCStrange that it doesn't look like just i444. Are you using madVR and LAV video decoder?

[mzso]

As I said earlier this is rgb -> yuv conversion issue.

This is your "i444 (results in TV range.):" picture, converted to yv24 using Rec709 matrix and then back to rgb using Rec601 matrix.
http://5.firepic.org/5/images/2013-06/08/y63usdvxlsnv.png
Looks as source, right?

If you don't want to use avisynth for proper rgb -> yuv conversion you have to add --colormatrix bt470bg to x264 encdoing string. However it will work only with madVR + LAV video decoder, other renderers ignore colormatrix flag, and other decoders doesn't send it to the renderer.

As for Strange that it doesn't look like just i444. Are you using madVR and LAV video decoder?

But why can't x264 do it properly, why do I need to use avisynth?

Yes, I use LAV + madVR.

[Dark Shikari]

x264 doesn't currently have the ability to select whether or not to use BT.709 or BT.601 matrixes when doing conversion; patches welcome.

This only matters when converting from RGB->YUV within x264 for input.

[mzso]

x264 doesn't currently have the ability to select whether or not to use BT.709 or BT.601 matrixes when doing conversion; patches welcome.

This only matters when converting from RGB->YUV within x264 for input.

Too bad...
I seem to always find something that's not doable. Whether it's ffmpeg or x264 doesn't matter.

[poisondeathray]

Too bad...
I seem to always find something that's not doable. Whether it's ffmpeg or x264 doesn't matter.

It's "doable" with avisynth

ffmpeg can do it with -vf colormatrix=bt601:bt709 , (and -pix_fmt yuv444p for i444), but for some reason the quality is noticably lower than doing it through avisynth (I suspect something is wrong with the colormatrix patch in ffmpeg)

[mzso]

It's "doable" with avisynth

ffmpeg can do it with -vf colormatrix=bt601:bt709 , (and -pix_fmt yuv444p for i444), but for some reason the quality is noticably lower than doing it through avisynth (I suspect something is wrong with the colormatrix patch in ffmpeg)

On the other hand you can't make ffmpeg to keep full range with libx264 nor can it encode into rgb.

I'll just stick with rgb. Even though I have to in two/three steps. A good encoder tool just doesn't exist.
(I even tried mencoder, but it doesn't accept output_csp even though it's in the documentation and fails with "FATAL: Cannot initialize video driver." which I can't even comprehend :) )

[poisondeathray]

Yes, ffmpeg libx264 doesn't seem to have all the switches or at least I don't know the equivalent syntax

Just curious - Why the aversion to avisynth + x264 ?

[mzso]

Yes, ffmpeg libx264 doesn't seem to have all the switches or at least I don't know the equivalent syntax

Just curious - Why the aversion to avisynth + x264 ?

Well, I don't feel like familiarizing myself with yet another program. I already wasted a lot of time trying to make ffmpeg, than x264 to do what I want.

[STaRGaZeR]

Well, I don't feel like familiarizing myself with yet another program. I already wasted a lot of time trying to make ffmpeg, than x264 to do what I want.

You wasted your time, indeed. All you need is a 2 line Avisynth script with ConvertToYV24 in it. No need to familiarize yourself with anything else. Now, if you want to keep full range or something exotic like that, it'll do it too without issues but forget about correct output outside of a very specific set of tools.

[mzso]

You wasted your time, indeed. All you need is a 2 line Avisynth script with ConvertToYV24 in it. No need to familiarize yourself with anything else. Now, if you want to keep full range or something exotic like that, it'll do it too without issues but forget about correct output outside of a very specific set of tools.

Kind of sad, that full range is considered exotic.

[aufkrawall]

Kind of sad, that full range is considered exotic.
Not really bad since eyes don't notice the difference.

[mzso]

Not really bad since eyes don't notice the difference.

It's an unreliable mess though. And it's absurd.

[aufkrawall]

It's an unreliable mess though. And it's absurd.
Why? Usually YUV isn't lossless and so why not categorically dump some information that aren't needed anyway?

Well, of course you can use full range. Works fine with MPC HC + LAV + madVR. Apart from that combo, I444/10bit support is hardly available anyway.
Problem are lazy developers of other A/V software. :(

[STaRGaZeR]

Kind of sad, that full range is considered exotic.

Indeed, but that's how things are. You should be happy about what you got, because if it were for the big guys in the industry you'd have never had the chance to even see how it looks.

You want compatibility, you have to play by the rules and use what almost everything else out there expects. You want the best quality, you have like 1 combination of tools that produce correct output (and that's if you flag the stuff properly). I stopped aiming for that little % of quality that you'll never see except in very specific situations and instead aim for better compatibility.

PS: Well, as compatible as a 4:4:4 file can get :D

[mzso]

Did anything change? It'd nice if I could maintain proper color and range without having 2.5x the filesize (because of rgb colorspace).

[mzso]

Apparently I only need to use yuvj444p with ffmpeg instead of yuv444p and I get proper output...

[Asmodian]

It's "doable" with avisynth

ffmpeg can do it with -vf colormatrix=bt601:bt709 , (and -pix_fmt yuv444p for i444), but for some reason the quality is noticably lower than doing it through avisynth (I suspect something is wrong with the colormatrix patch in ffmpeg)

using -pix_fmt yuvj444p will use full range but you still need the -vf colormatrix=bt601:bt709 to get correct colors don't you?

[mzso]

using -pix_fmt yuvj444p will use full range but you still need the -vf colormatrix=bt601:bt709 to get correct colors don't you?

I didn't think so. They looked about the same.

Now that you mention it there is some less obvious difference in lightness. But that might be inaccurate colorspace conversion right? Someone mentioned earlier that ffmpeg's is not the best. Anyway I don't have the original anymore to test it.

Images removed, becaus they're invalid

(By the way does anyone know which image hoster has that overlaid comparison feature, where I can switch between images by hovering with the cursor?)

[mzso]

Okay so the previous snapshots can be considered invalid, because I used the player's snapshot feature which produces borked images.

Made some new ones:

Normal:

-vf:

Original:



Both seem to be brighter. Maybe with colormatrix=bt601:bt709 less so but surely a lot worse with a lot of detail lost.

Edit:
Removed images.

[nevcairiel]

You want http://screenshotcomparison.com/

The difference in those images looks like the difference between 601 and 709 could cause.

[mzso]

You want http://screenshotcomparison.com/

The difference in those images looks like the difference between 601 and 709 could cause.

Thanks for the info. You mean the original and "normal"?

If "-vf colormatrix=bt601:bt709" is no good for fixing it then what is?

Edit: apparently using -preset ultrafast also causes something like that. :)
I can't tell the difference with slower preset and crf15.
Original:

Slower, crf15:

[Asmodian]

If "-vf colormatrix=bt601:bt709" is no good for fixing it then what is?

Avisynth :p

Edit: apparently using -preset ultrafast also causes something like that. :)
I can't tell the difference with slower preset and crf15.

The detail lost could be due to -preset ultrafast but the color difference is due to the 601:709 issue.

"Normal" looks like watching 601 at 709 to me, "-vf" looks more like the original. I notice most of the difference between them in the reds, "Normal" has an oranger tint.

I think you used the same shot for both "Original" and "Slower, crf15" (they look identical and have the same file name).

[mzso]

The detail lost could be due to -preset ultrafast but the color difference is due to the 601:709 issue.

"Normal" looks like watching 601 at 709 to me, "-vf" looks more like the original. I notice most of the difference between them in the reds, "Normal" has an oranger tint.

I think you used the same shot for both "Original" and "Slower, crf15" (they look identical and have the same file name).

Yeah I messed up the links (I removed them) and who knows what. Also I'm quite sure I couldn't tell the difference locally. So I re-did it.
And it looks like you're right. Darn...
Original vs. Encode
http://screenshotcomparison.com/comparison/64322
Original vs. Encode with colormatrix=bt601:bt709
http://screenshotcomparison.com/comparison/64323

Well. maybe with the colormatrix filter the color deviation is less, but the quality is definitely a lot worse.

Back to square two (not one, because I can encode full range properly at least...) I probably couldn't tell the difference because my image viwer kind of turns (blinks) to black before showing the next image...

[nevcairiel]

The colormatrix filter is indeed not very high quality, probably a better idea to try to use AviSynth for such changes.

[mzso]

The colormatrix filter is indeed not every high quality, probably a better idea to try to use AviSynth for such changes.

It seems so. I wanted to avoid using(and learning to use) separate programs for one task.

[mzso]

Anyone know of a guide about setting up and using avisynth with ffmpeg?

[mzso]

Well I got as far as getting msvcr71.dll which ffmpeg missed when I opened an avs file.
But then I got: Script error: there is no function named "ConvertToYV24"
Which is supposed to be an internal filter. So I'm lost.

[Blue_MiSfit]

Are you running AviSynth 2.6? 2.5.8 doesn't support YV24.

[mzso]

I removed the old avisynth and installed Avisynth+

Used this CL and AVS script:
ffmpeg -i rec709.avs -acodec flac -compression_level 12 -vcodec libx264 -preset slower -crf 18 -pix_fmt yuvj444p avsteszt-rec709.mkv

AVISource("e:\Videó Felvételek\motor2.avi")
ConvertToYV24(matrix="PC.709")

And the result is about the same! :confused: http://screenshotcomparison.com/comparison/64382
So now I'm really confused. The exception is that for some reason when I used the jump the middle function of potplayer it jumped to frame 118 instead 117. So something is different just not in a useful way. (Edit: Maybe because of this: "Stream #0: not enough frames to estimate rate; consider increasing probesize" Don't know why I get it or what to do with it...)

PS:
It seemed to me that I should use PC.709 it's described as such: "Uses Rec.709 coefficients, keep full range"
But I got a way too dark picture.

[mzso]

Are you running AviSynth 2.6? 2.5.8 doesn't support YV24.

Yes, I was. Sorry, I didn't notice your comment at first.


Anyway. I noticed that I'm still getting this warning: "[swscaler @ 03ff88a0] deprecated pixel format used, make sure you did set range correctly"

So maybe ffmpeg still manipulates the image before it gets to libx264?

Or that's irrelevant and libx264 fails at encoding?

[mzso]

Why is that if I use x264 with avisynth I need to use "ConvertToYV24(matrix="PC.709")" to get proper lightness levels (otherwise too light), but if I use ffmpeg with libx264 I need to use "ConvertToYV24(matrix="Rec709")" otherwise the video is too light?
It looks like one of them is doing something something nasty. My bet is on ffmpeg because "PC.709" is supposed to be full range, yet it can't produce a full range output with it.

Otherwise the colors are just as wrong (the result being more orage-ish) if I encode with x264 and "ConvertToYV24(matrix="PC.709")".

[sneaker_ger]

Maybe there's a conversion already going on in the decoder. Instead of using AviSource() try L-Smash or ffms2 which usually keep the original colorspace (if your format is supported). Then do "ConvertToYV24(matrix="Rec709")" (most players assume limited range BT.709 for HD resolutions). Also, check what colorspace you actually have directly after the source filter ("info()"). Also, any reason for using -pix_fmt yuvj444p instead of -pix_fmt yuv444p? I'm not even sure it's needed. Might also want to test x264cli instead of ffmpeg.

[mzso]

Maybe there's a conversion already going on in the decoder. Instead of using AviSource() try L-Smash or ffms2 which usually keep the original colorspace (if your format is supported). Then do "ConvertToYV24(matrix="Rec709")" (most players assume limited range BT.709 for HD resolutions). Also, check what colorspace you actually have directly after the source filter ("info()"). Also, any reason for using -pix_fmt yuvj444p instead of -pix_fmt yuv444p? I'm not even sure it's needed. Might also want to test x264cli instead of ffmpeg.

""ConvertToYV24(matrix="Rec709")" (most players assume limited range BT.709 for HD resolutions)"
I want to preserve full range. So should I use tv.709? (madvr+LAV handles full range properly)

"-pix_fmt yuvj444p instead of -pix_fmt yuv444p"
With yuv444p got limited range output. I want full range.

[sneaker_ger]

I want to preserve full range. So should I use tv.709? (madvr+LAV handles full range properly)
Well, if you really want to preserve full range (why?) you'd use "PC.709".

[mzso]

Well, if you really want to preserve full range (why?) you'd use "PC.709".

This troubles me, because ffmpeg produces a way too dark image when I use it, as if it doesn't handle full range properly in this case (If I decode the stream with ffmpeg itself the output is correct in this regard.) Didn't figure out a way to get past it.

So I tried it with FFVideoSource. The source is an UT Video avi which LSMASHVideoSource doesn't support apparently. And I pretty much got the same result. http://screenshotcomparison.com/comparison/64616

[sneaker_ger]

What does info() say after ffvideosource()?
How did you create your "original" screenshot?
How did you create your "x264_pc709-ffms2" screenshot?