Avisynth's ConvertToYV12() giving 4 different results

[pancserzso]

I have a computer generated, 8-bit RGB image [0-255].

I'd like to convert it to YV12 using Avisynth. I'm using 2.6.0 Alpha 4 (AviSynth_130114.exe).

My problem is that every single option for ConvertToYV12() gives a different result, and none of them are close to the original one.

I'm using ffmpeg.exe -i avs.avs yv12.png for saving an RGB image.

The Avisynth file is a simple
ImageSource( "src.png", 1, 1, 24 )
ConvertToYV12( matrix = ... )


Here are the results I get:
source (https://dl.dropbox.com/s/ccgu8mmvn99khqe/src.png)
default, just () (https://dl.dropbox.com/s/e7wufcz47ld450n/default.png)
PC.601 (https://dl.dropbox.com/s/s7kaev0atut1rwq/PC.601.png)
PC.709 (https://dl.dropbox.com/s/2x83f22n9kl4yvu/PC.709.png)
Rec709 (https://dl.dropbox.com/s/xcjz0btfmixigkl/Rec709.png)

I'm a bit confused:
1. Why are all options different? I thought the default option should be one of them. Isn't it?
2. Which one should I use? I'd like to render a 720p or 1080p material, using x264. Should it be Rec709?
3. Do I get these false colors because of some problem with ffmpeg, or its in Avisynth? What matrix is used by ffmpeg?

[Dark Shikari]

The default is probably TV-range 601 (Rec601).

[vivan]

I'm a bit confused:
1. Why are all options different? I thought the default option should be one of them. Isn't it?No, default is Rec601. http://avisynth.org/mediawiki/ConvertToYV12

2. Which one should I use? I'd like to render a 720p or 1080p material, using x264. Should it be Rec709?Yes, Rec709.

3. Do I get these false colors because of some problem with ffmpeg, or its in Avisynth? What matrix is used by ffmpeg?for YUV <-> RGB conversion ffmpeg also always uses Rec601... That's why closest result was "default".
If you want to perform correct YUV -> RGB conversion you should use player (e.g. MPC-HC) or avisynth (ConvertToRGB24 (matrix = "Rec709")). That's how it shoud be: http://4.firepic.org/4/images/2013-06/18/xtn0kr7pa9d5.png (note that it is the same as your "default" result).

However with YV12 you can't get exactly the same image as source. That's because YV12 colorspace uses subsampling (4:2:0) - color (chroma) planes have half resolution. It's only noticeable on images with small color details... and that's exactly your case. You can read about it on wiki (https://en.wikipedia.org/wiki/Chroma_subsampling).
You can use YV24 colorspace, but it's the same as with RGB encoding - only some software players on PC support it.

[feisty2]

only at least 10bit yv24 can convert back to 8bit rgb losslessly (all rounding,no dither)
in this case I suggest using 10bit yv24 ycgco because there's no dither needed in the colorspace conversion

[pandy]

I have a computer generated, 8-bit RGB image [0-255].
I'm a bit confused:
1. Why are all options different? I thought the default option should be one of them. Isn't it?
2. Which one should I use? I'd like to render a 720p or 1080p material, using x264. Should it be Rec709?
3. Do I get these false colors because of some problem with ffmpeg, or its in Avisynth? What matrix is used by ffmpeg?

Ad1. Yes - mentioned Rec601 is default
Ad2. depends from render - if target render is PC then go for PC.709 - Rec709 is for consumer devices that usually expand Rec709 (limited amount levels) to full RGB.
Ad3. Usually default is for SD content Rec601 and for HD Rec709 but on PC this can be sometimes not supported correctly (there is no clear convention).

You need provide correct information what is source in your case - for example for Rec709 you need to add in ffmpeg or x264 something like this:

in case of the ffmpeg:
-x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709:fullrange=off

for x264:
--colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --range "tv"

[raffriff42]

You have chosen a very difficult source. All those thin colored lines MUST become less vivid when converting to YV12. It's difficult to judge what is due to a matrix color shift, what's a luma range error, and what is an artifact of the loss of chroma resolution. You need to work with color bars -- or at least with large, flat areas of known color -- so you can see what is happening.

Comparing this...
C=ImageSource("src.png")
Return C.SubTitle("original")
...to this (viewing in VirtualDub or other player with frame jogging)...
C=ImageSource("src.png")
Return
\ C.ConvertToYV24()
\ .Trim(0,-1).SubTitle("default", align=5) +
\ C.ConvertToYV24(matrix="Rec601")
\ .Trim(0,-1).SubTitle("Rec601", align=5) +
\ C.ConvertToYV24(matrix="Rec709")
\ .Trim(0,-1).SubTitle("Rec709", align=5) +
\ C.ConvertToYV24(matrix="PC.601")
\ .Trim(0,-1).SubTitle("PC.601", align=5) +
\ C.ConvertToYV24(matrix="PC.709")
\ .Trim(0,-1).SubTitle("PC.709", align=5)
...it seems to me after close comparison that one of them is correct and the others are not*. Note I am using ConvertToYV24 for this test to eliminate the subsampling problem while we are looking for the best chroma matrix. Also note "default" is exactly "Rec601". Again, this is a bad source for judging color; it's best to do the test with color bars to be sure.

* pick your own favorite, using your particular playback signal chain.

[pancserzso]

Thanks a lot for all the replies!

So to get the theory right, I've simplified my test to AVS only with RGB > YUV > RGB conversion inside a single script.

ImageSource( "src.png", 1, 1, 24 )

ConvertToYV12( matrix = "..." )
ConvertToRGB24 (matrix = "...")

ImageWriter( "AVS-...", 0, 0, "png" )

This way I have to conclude the following:

1. All 4 conversion look visually the same to each other, but yes, different from the source! So in theory everything is correct, I just need to pass on the colorspace to the encoder and then to the player to get everything right!

2. RGB -> YV24 -> RGB with PC.709 looks visually the same as the original source! Even in an image comparison program, the difference is only 1%!

So my question is:
1. Do I generally always have to pass on the color information to the encoder? In the x264 wiki, I've read that it uses 'undef' by default. What is that? Is it x264 which 'hints' the decoder about what color profile to use, or it's totally up to the player? How does say LAV filter decide which colorspace to use?

2. What happens with PC.709 on hardware players? Do they 'understand' this color space? Or PC.xxx color spaces should only be used when targetting PCs?

3. So far I got amazing results using
ConvertToYV24( matrix = "PC.709" )
+
x264 --input-range PC --output-csp i444

After opening a file encoded like this in MPC-HC + LAV results in visually really similar results, while only taking +10% more bitrate. Comparing this to RGB encoded h264 (--output-csp rgb) took 2.3x or + 130% bitrate!

So can I generally use YV24 encoded h264? Is it supported on say BluRay players?

New samples:
x264-mpc-rgb.png (https://dl.dropbox.com/s/y6d1y53jplgj5gr/x264-mpc-rgb.png)
x264-mpc-yv12.png (https://dl.dropbox.com/s/sk3rvkon2pn17pr/x264-mpc-yv12.png)
x264-mpc-yv24.png (https://dl.dropbox.com/s/m20865zhq7e66ij/x264-mpc-yv24.png)

[poisondeathray]

So can I generally use YV24 encoded h264? Is it supported on say BluRay players?


No standalone players, only computer playback

[sneaker_ger]

1. Do I generally always have to pass on the color information to the encoder? In the x264 wiki, I've read that it uses 'undef' by default. What is that? Is it x264 which 'hints' the decoder about what color profile to use, or it's totally up to the player?

It is recommended to explicitly store color informations into the stream, but many players only guess by resolution ( Rec601 TV for SD and Rec709 TV for HD) or are hard-coded to a single matrix.
You can store the matrices via:
--colormatrix (--colorprim --transfer)

So your best bet is to use Rec601 TV for SD and Rec709 TV for HD and on top of that store the color information explicitly in the stream.

How does say LAV filter decide which colorspace to use?

It first tries to read the info that is stored into the stream for H.264, but usually LAV outputs YUV to the renderer and does not do any color conversion. It's then up to the renderer to pick the correct colors and many only do resolution-based guessing - madVR being one notable exception. If you use LAV + madVR the information written into the stream will be used. If you deactivate everything but RGB output in LAV, it will do the conversion itself (not recommended for use with madVR).

2. What happens with PC.709 on hardware players? Do they 'understand' this color space? Or PC.xxx color spaces should only be used when targetting PCs?

They mostly do not understand it and even many software players do not expect it, so I wouldn't recommend to use this for any kind of distribution at all.

[vdcrim]

x264 defaults to TV range output, so all you're doing here is one superfluous conversion
It defaults to auto (same as input), which is what he wants.

[sneaker_ger]

Sorry, you are correct. I have misremembered it and will correct my post.

/edit:
Ah, I know why I had this misconception:
http://git.videolan.org/?p=x264.git;a=commit;h=e047b3c475cd42b6647397a244e239ebfca53bf6

[pancserzso]

OK, thanks for all the clarification. So the recommended settings are:

1. For wide-compatibility:
ConvertToYV12( matrix = "Rec709" )
+
--colorprim "bt709" --transfer "bt709" --colormatrix "bt709"

2. Only want to play back via PC, for example a computer at an exhibition, I can either use:

ConvertToRGB24()
+
--output-csp rgb, but it uses a _huge_ amount of data

or

ConvertToYV24 (matrix = "PC.709")
+
--input-range PC --output-csp i444 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" , which is more efficient.

Is this correct?

[pandy]

OK, thanks for all the clarification. So the recommended settings are:

1. For wide-compatibility:
ConvertToYV12( matrix = "Rec709" )
+
--colorprim "bt709" --transfer "bt709" --colormatrix "bt709"


For this it will be good to filter all above V/2 (all details smaller than 2 lines)


2. Only want to play back via PC, for example a computer at an exhibition, I can either use:

ConvertToRGB24()
+
--output-csp rgb, but it uses a _huge_ amount of data

or

ConvertToYV24 (matrix = "PC.709")
+
--input-range PC --output-csp i444 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" , which is more efficient.

Is this correct?

There is also decent YUY2 - almost free from YV12 problems and still providing high quality (perhaps except chroma key).
For sure more compatible than RGB or YV24.