RGB vs. YV12

[Christo]

Everyone knows 24bit is better than 16bit or 12 bit. So what is the point other than the encoding speed increase to use YUY2 or YV12 instead of RGB in AVISynth?

Most DVDs use YV12 to store data. So to do a strait conversion into another YV12 format seems like a good idea. But how many of us do it without appling any filters first?

The filters work the best in RGB24 or RGB32 and defiantly not in 12bit colorspace!

However many filters for AVISynth are designed to work only in YUY2 or YV12. That makes their potential somewhat limited. It seems like a majority cares more about the speed than a quality and that is very disappointing…

[Zarxrax]

No, by staying in YV12 mode, you retain quality. When you go and convert to RGB and then back to YV12, you lose quality.

[-h]

Even though there is more colour information in RGB space, that colour information was invented from the decimated YV12 data.

Filtering invented data is pointless when you can just process the raw data it came from, and you'll get faster and more accurate results too (since you avoid rounding errors from YV12->RGB->YV12 conversions).

-h

[Christo]

Good answers!
But before someone starts using AVISynth 2.50 he needs to be sure his source colorspace format is YV12 and nothing else. My question is do all DVDS use YV12? Thank you in advance.

[Christo]

p.s. I'm asking because maybe MPEG2DEC.dll simply convert to YV12 from something else

[trbarry]

I think MPEG2 supports some sort of 4:2:2 and even 4:4:4 studio profiles but don't believe they are ever used on commercial DVD's.

- Tom

[Trahald]

afaik dvd spec only allows for 4:2:0

[fisix]

and, just to be clear, most capture cards capture in something other than yv12, yes?

-f

[trbarry]

I think most capture cards default to YUY2.

- Tom

[scmccarthy]

My question is do all DVDS use YV12?

The answer is yes always, only YV12, except, to get technical, fourcc codes don't belong in any MPEG spec, they are header codes that tell applications how to play files that are only container files like avi files. It is supported by Microsoft to tell the operating system which codecs to apply. This explains why you hardly ever see DVD described as using YV12. All fourcc is is four bytes in a header to tell how to interpret the data, not the data itself.

Instead, this is how I put the question:

Does DVD always use 12-bit 4:2:0 YCbCr with luma defined by Y = .299R + .587G + .114B?

Yes.

Searching the DVDFAQ, I found some quotes:

Toshiba's "Super Digital Progressive" players and the Panasonic HD-1000 use 4:4:4 chroma oversampling, which provides a slight quality boost from DVD's native 4:2:0 format.

Picture dimensions are max 720x480 (for 525/60 NTSC display) or 720x576 (for 625/50 PAL/SECAM display). Pictures are subsampled from 4:2:2 ITU-R BT.601 down to 4:2:0, allocating an average of 12 bits/pixel in Y'CbCr format. (Color depth is 24 bits, since color samples are shared across 4 pixels.)

Now that I think about it, do you suppose a DVD decoder would need a fourcc code to tell it how to decode the video? Of course not, so there may not be anything like that in the header, which would explain the lack of references to it.

Tom - DVD-Video is only a subset of MPEG-2. You know that.

Many users are asking the same question: Why is mpeg2dec converting to the YV12 color space now. We never thought about what mpeg2dec did before. Many users think that mpeg2dec never did a color space conversion before and is doing one now; just the opposite is true. MPEG2DEC3 can still convert to YUY2 with the right parameters and it is important to keep in mind that that is still an option, but it is AviSynth that is changing so that no color space conversion is nesessary for DVD.

Stephen

[trbarry]

Stephen -

I think you are probably correct. Many people are wondering if MPEG2DECx is now down-converting from YUY2 to YV12. This is of course not the case.

So if there is any doubt for anyone, DVD's and HDTV are already in a YV12 (4:2:0) color format. All we are doing now is putting off the conversion to YUY2 or RGB as long as possible. And hopefully avoiding it alltogether.

Since 4:2:2 (YUY2) and 4:2:0 (YV12) don't have the same number of color samples or even have any from matching sample points there is an unavoidable blurring that takes place when you convert from one to the other.

In most cases I believe it is best to avoid these conversions, though you probably could come up with (more) counter examples where they would be needed.

And of course you can still use YUY2 if you want anyway. It is not gone. But even if you are going to convert to YUY2 it is probably better to IVTC or deinterlace first.

- Tom

[scmccarthy]

Tom - Blurring is a good word for it, except that only the color information gets blurred and the eye is not very sensitive to color information, so the old YUY2 conversion was not so bad. Conversion to RGB is different, because it involves clipping as well. Going from YUV to RGB and back to YUV is very lossy. It is better if that does not happen until it is displayed on the screen.

Stephen

[morsa]

Is there any filter that works in this colorspace?
Is the conversion between RGB and YUV 4:4:4 lossy?

This questions came to me because I'm upsizing some Standard resolution material in 4:2:0 and I noticed that the colors are always shifted in some parts.( I mean some parts have an incorrect color, you can notice big square blocks of color)
I made a test by hand doing this:
I get data in 4:2:0 luma 720x576, red 360x288, blue 360x288.
Then I upsize the two color layers to 720x576 using Lanczos3.
Now I've got three layers, luma,red,blue, with 720x576.
Then I upsample to 1600x1200 in RGB, and guess what, The color problems dissapeared.

Is there a way to make this with Avisynth or Vdub?

[scmccarthy]

No, there is not YUV 4:4:4 format supported.

Yes, YUV 4:4:4 is still lossy.

In fact, this is already being done. ConvertYV12ToRGB is done in several steps.

4:2:0 --> 4:2:2 --> 4:4:4 --> RGB.

That is how it is done. YUV formats are done to allow compression of the color information. YUV 4:4:4 is rare, because it takes the same amount of space as RGB. It is used internally in studio to process video data; it is just not ever produced in that form. There is no fourcc for it and it cannot be played in Windows.

There is a problem here that needs to be addressed, but I am not sure I know what you are asking.

Stephen

[morsa]

But, why is it lossy?
What happens if I convert RGB to YUV 4:4:4 and again to RGB?

If don't understand please ask me.

[scmccarthy]

To convert YV12 to RGB, first convert YV12 to YUV 4:4:4. This is done by interpolating the chroma horizontally and vertically. Then convert the YUV to RGB. Throw away values for R, G, and B that are greater than 255 of less than zero. You'll never see those values again. If you did the conversion right, it won't matter, just don't try to convert back to a YUV format.

AviSynth already has a filter that does this, so you are alright there. It is possible for the interpolation to be done wrong, but not to the extent you mentioned. Something else must be causing it.

Generally, it is much clearer if you keep your description of the problem separate from your speculation on what is causing the problem. You've drawn me into a discussion of color spaces and why that is not the problem rather than discussing the problem itself.

Stephen

[scmccarthy]

@morsa

OK, so let me get back to the question. Quality wise every color space conversion is bad. Speed wise every color space conversion obviously takes time.

It you have a file in 4:2:0 either apply filters in that color space or convert to rgb. Once you make that conversion, do not convert back to any YUV color space, this means don't encode in any mpeg format whether mpeg2 or mpeg4 like cce or TMPG or divx or xvid.

In order to get to 4:4:4 from 4:2:0, you have to interpolate and interpolation is nesessary but imperfect. If you captured something dircetly in YUV 4:4:4, yes! the quality would be fantastic. Everyone agrees, however, that the difference would not be worth the sacrifice of storage capacity. That is why it does not exist except possibly in-house at Pixar or the like. Bottom line, the changes in AviSynth are being done for both speed and quality, period.

A lot of people are asking why are we sacrificing quality for speed.

We're not!

The only complication with the YUV color space is whether or not chroma samples should be treated diferently than luma samples. Filters like MergeChroma() allow us to filter them separately if we want.

Stephen

[morsa]

My post was a pair of questions about colorspaces and another question about the existence or not of a filter which could be used to perform the task I've described.

If I'm not terribly wrong, when I open, for example a DV PAL format the two color layers interpolation is just a nearest neighborhood.
That's what I see when I resample to 1600x1200, big color blocks that don't match exactly the luma.
But if you tried what I've described about interpolating you would see what I'm talking about. If you resample the two color layers with Lanczos, bicubic, etc ,better than nearest neighborhood the differece becomes noticeable.The colors match almost perfect luma.

That was the cause of my questions about availability of that kind of filter.

[scmccarthy]

@morsa

1) Post a frame that shows the color block problem. I cannot visualize it.

2) Are you using AviSynth 2.5 alpha?

Stephen

[Richard Berg]

What follows is my interpretation of what he's asking. Correct me if I'm wrong.

Is there a ConvertYV12ToRGB function that doesn't use a box filter (aka "nearest neighbor" resampling)? According to his experiments, doing the chroma upsampling "by hand" with more advanced filters yields better results.

For example, he claims that the output of the following two pseudo-scripts should be identical or comparable, but in reality isn't:

# method one: use built-in functions for manipulating YV12
src = AviSource("some_YV12_file.avi")
method1 = src.ConvertYV12ToRGB

# method two: upsample manually
chroma_u = src.GetChannelU.LanczosResize(src.width, src.height)
chroma_v = src.GetChannelV.LanczosResize(src.width, src.height)
method2 = MergeChromaPlanar(src, chroma_u, chroma_v).ConvertYUV24ToRGB


I haven't been paying attention to the coding of YV12 stuff so I'm not sure if he's right, but it deserves a look.

[morsa]

Exactly!!!
That is what I was saying.

[Cyberia]

Im still not convinced its better than RGB32. I know the 32 RGB bits were created from the 12 YUV bits, but once that conversion is complete, we should have much greater accuracy from that point on.

In fact it reminds me of how the Radeon 9700s use 128 bit color internally, and scale it down for output. This greatly improves the output quality.

Am I misunderstanding?

[trbarry]

I don't know the correct terminology for it but don't confuse the number of bits in a color sample with the number of color samples.

When you talk about the Radeon card (is it really 128 bits?) you are talking about the precision of the 3 color components at one point on the screen. RGB has 3 color components and in Avisynth each has 8 bit precision. Also in Avisynth YV12 has 3 color components YUV, also with 8 bits each. But there are only 1/4 as many U and V samples so there is less spatial resolution for chroma.

If you are going to be eventually converting to RGB anyway then it may be worthwhile to do it sooner, if slower. But if you are going to pass YV12 to the compressor I don't think there is much to be gained by keeping the extra spatial accuracy only to throw it away on conversion. And indeed, multiple unneeded conversion will lose accuracy. The author of Avisynth, Ben RG, posted an example of this with his original write-up somewhere around here.

But experiments may prove differently. YMMV.

- Tom

[Kyo]

@Cyberia

Yes you have more accuracy but in the wrong side, since the convertion YV12->RGB will *destroy* some part of the original information, i.e. if in YV12 you have only 12 bits for a pixel in RGB you have 32 for the same pixel but the data that this pixel represents have less information about the original pixel, so this way the calculate it is less accurate.

[scmccarthy]

What TRBarry said explains it the best so far. Every sample has 8-bit accuracy for all the color spaces we are talking about. YV12 does not really have 12 bits per pixel, only an average of 12 bits per pixel. In fact, a person could even write a filter that converts YV12 --> RGB32 called ConvertYV12ToRGBReduceBy2() that reduces both the horizontal and vertical resolution by 2 to avoid interpolation.;) Such a filter would have limited usefulness and might confuse people even though it could illustrate the inter-relation between color spaces.:rolleyes:

'Average 12-bit?' you say. Yes,
1) Y always gets 8-pits per pixel in every YUV color space.
2)The two 8-bit chroma samples are cosited, adding up to 16-bits.
3)The 16-bits worth of chorma samples (two each 8-bits) are sampled from the original for every other pixel on each line between every two lines.
4)So the two 8-bit chorma samples are shared between 4 8-bit Y samples, or more accurately yet, they are shared between two 8-bit Y samples and two sets of half 8-bit Y samples.
It comes out to an average of 12 bits per pixel either way.

The pixels with no color infomation for themselves are given the average of the color information of the pixels next to then; this is interpolation. This process gives a very accurate representation of the original color, but once you do it you do not want to subsample back down again to YV12 if you can avoid it. Now you can avoid it with Avisynth 2.5 alpha. (Soon to be beta?).

The above explanation will make more sense to you if you read the YV12 FAQ and follow the link from there to Microsofts MSDN 'Video Rendering with 8-Bit YUV Formats' page. I started reading that just before I learned that AviSynth 2.5 was being developed. It has been fun putting that theory into practice and seeing that it really works the way it says on that page.

This is the long version of what Tom already said about chroma only having 1/4 the resolution of the luma. YV12 color depth is equal to RGB32 even though the resolution is not. Iterpolation is analogous to resizing up, but there are siting issues that make it different. Just as enlarging does not really give more picture information, converting to RGB will not increase color information.

Finally, if UV subsampling wasn't high quality, all DVDs would look like crap and they don't.

Are HDTV broadcasts in YV12 Tom?

Stephen

[sh0dan]

Originally posted by Richard Berg


# method one: use built-in functions for manipulating YV12
src = AviSource("some_YV12_file.avi")
method1 = src.ConvertYV12ToRGB

# method two: upsample manually
chroma_u = src.GetChannelU.LanczosResize(src.width, src.height)
chroma_v = src.GetChannelV.LanczosResize(src.width, src.height)
method2 = MergeChromaPlanar(src, chroma_u, chroma_v).ConvertYUV24ToRGB



Which translates into reallife as:


# method one: use built-in functions for manipulating YV12
src = AviSource("image.avi")
method1 = src.ConvertToRGB()

# method two: upsample manually
chroma_u = src.utoy().LanczosResize(src.width, src.height)
chroma_v = src.vtoy().LanczosResize(src.width, src.height)
method2 = ytouv(chroma_u,chroma_v)
mergechroma(src.pointresize(src.width*2,src.height*2),method2)
converttorgb()
method2 = pointresize(src.width,src.height)

return stackvertical(method1,method2)


I cannot see any visible difference at all. Subtract doesn't show any changes, and compare has very small differences. Chroma _is_ interpolated in converttorgb() already.

In general, you shouldn't be concerned about using YV12. ALL MPEG (1,2 & 4) files are 4:2:0, so there is 95% probability that your destination uses YV12. If you require video for more editing (compositing, etc), you can use RGB32. All your DVD's and DivX'es are in YV12 - how visible it that?

Instead you should experiment on further reducing chroma accuracy. When compressing to XVID it actually both helps compression and visual quality if you softly blur the chroma channel:
Mpeg2source("gammel_sm0lf.d2v")
mergechroma(blur(1.0))
This will in many cases give better compression, _and_ better looking results. Try it. Perhaps even with blur(1.5).

@Kyo: In principle yes, so if your source has better accuracy (RGB32), and you _need_ the better chroma accuracy (for chroma keying for instance), keep it in RGB. If your destination changes it to YV12 it doesn't matter at all.

@Cyberia: Don't where you got the 128bit from - I cannot find and haven't heard anything about it. The 3D-part is 100% float point calculations, and is then transformed before it is displayed - this is not done for video. It does have some kind of post-processing for real-video that smooths out blocks.

@All: Creating support for 4:4:4 15/16bpp YUV later would IMO be a better option.

[scmccarthy]

@Sh0dan

@All: Creating support for 4:4:4 15/16bpp YUV later would IMO be a better option.

Is there a 4:4:4 YUV color space after all? Or do you mean you can make a color space that only AviSynth filters can pass to each other, but must not be output at the bottom of the Script? Would it have to be planar?

Thanx

Oh, and I've been meaning to ask-comment:p :

After an interuption, I started using AviSynth 2.5 the way I was using the previous versions of AviSynth. When are you going to upgrade it to BETA?

Stephen

[sh0dan]

Is there a 4:4:4 YUV color space after all? Or do you mean you can make a color space that only AviSynth filters can pass to each other, but must not be output at the bottom of the Script? Would it have to be planar?

No, there is no such format (I know of), so it will be an internal format only - but it's still way out in the future - we're talking v3.0, or something similar.
It will most probably be planar - planar is much easier to work with compared to interleaved.

[morsa]

@Shodan :I-m not finishing these images in YV12, but in RGB24 at a resolution of 4096 x 3500.The bigger the image the bigger the difference upon how you have resampled color layers.

[trbarry]

Is there a 4:4:4 YUV color space after all? Or do you mean you can make a color space that only AviSynth filters can pass to each other, but must not be output at the bottom of the Script? Would it have to be planar?

scmccarthy -

I think that 4:4:4 "Studio Profiles" were added to the mpeg2 spec but I don't know of them being used for anything.

I've also heard that 4:2:2 is also allowed for HDTV but I'm not sure I believe it. Everything I've seen has been 4:2:0.

- Tom

[morsa]

In fact the colorsampling scheme for HDTV is called 22:11:11.

[trbarry]

In fact the colorsampling scheme for HDTV is called 22:11:11

Huh?

I've never heard of that before.

And DVD2AVI can decode it just fine thinking it is 4:2:0. Do you mean ATSC in the USA? Or is that specific to the Argentina standard?

- Tom

[neuron2]

Tom, do a Google search for "22:11:11 HDTV" and you will have heard of it. :)

[trbarry]

Tom, do a Google search for "22:11:11 HDTV" and you will have heard of it.

Interesting, I just did so.

It looks like it is just a 4:2:2 format with higher resolution. It was talked about for HDTV but when they went to MPEG2 it seems they also chose 4:2:0, and in the ATSC documents don't use 22:11:* terminology at all anymore.

- Tom

[Cyberia]

@trbarry and shodan

Yup both the Radeon 9700 and GeForceFX use 128-bit color internally and sale it down for output. This increases accuracy when they end up applying a zillion textures/transformations to the pixels. Similar to us applying filters to video. There is even a quote by Carmack himself about it reducing color banding. I can't find that quote right now.

@trbarry
I agree with you... If you are going to goto RGB anyway, do it early. Otherwise stay YV12.

ATI's info
http://mirror.ati.com/products/pc/radeon9700pro/index.html

Nvidia's Info
http://www.nvidia.com/view.asp?IO=feature_precision

[morsa]

TV broadcast standard in Argentina is a variant of PAL format called PAL-N (like Brazil with its PAL-M).
They differ only in the color frequency carrier.
HDTV standard is the same as in USA, although it is not working yet.
I've heard something about going Mpeg-4 these days, but I don't really know.
By the way if anyone is interested, I've been working with Cinealta cameras for some time, the quality is excellent and the manual says 22:11:11. May be because of its 7:1 DCT compression, variant of DVCAM ?


@Cyberia:
Do these cards display color (on a monitor) with a bit depth precision better than 8 bit, I mean 10, 12, 16 bpp?

[Cyberia]

For both cards, the maximum output bit depth per pixel (bpp) is 32bpp.

That is, 8 Blue bits, 8 Red, 8 Green, 8 Alpha.

[morsa]

So the only card with higher resolution than 8 bit is still Parhelia?

BTW a new discussion about making YUV2 to RGB via LUT's has begun at Vdub's unnoficial forum.

[Prosper]

the only time you really need 128 bit accuracy is when you're adding/multiplying/manipulating layer after layer of alpha-blended images. For a filter such as a convolution or blur, the additional accuracy is of very limited (if any) benefeit