Artifacts resulting from YUY2 to YV12

[Seraphic-]

Hi,

While capturing different sources at 720p for trial encoding using x264, and at the same time testing 4:2:2 to 4:2:0, I came across an issue resulting from the step down.

Left side is 4:2:2 and the right is after going to 4:2:0.
As you can see, artifacts have been produced. While the artifacts can almost be seen in the raw, in the 4:2:0, they are now quite visible.
(I'm working with 3D rendered graphic games from Xbox360/PS3 - Not Anime)
http://temp.seraphicgate.com/problem.png

My first question would be, how can I avoid this kind of problem?

My second would be is there any codec that support retaining 4:2:2 or do all require the step down to 4:2:0? (x264/DivX/Xvid/WM9 etc)

[Dark Shikari]

This looks like a bad chroma conversion function rather than an actual problem with subsampling.

Of course, regardless of situation, either of them will end up getting bandy in x264.

Have you tried gradfun2db?

[Seraphic-]

No, I haven't tried gradfundbd as of yet, but I'll give it a try it tomorrow.
As for my encodes, they have been done using YUY2 to YV12 with VirtualDub.
If I was to using MeGUI and add converttoYV12, do you think I would have the same issue? I'll have to try that as well.
Also, just to be clear, everything I'm working with is uncompressed, nothing lossy.

When you say "bandy" when using x264, were you talking about if a low bit-rate was used? Once I do start 480p/720p production videos, I had planned on using HQ-Slowest or HQ-Insane with a bit-rate of 10,000 (or higher if needed).
Just would be great if I could retain the full/true color levels, sucks to have to lose 1/6 the color information to encode with x264 when going from 4:2:2 to 4:2:0.
That is why I was asking if any codecs even allow/accept 4:2:2 for encoding.

Thanks

[Dark Shikari]

Just would be great if I could retain the full/true color levels, sucks to have to lose 1/6 the color information to encode with x264 when going from 4:2:2 to 4:2:0.
That is why I was asking if any codecs even allow/accept 4:2:2 for encoding.
The problem you demonstrated is not caused by a lack of chroma information, at least not in the sense of chroma subsampling. Banding is caused by a lack of bit depth and dithering not being used to compensate for this; 4:2:0 has the same bit depth per sample as 4:2:2, so the fault likely lies with Virtualdub.

I don't know any modern video formats (at least high efficiency ones) that accept 4:2:2 other than H.264, and there are no cheap or free encoders that handle 4:2:2.

Banding is nearly unavoidable in x264 even at high bitrates simply because dither is nearly impossible to keep properly. There are some tricks though.

[bond]

moved as i understood its a vdub filter issue

[Seraphic-]

The problem you demonstrated is not caused by a lack of chroma information, at least not in the sense of chroma subsampling. Banding is caused by a lack of bit depth and dithering not being used to compensate for this; 4:2:0 has the same bit depth per sample as 4:2:2, so the fault likely lies with Virtualdub.

I don't know any modern video formats (at least high efficiency ones) that accept 4:2:2 other than H.264, and there are no cheap or free encoders that handle 4:2:2.

Banding is nearly unavoidable in x264 even at high bitrates simply because dither is nearly impossible to keep properly. There are some tricks though.

I tried convert to YV12 using MeGUI and it didn't look as bad. I think part of the problem might be that when the color information is removed, those areas are staying darker then the surrounding sections.

As for H.264 allowing for 4:2:2, are you talking about "High 4:2:2 Profile (Hi422P)"? Does any H.264 encoder support that? (like QuickTime for example)

What kind of tricks to reduce banding are you talking about?

Also, would x264 be recommended for 3D rendered graphic games from Xbox360/PS3 at 480p/720p? Or does it work better for movies/anime?

Thanks

[Dark Shikari]

As for H.264 allowing for 4:2:2, are you talking about "High 4:2:2 Profile (Hi422P)"? Does any H.264 encoder support that? (like QuickTime for example)The only one I know that supports it is Ateme's encoder (and maybe JM?).
What kind of tricks to reduce banding are you talking about?Gradfun2db, FGO, artificial grain, low deadzones, etc.
Also, would x264 be recommended for 3D rendered graphic games from Xbox360/PS3 at 480p/720p? Or does it work better for movies/anime?CGI/games/etc is what x264 tends to do best at, though it can handle pretty much any source.

[GodofaGap]

I just converted your png in VirtualDub 1.7.8, but I don't see the issue there. Perhaps you should inform the developer anyways.

[Seraphic-]

The only one I know that supports it is Ateme's encoder (and maybe JM?).
Gradfun2db, FGO, artificial grain, low deadzones, etc.
CGI/games/etc is what x264 tends to do best at, though it can handle pretty much any source.

What is the price of the Ateme's Kyrion File Encoder 2.0?
Don't see a price anywhere on the website.

[Dark Shikari]

What is the price of the Ateme's Kyrion File Encoder 2.0?
Don't see a price anywhere on the website.You don't see a price because these enterprise-class software/hardware encoders generally cost in the tens of thousands of dollars.

There's a reason x264 is very popular beyond its mere efficiency; it has basically no competition in its target market.

[Seraphic-]

You don't see a price because these enterprise-class software/hardware encoders generally cost in the tens of thousands of dollars.

There's a reason x264 is very popular beyond its mere efficiency; it has basically no competition in its target market.

I knew the price would be up there, but was looking to get a laugh at the actual cost. Anyway, is "High 4:2:2 Profile (Hi422P)" something that could be added to x264 in the future?

I just converted your png in VirtualDub 1.7.8, but I don't see the issue there. Perhaps you should inform the developer anyways.

Would it be the same for an image as for a video? Anyway, using 1.7.8 as well, all I did was set color depth to YV12 and compression to uncompressed. Is that what you did?

[Dark Shikari]

I knew the price would be up there, but was looking to get a laugh at the actual cost. Anyway, is "High 4:2:2 Profile (Hi422P)" something that could be added to x264 in the future?Yes, there was a GSOC project proposal to add 4:2:2 and 4:4:4 color support, though we didn't have enough slots to accept it.

If anyone wants to work on the project, I'd be happy to assist; the primary things that would need to be done are:

1. Find all places where we assume chroma blocks are width8 and luma are width16 and change them to variables which are set at the start based on chroma mode.

2. Handle the bitstream syntax (relatively simple).

3. Change the DCT and hadamard transform method to match different-size chroma blocks.

[Seraphic-]

Yes, there was a GSOC project proposal to add 4:2:2 and 4:4:4 color support, though we didn't have enough slots to accept it.

If anyone wants to work on the project, I'd be happy to assist; the primary things that would need to be done are:

1. Find all places where we assume chroma blocks are width8 and luma are width16 and change them to variables which are set at the start based on chroma mode.

2. Handle the bitstream syntax (relatively simple).

3. Change the DCT and hadamard transform method to match different-size chroma blocks.

That would be awesome, would make x264 "Duke of New York, A-Number-1" :)

You should post a topic in the "MPEG-4 AVC" forum titled something like "The x264 4:2:2 and 4:4:4 color support project". See how many people would offer their support, maybe other x264 Developers will offer to help. No-one will see your idea since this topic was moved to this forum.

Sadly, I don't think I would be much help in the technical area of this, as I haven't worked with x264 much.

[Dark Shikari]

That would be awesome, would make x264 "Duke of New York, A-Number-1" :)

Maybe you should post a topic in the "MPEG-4 AVC" forum titled something like "The x264 4:2:2 and 4:4:4 color support project". See how many people would offer their support, maybe other x264 Developers will offer to help.Support is pretty useless; there are only three ways to make such a thing happen:

1. Convince an existing developer to do it.
2. Learn x264's code and do it yourself.
3. Offer a couple thousand euros for the feature's completion (corollary of option 1).

Unfortunately I'm basically booked for the summer and Avail isn't really interested in alternate colorspace support... and another issue is that ffmpeg doesn't support it for decoding, either.

[Seraphic-]

Support is pretty useless; there are only three ways to make such a thing happen:

1. Convince an existing developer to do it.
2. Learn x264's code and do it yourself.
3. Offer a couple thousand euros for the feature's completion (corollary of option 1).

Unfortunately I'm basically booked for the summer and Avail isn't really interested in alternate colorspace support... and another issue is that ffmpeg doesn't support it for decoding, either.

You would think being able to retain the true color levels of your source would be a big issue for most people.
As someone who has the hardware to capture uncompressed using HDMI at 4:2:2, it really is unfortunate to have to step down to 4:2:0, as I'm trying to achieve the highest quality videos possible.

I'm interested, as a x264 developer, is 4:2:2 support something you could do on your own if you wanted? Or would you need the support of other x264 developers?
I mean, is it a major understand or something that can be done, but just after with a fair amount of work?

WMV9 goes into some detail about 4:2:2 support
Capture to a YUY2 (4:2:2) pixel format to avoid color conversion problems during encoding. The Windows Media Video 9 codec uses primarily a 4:2:0 pixel format; however, if you choose to maintain the interlacing in your content (a new feature available with Windows Media Encoder 9 Series), then the codec uses a 4:1:1 pixel format internally. By capturing to a YUY2 pixel format, you can encode either progressive or interlaced content, because 4:2:2 is a superset of both the 4:2:0 and 4:1:1 data structures. It is important to note that if you capture to an .avi file with a 4:2:0 pixel format (for example I420, YV12, or IYUV), you will not be able to maintain the interlacing in the video because the 4:2:0 pixel formats do not carry chroma information on the alternate lines that contain the data for the second field. If you are working with progressive-scan source video or if you are working with interlaced source video that you plan to deinterlace, then a 4:2:0 pixel format is appropriate because the output file is smaller than an .avi file with a 4:2:2 pixel format.

[Dark Shikari]

You would think being able to retain the true color levels of your source would be a big issue for most people.
As someone who has the hardware to capture uncompressed using HDMI at 4:2:2, it really is unfortunate to have to step down to 4:2:0, as I'm trying to achieve the highest quality videos possible.I can't visually tell the difference between the two except in 8-bit/similar video game capture, so its very hard to motivate myself to want to do it. On the other hand, I actually do care (http://mirror05.x264.nl/Dark/Flash/index_lowbitrate.html) about such low-resolution video game capture, unlike most codec developers. Its also on my list of "Things Ateme Has That Are Better Than x264." Unfortunately, there are other things on this list I have to fix first (frametype decision, MBAFF, etc).
I'm interested, as a x264 developer, is 4:2:2 support something you could do on your own if you wanted? Or would you need the support of other x264 developers?
I mean, is it a major understand or something that can be done, but just after with a fair amount of work?I would probably need some assistance from akupenguin in the bitstream syntax, but certainly he'd be willing to provide it.

[GodofaGap]

Would it be the same for an image as for a video? Anyway, using 1.7.8 as well, all I did was set color depth to YV12 and compression to uncompressed. Is that what you did?
Yes, that's what I did.

[Seraphic-]

I can't visually tell the difference between the two except in 8-bit/similar video game capture, so its very hard to motivate myself to want to do it.
I would probably need some assistance from akupenguin in the bitstream syntax, but certainly he'd be willing to provide it.

In this case, it is 4:2:2 at 8-Bit, I could have gotten a card that supported 4:2:2 at 10-Bit, but since it didn't support 1080p, I could not justify the extra cost. However, after talking to a person at BlackMagicDesign, he said in the near future they will be moving to 4:4:4/10-Bit/720p/1080p. So I'm looking forward to that. But now that I think of it, maybe not, since no normal consumer level codecs support 4:4:4 or even 4:2:2.

I've done several dozen high-bit-rate test encodes with x264 an I can see a clear color saturation difference between the raw and the x264 encode. Color looks washed out.

Maybe I can start some kind of donation box in the "MPEG-4 AVC" forum. People could donate like $5 or $10 toward the undertaking. I know I would donate some to help the project along.

Yes, that's what I did.

Must be different for video then it is for image if we both did the same thing.

[Dark Shikari]

I've done several dozen high-bit-rate test encodes with x264 an I can see a clear color saturation difference between the raw and the x264 encode. Color looks washed out.Again, washed-out color has nothing to do with chroma subsampling; it has everything to do with improper color conversion. This means, for example, using BT.601 instead of BT.709 for color conversion (or vice versa), or using a bad color conversion method that uses RGB as an intermediate.

4:2:2 has the exact same range of color representation as 4:2:0 and uses the exact same conversion coefficients.

[GodofaGap]

Must be different for video then it is for image if we both did the same thing.
I don't see any reason why... the conversion method remains the same.

[Seraphic-]

Again, washed-out color has nothing to do with chroma subsampling; it has everything to do with improper color conversion. This means, for example, using BT.601 instead of BT.709 for color conversion (or vice versa), or using a bad color conversion method that uses RGB as an intermediate.

4:2:2 has the exact same range of color representation as 4:2:0 and uses the exact same conversion coefficients.

Isn't it correct that when moving from 4:2:2 to 4:2:0, it means loosing 1/6th of the color information?

As for improper color conversion, are you talking about an error on my part or an issue with the software I'm using?
I mean, a few times I've done a capture at 4:2:2 and then moved right into x264 encoding after adding converttoyv12.
Other times I've done the color space conversion using VirtualDub.
These were just for testing, but are both those improper methods?

I don't see any reason why... the conversion method remains the same.

What did you use for the decompression format?

[Dark Shikari]

Isn't it correct that when moving from 4:2:2 to 4:2:0, it means loosing 1/6th of the color information?All it means is that chroma is stored for every 4 pixels instead of every 2 pixels. That's half the chroma information.

But "washed out" color and banding cannot be caused by this; the only possible result is lost detail, which is pretty unlikely since chroma almost never has much detail anyways.
As for improper color conversion, are you talking about an error on my part or an issue with the software I'm using?
I mean, a few times I've done a capture at 4:2:2 and then moved right into x264 encoding after adding converttoyv12.
Other times I've done the color space conversion using VirtualDub.
These were just for testing, but are both those improper methods?ConvertToYV12() in Avisynth should work just fine if you're not doing an RGB intermediate.

[GodofaGap]

What did you use for the decompression format?
Autoselect.


A little OT:

I'd like to say that YV12 has become a monstrosity of a format. I believe it was originally a 4:2:0 rec.601 format, but somewhere along the line people added other stuff to it out of convenience (interlacing, other color codings). That's understandable when you need interlaced 4:2:0 or rec.709, but it defies the point of having a format descriptor. A program can't really tell by anything which 'version' of YV12 it is dealing with. It just has to rely on the user to do it correctly, but most users are not even aware there is such a thing as rec.601 and rec.709 and only see the washed out colors.

[foxyshadis]

or worse, "pc.601" and "pc.709", which is really where the vast majority of "washed out colors" complaints come from. 601<->709 only results in slightly redder or greener colors.