How to correctly interpolate YUV?

[wonkey_monkey]

For years I wrote filters that only handled RGB clips, and left it up to users to convert in and out. These days I try to keep the original colourspace as much as I can, but I'm running into problems with interpolating pixel values in YUV.

For example, consider this 8x4 YV24 image:

https://i.imgur.com/NIdiIDS.png

The numbers in each pixel are Y, U, and V respectively.

If I then try to shear the image a bit, interpolating new values, I get this:

https://i.imgur.com/mB3U5L5.png

This is clearly visually wrong, because I've suddenly got white pixels. But it's mathematically correct - if Y U and V are treated independently - because all the pixels still have a Y value of 255, as before.

But really they should have a lower Y value. That white pixel on the top line should be around 180, taking gamma into account.

I'm pretty sure there's no good way to do this, other than to convert to RGB - which is fine for YV24, but I'd rather avoid resampling chroma for YV12.

Does anyone have any tips or insight into this problem that might avoid converting to RGB, and/or be able to handle colourspaces with subsampled chroma?

[StainlessS]

I think that the problem is maybe that eg 255,127,127 and 255,-128,128 are both illegal YUV colors.

EDIT: see here:- https://forum.doom9.org/showthread.php?t=154731

EDIT: Gavino YUV mp4 Animation(~480KB):- http://www.mediafire.com/file/g38sw8k6cl9chyt/Gavino_YUV.mp4/file

And Dark Shikari Gif
http://web.archive.org/web/20140315154448/http://img245.imageshack.us/img245/7905/coloryuv.gif

EDIT: Above are two different animations, (I originally thought that they were both the same, but not).

[wonkey_monkey]

I get the same problem with legal values (assuming colorbars outputs legal values?):

https://i.imgur.com/Zi40VxR.png

The transition line between magenta and green is too dark (this is with bilinear interpolation). The individual channels look okay, but once you put them together you get this (this is with gamma taken into account, approximately, for the Y channel).

Do I maybe need to somehow rescale chroma to a wider range, based on the value for Y at that pixel?

[shekh]

I get the same problem with legal values (assuming colorbars outputs legal values?):

https://i.imgur.com/Zi40VxR.png

The transition line between magenta and green is too dark (this is with bilinear interpolation). The individual channels look okay, but once you put them together you get this (this is with gamma taken into account, approximately, for the Y channel).

Do I maybe need to somehow rescale chroma to a wider range, based on the value for Y at that pixel?

These edges look like it is in linear color, you still need to convert it back to srgb.

Did similar experiment with vd2 filters, looks good to me.

https://i.postimg.cc/8zTkrXfZ/untitled2.png

[wonkey_monkey]

That's what I thought - there's no way to do it without converting to RGB, or something approximating it. To properly interpolate U and V you need to take Y into account as well, and to do that means upsampling to the same resolution, to start with.

Still, I expect in most cases, treating U and V as linear will be fine and no-one will see any problems.

[Emulgator]

I guess this is why serious compositing engines (Vegas etc) finally ended up doing exactly that:
Before anything: Decode -> 32bit floatconvert -> RGB 32 bit float.
Then any processing like resizing will be correct if kept in 32bit float, linear light, RGB.
Finally convert back to whatever user wants to see.
(User can opt to keep that process in 8 bit, gaining speed but losing any precision.)

[TheFluff]

The channels are not independent because of gamma. You don't strictly need a round trip to RGB, but you do need linear light. I think.

[wonkey_monkey]

That's what I thought. I'm just going to leave it to the user to convert to RGB if they want the best quality from my filter.