Convert from YV12 to Lab

[MysteryX]

In an attempt to translate SuperRes shader to AviSynth, it deals with the Lab color space. That's not supported in AviSynth and I'm not finding anything that converts the color space to Lab.

Can AviSynth support Lab?

Has anything been written to convert from YV12 to Lab and from Lab to RGB32?

Where can I find the source code of ConvertToRGB32?

Anything else I should know about this?

[MysteryX]

OK I downloaded AviSynth's source code and found the code of ConvertToRGB32.

And even if Lab isn't supported, it can be stored as a 'fake' RGB32. It won't work for display, but it can then interpreted as such.

[feisty2]

Lab is YUV like (L--Y, ab--UV), not RGB like
and you don't want CIE stuff with precision less than float, which is something avisynth not good at

[MysteryX]

Originally Posted by feisty2

Lab is YUV like (L--Y, ab--UV), not RGB like

Could you expand on this?

Originally Posted by feisty2

and you don't want CIE stuff with precision less than float, which is something avisynth not good at

Oh. YUV is 4 bytes per 2 pixels, while Lab is 3 float (3x4), 12 bytes per pixel? Can this still be done?

Parsing the data back and forth to float in c++ can be done. The question is, can AviSynth store all that data in its Clip objects?

If not, then another option would be to code the whole thing within C++ and use internal buffers without passing Lab frame data back.

[MysteryX]

I need to have float4 for each pixel in Lab. I found this in AviSynth's source code to convert YUV data into into RGB. Then I can use some other code to convert from RGB to Lab. But... what is this doing? YUV stores the data of 2 pixels at once, so ideally I would return float4x2 as RGB and then convert those two pixels to Lab. The last float will contain some other data.

This code doesn't seem to be doing exactly that... and then what about color matrix?

inline void YUV2RGB2(int y, int u0, int u1, int v0, int v1, BYTE* out)
{
const int crv = int(1.596*32768+0.5);
const int cgv = int(0.813*32768+0.5);
const int cgu = int(0.391*32768+0.5);
const int cbu = int(2.018*32768+0.5);

const int scaled_y = (y - 16) * int((255.0/219.0)*65536+0.5);

out[0] = ScaledPixelClip(scaled_y + (u0+u1-256) * cbu); // blue
out[1] = ScaledPixelClip(scaled_y - (u0+u1-256) * cgu - (v0+v1-256) * cgv); // green
out[2] = ScaledPixelClip(scaled_y + (v0+v1-256) * crv); // red
}


How can I loop through the YV12 source frame data, extract each pairs of pixels, and convert them into RGB?

Or can I simply call ConvertToRGB function within my code? It expect a clip but what I have is a frame, so how can I call it?

[foxyshadis]

Use 3 RGBA clips -- where each of RGBA is actually a 32-bit float, doubly handy since the interleaved nature means fast, easy memory access. The three clips should probably be stacked into one for convenience. Tada, new colorspace, just like 16-bit stacked. Fizick did this with MVTools, the output of MSuper is a clip that is really just a bunch of floating-point values.

To read and write, you should be able to cast GetWritePtr to float and work with it fairly easily. Just create convenience functions to separate the stack into each channel.

Don't use ConvertToRGB, you lose precision with each conversion. Use DitherTools to work in 16-bit or just convert directly from YUV to float RGB to float LAB.

[MysteryX]

Has someone already written code that convert YUV into stacked RGBA, and from stacked RGBA into YUV?

[jackoneill]

Originally Posted by foxyshadis

Tada, new colorspace, just like 16-bit stacked. Fizick did this with MVTools, the output of MSuper is a clip that is really just a bunch of floating-point values.

From what I have seen, the output of MSuper is simply several resized versions of the input, still 8 bits per sample.

[MysteryX]

Hah. Everybody has a different idea about a detail like this.

I think Shader support should be part of AviSynth's core, and would allow for GPU-support. And this should be done "right" by a master AviSynth developer. It shouldn't be difficult, but if it is to be done, and if it is to become a core component, it should be done right.

[MysteryX]

In EEDI3's code (the only one that compiled out-of-the-box for me), it has planar functions like convYUY2to422 and conv444toRGB24 that are highly optimized.

What would be needed is a similar function but that converts from YUY2 to 444 as float instead of int. Then it would be easy to loop through the pixels and convert the float values from RGB to Lab one by one.

Float data may be necessary for doing any kind of work with shaders.

[MysteryX]

The conversation is kind of split in-between this thread and the Shaders thread. Anyway.

This can *definitely* be done.
http://www.avisynth.nl/index.php/Fil...nge_frame_size

All you need is to resize your frame buffer. Basically, to go from pixels of 2 bytes to pixels of 16 bytes, you need to create a new frame with the width * 8.

Then it's up to you how you store the data. Make one pass through the source video to copy YUY2 to float RGB 2 pixels at a time. Then make a second pass in that same frame to replace each RGB pixel data with Lab pixel data. Voila!

The only downside is that AviSynth won't recognize that frame type and it won't be a valid output, so you just need to know it's in that format and convert it back into a valid display format afterwards.

This can definitely be done, calling shaders can definitely be done, and passing float RGBA (or float Lab) data from one filter to another in a clip can also be done. It just needs to be put together.

I believe planar is mostly useful when working with individual planes without needing the other planes? The work in shaders is done pixel per pixel so I don't think planes should be used here.

[MysteryX]

Before talking about Lab, we first have to look at converting YUV to float RGB back and forth.

Wikipedia says this:

Since the equations defining YCbCr are formed in a way that rotates the entire nominal RGB color cube and scales it to fit within a (larger) YCbCr color cube, there are some points within the YCbCr color cube that cannot be represented in the corresponding RGB domain (at least not within the nominal RGB range). This causes some difficulty in determining how to correctly interpret and display some YCbCr signals. These out-of-range YCbCr values are used by xvYCC to encode colors outside the BT.709 gamut.

How good are these equations found in ConvertToRgb and ConverToYV12? I believe these are using Rec601 instead of Rec709. Converting to RGB is one thing, and how expensive will be the conversion from RGB to YUV for running on every single pixel? Will converting back and forth cause a loss of data even if using float? What happens to YUV data that, according to Wikipedia, has no RGB representation?

inline void YUV2RGB(int y, int u, int v, BYTE* out)
{
const int crv = int(1.596*65536+0.5);
const int cgv = int(0.813*65536+0.5);
const int cgu = int(0.391*65536+0.5);
const int cbu = int(2.018*65536+0.5);

int scaled_y = (y - 16) * int((255.0/219.0)*65536+0.5);

out[0] = ScaledPixelClip(scaled_y + (u-128) * cbu); // blue
out[1] = ScaledPixelClip(scaled_y - (u-128) * cgu - (v-128) * cgv); // green
out[2] = ScaledPixelClip(scaled_y + (v-128) * crv); // red
}

inline int RGB2YUV(int rgb)
{
const int cyb = int(0.114*219/255*65536+0.5);
const int cyg = int(0.587*219/255*65536+0.5);
const int cyr = int(0.299*219/255*65536+0.5);

// y can't overflow
int y = (cyb*(rgb&255) + cyg*((rgb>>8)&255) + cyr*((rgb>>16)&255) + 0x108000) >> 16;
int scaled_y = (y - 16) * int(255.0/219.0*65536+0.5);
int b_y = ((rgb&255) << 16) - scaled_y;
int u = ScaledPixelClip((b_y >> 10) * int(1/2.018*1024+0.5) + 0x800000);
int r_y = (rgb & 0xFF0000) - scaled_y;
int v = ScaledPixelClip((r_y >> 10) * int(1/1.596*1024+0.5) + 0x800000);
return ((y*256+u)*256+v) | (rgb & 0xff000000);
}

[MysteryX]

This is the best answer about color conversion
http://www.fourcc.org/fccyvrgb.php