Process chroma and luma together in an avisynth plugin

[Ceppo]

I'm making a filter that works on a block size of 4x4 on luma thus a 2x2 on chroma for YV12 and I need to process them together. Now the block size difference is giving me a headache. So is there some colorspace I can use that has luma and chroma in one channel so it is easier?

[StainlessS]

YUY2 ?, 4 bytes interleaved, YUYV, 1 Chroma sample for 2 luma samples horizontal.

http://avisynth.nl/index.php/Color_spaces

Most plugins [Planar] handle Luma first,
and then chroma either in one pass or two ie both chroma channels processed in one pass together.

YUY2, is currently not the favourite flavour, everybody is tryin' to drop it altogether.

[pinterf]

You can make a one-liner which puts together luma+2 chroma planes of YV12 into a single plane so you can handle it as a greyscale Y8. After you finished you can convert it back.
Handling the three planes separately should not be giving you headache, you will quickly get used to it.
In general: later you have to adapt to the horizontal and vertical subsampling of U and V planes (YV12, YV16, YV24), but only after keeping the learning curve.
Heads up, when I first looked at the Avisynth+ source code, I have not even recognized the c++ language in it :)

[StainlessS]

If I was starting today and wanted single pass just to figure out what I was doing, think I'de try YV24,
so do all 3 planes in single pass. You'de have to check that input was YV24, and throw an error if not.

[pinterf]

Yep, my idea of putting yv12 planes together does not work since it should be processed with different block sizes. YV24 is the easiest way.

[Ceppo]

YUY2 ?, 4 bytes interleaved, YUYV, 1 Chroma sample for 2 luma samples horizontal.

http://avisynth.nl/index.php/Color_spaces

Most plugins [Planar] handle Luma first,
and then chroma either in one pass or two ie both chroma channels processed in one pass together.

YUY2, is currently not the favourite flavour, everybody is tryin' to drop it altogether.
My source is YV12 so that would be upscaling I guess.

You can make a one-liner which puts together luma+2 chroma planes of YV12 into a single plane so you can handle it as a greyscale Y8. After you finished you can convert it back.
Handling the three planes separately should not be giving you headache, you will quickly get used to it.
In general: later you have to adapt to the horizontal and vertical subsampling of U and V planes (YV12, YV16, YV24), but only after keeping the learning curve.
Heads up, when I first looked at the Avisynth+ source code, I have not even recognized the c++ language in it
I like your suggestion. Sadly I don't know how to do it. Can you link some examples, please?
If I was starting today and wanted single pass just to figure out what I was doing, think I'de try YV24,
so do all 3 planes in single pass. You'de have to check that input was YV24, and throw an error if not.
I was thinking the same, like pointresize the chroma, filter, then point resize it back. But I have to do matrics and it would heavily influence them.

EDIT: if I were to do something like:
YYYYU
YYYYV
YYYYU
YYYYV
and process it as a 5x4 block size would it not work?

[pinterf]

See UtoY8 and VtoY8 and the others in http://avisynth.nl/index.php/Swap
or the Extract family + CombinePlanes if you have avs+
http://avisynth.nl/index.php/Extract and http://avisynth.nl/index.php/CombinePlanes

edit:
for YV12 (in general 4:2:0, because YV12 is a 8 bit format)
c=last
StackVertical(c.ExtractY,StackHorizontal(c.ExtractU, c.ExtractV))

[pinterf]

We'd better stick with sizes of power-of-two. Later it helps optimizations, even modern compilers can 'vectorize' known access patterns.

On the other hand, you have made a small mistake, four luma (Y) pixels have 1-1 U and V counterparts in YV12. In your drawing you have eight Y for an U-V pair.

[Ceppo]

I didn't realize my mistake, now I see why is wrong.

Thanks for the avs solution, didn't think about it.

Tomorrow I will start making experiments and see if YV24 is the only way.

[StainlessS]

Here a mod of FredAverage, MyAverage:- https://www.mediafire.com/file/p49pel6rreo0xyp/MyAverage.zip/file



MyAverage, v2.6+

A simple average filter for Avisynth v2.60 standard colorspaces, only.

Returns a clip where each return frame is a single color average of input frame, same size and colorspace as input.
Does an invert on result if Bool Invert==true.


ColorSpace, YV12, YV16, YV24, YV411, Y8, YUY2, RGB24, RGB32, only.

Return clip Y, U and V, or R, G and B, will be channel averages, unless Invert==True, where channels averages will be inverted.

MyAverage(clip c, Bool "Invert"=false,Bool "TV_YUV"=False,Bool "MyYV24"=False)

Invert, Default false == sampled average. Otherwise Inverted average.
TV_YUV, Default false, If True(And YUV), then photo negative invert around TV levels mid Y(125.5), rather than 127.5.
MyYV24, If true and YV24 (only YV24), then process Y,U,V, together, else by planes.

Returns clip same colorspace and size as input.


I cant guarantee that it works, only that it compiles.
I changed a number of lines of code as they may have been a bit obscure, so I could have cocked up doing that.
It just has two different ways of doing the same thing, (1) regular planar proc luma and then choma, or (2)all 3 channels proc at once
(YV24 only), if not YV24 then MyYV24 arg ignored.

Anyway, I gotta get to the shop, and then crawl into my bed, I'm exhausted.

EDIT: part of the code

PVideoFrame __stdcall MyAverage::GetFrame(int n, IScriptEnvironment* env) {
n = (n<0) ? 0 : (n>= vi.num_frames) ? vi.num_frames - 1 : n; // range limit n

PVideoFrame src = child->GetFrame(n, env);
const int rowsize = src->GetRowSize(PLANAR_Y); // PLANAR_Y no effect on RGB or YUY2
const int pitch = src->GetPitch(PLANAR_Y);
const int height = src->GetHeight(PLANAR_Y);
const BYTE * srcp = src->GetReadPtr(PLANAR_Y);
//
PVideoFrame dst = env->NewVideoFrame(vi);
const int drowsize = dst->GetRowSize(PLANAR_Y);
const int dpitch = dst->GetPitch(PLANAR_Y);
const int dheight = dst->GetHeight(PLANAR_Y);
BYTE * dstp = dst->GetWritePtr(PLANAR_Y);

const int samples = vi.width * vi.height;
int x,y;

// +++++++++++++++++ CRAZY CEPPO STUFF ++++++++++++++++++++++

if(vi.IsYV24() && myYv24) { // YV24 Special code
// Some properties of all 3 planes are identical for YV24
__int64 acc=0,accU=0,accV=0;
unsigned int sum=0,sumU=0,sumV=0;

const int spitchUV = src->GetPitch(PLANAR_U); // Planar, Same pitch for both U and V

const BYTE * srcpU = src->GetReadPtr(PLANAR_U);
const BYTE * srcpV = src->GetReadPtr(PLANAR_V);

for(y=0; y < height ; y++) { // sum luma
for(x=0; x < rowsize; ++x) {
sum += srcp[x];
sumU += srcpU[x];
sumV += srcpV[x];
}
if(sum & 0x80000000) { acc += sum; sum =0; } // Overflow 31 bit ?
if(sumU & 0x80000000) { accU += sumU; sumU=0; }
if(sumV & 0x80000000) { accV += sumV; sumV=0; }
srcp += pitch;
srcpU += spitchUV;
srcpV += spitchUV;
}
acc += sum;
accU += sumU;
accV += sumV;

double ave_D = double(acc) / samples; // luma average
int ave;
int aveU = int(double(accU) / samples + 0.5); // u an v averages
int aveV = int(double(accV) / samples + 0.5);

if(invert) { // invert ?
if(tvy) { // tv levels invert ? [ TV levels center is 125.5 not 127.5, ie (16 + 235)/2 ]
ave = int(-(ave_D - 125.5) + 125.5 + 0.5); // TV_YUV Y mid = 125.5, invert, and Round
} else {
ave = int(ave_D + 0.5) ^ 0xFF; // PC_YUV Y mid = 127.5, symmetrical about 127.5
}
aveU ^= 0xFF ;
aveV ^= 0xFF;
} else {
ave = int(ave_D + 0.5);
}
ave = max( min( ave, 255) ,0);

const int dpitchUV = dst->GetPitch(PLANAR_U);
BYTE * dstpU = dst->GetWritePtr(PLANAR_U);
BYTE * dstpV = dst->GetWritePtr(PLANAR_V);

for(y=0 ; y < dheight ; ++y) { // write return clip luma
for(x=0 ; x < drowsize; ++x) {
dstp [x] = ave;
dstpU[x] = aveU;
dstpV[x] = aveV;
}
dstp += dpitch;
dstpU += dpitchUV;
dstpV += dpitchUV;
}
}

// +++++++++++++++++ END CRAZY CEPPO STUFF ++++++++++++++++++++++


else if(vi.IsPlanar()) { // Planar

// ... etc

[Ceppo]

// +++++++++++++++++ CRAZY CEPPO STUFF ++++++++++++++++++++++
AHAHAHHAHAH

So anyway, I noticed that it uses the same pitch for U and V channels. So it is guaranteed that they have the same pitch?

[StainlessS]

Yes, always U & V same pitch.
NOTE, Pitch can change frame by frame, so must fetch pitch always, cannot fetch once and assume that is always the same.

IanB torture test:- https://forum.doom9.org/showthread.php?p=1628159#post1628159

The YV12 path of ConvertToYUY2() outputs mod 32 pitch so it can do 16 pixels per loop. Line 132 of convert/convert_yuy2.cpp

And must not assume any relationship between pitch of the luma planes and chroma planes for the planar formats.

You must use the GetPitch() method on each and every PVideoFrame you get.

Pitch torture test for filters....
A=SelectEvery(4, 0)
B=SelectEvery(4, 1).AddBorders(0,0,8,0).Crop(0,0,-8,0)
C=SelectEvery(4, 2).AddBorders(0,0,16,0).Crop(0,0,-16,0)
D=SelectEvery(4, 3).AddBorders(2,0,22,0).Crop(2,0,-22,0)
Interleave(A,B,C,D)
FilterUnderPitchStressTest(...)
...

EDIT:
Or I made it a function here

Function PitchTortureTest(clip c) { # IanB:- https://forum.doom9.org/showthread.php?p=1628159#post1628159
c
A=SelectEvery(4, 0)
B=SelectEvery(4, 1).AddBorders(0,0,8,0).Crop(0,0,-8,0)
C=SelectEvery(4, 2).AddBorders(0,0,16,0).Crop(0,0,-16,0)
D=SelectEvery(4, 3).AddBorders(2,0,22,0).Crop(2,0,-22,0)
Interleave(A,B,C,D)
}


EDIT: If your clip is YV12 and you want to test with YV24, then just do a
ConvertToYV24(Interlaced = whatever)
# ... Your Test stuff
ConvertToYV12(Interlaced = whatever)

EDIT: Linked source also processes YUY2 [and the 2.6 colorspaces]

[Ceppo]

Thank you.

[Ceppo]

A problem arose. Since my source is interlaced, I deinterlaced it with this (https://forum.doom9.org/showthread.php?p=1935471#post1935471). My filter treats the chroma as interlaced since the source is of course always interlaced or has at least the chroma placement (YV12) set as interlaced. Now the output should be progressive but the chroma placement is still interlaced. So when I convert to YV24 I should use converttoyv24(interlace=true) or false? The difference is that interlace=true gives a blurred chroma while false doesn't. I can't see any artifacts, could someone explain?

[StainlessS]

I guess I would look up in existing deinterlacers to see how chroma is handled, if done properly then would use interlaced=false
in colorspace conversion.

Note, in above torture test thingy, if you follow with return Last.Info(), I think it shows pitch, so you should see it changing.

[StainlessS]

In this snippet from posted source,

if(invert) { // invert ?
if(tvy) { // tv levels invert ? [ TV levels center is 125.5 not 127.5, ie (16 + 235)/2 ]
ave = int(-(ave_D - 125.5) + 125.5 + 0.5); // TV_YUV Y mid = 125.5, invert, and Round
} else {
ave = int(ave_D + 0.5) ^ 0xFF; // PC_YUV Y mid = 127.5, symmetrical about 127.5
}
aveU ^= 0xFF ;
aveV ^= 0xFF;
} else {
ave = int(ave_D + 0.5);
}
ave = max( min( ave, 255) ,0);

Stuff in BLUE ain't exactly correct, xor with $FF would invert $80 U,V center to $7F,
but avoids problem where source U,V == 0 would invert to $100, we invert to $FF in 8 bit range,
also method adopted will arrive back to original source value if inverted twice.
As it is, it aint quite right but method I chose. Maybe I should invert and clip, there should be no source of $00 anyway.
EDIT: Source $00 equivalent to center - 128, and source $FF equiv to center + 127, ie not symmetrical about center 128.