Another filter idea: Adaptative CNR2

[SILICON]

The human vision can not see well the color changes when the image is dark.


We can:
- Cut the frame in 8x8 blocks (I don´t know the best size).
- Test the max luma value of the pixel in the block.
- If the max luma value is very low then high blurring the croma values of the block.
- If the max luma value is low then blurring the croma values of the block.
- If the max luma value is medium low then medium blurring the croma values of the block.
- If the max luma value is high then low blurring the croma values of the block.
- If the max luma value is very high then do not blurring the croma values of the block.

The idea are blurring more the croma in the dark areas (less noticiable for the eyes) and blurring less in the light areas (more moticiable for the eyes).
It can reduce bitrate need without produce noticiable artifacts.

[ronnylov]

Is this something similar to Kurosu's antiblink filter?

http://forum.doom9.org/showthread.php?s=&threadid=33319
The last version did have an adaptive block based chroma noise filtering function. Try it!

[Malcolm]

Hi SILICON,

i had the same idea for such a filter some days ago. But hadn't the time do post it here. one could write a test version with the filterfactory-plugin for VDub. i'd do it if i had the time...

Btw. i don't think it's necessary to cut the frame in 8x8 blocks. the filter can work pixelwise, using the average of all neighbour pixels for the luma-value.

bye, Malcolm

[slk001]

Where does one find the filterfactory-plugin for VirtualDub that you mentioned?

[Malcolm]

You can find the filterfactory-plugin here: filterfactory (http://shelob.mordor.net/dgraft/other.html)

But it's nearly useless without an explanation of it's language (it's C-like). The only explanation i've found so far is here: filterfactory-explanation (http://virtualdub.everwicked.com/index.php?act=ST&f=11&t=128&hl=filter+factory)

I wrote an example of the filter for the filterfactory.
I'm at work right now, so i haven't much time. And it's the first time that i'm coding something for filterfactory. :p
The biggest problem is, that in Virtualdub, everything works in RGB space. So i had so change from RGB to YUV and back. (i did this the first time an took the formula from the web. it seems to work, so i hope i did it right! :) )

The following version of the script does no blurring. Instead every pixel, which Luma is below 64 is colored blue.

filterfactory is somewhat limited; as far as i can see, it's not possible to define your own methods. (i used filterfactory only for one hour :rolleyes: ). So, i hadn't the time to write the code for the blurring-part!

Here's the script:

void run(bitmap dst, bitmap src) {
int x, y, w, h;

int Y, Cb, Cr, R, G, B;

int red, green, blue;

int threshhold;
threshhold = 64;

scan(dst, src) {
Y = ((77 * in.r + 150 * in.g + 29 * in.b) >> 8);

if (Y < threshhold) {
Cb = ((-43 * in.r - 85 * in.g + 128 * in.b) >> 8) + 128;
Cr = ((128 * in.r - 107 * in.g - 21 * in.b) >> 8) + 128;

Cb = 160;
Cr = 96;

red = (Y - 16)*298/256 + (Cr-128)*516/256;
green = (Y - 16)*298/256 - (Cr-128)*208/256 - (Cb-128)*100/256;
blue = (Y - 16)*298/256 + (Cb-128)*409/256;

if (red < 0) {
red = 0;
} else if (red > 255) {
red = 255;
}
if (green < 0) {
green = 0;
} else if (green > 255) {
green = 255;
}
if (blue < 0) {
blue = 0;
} else if (blue > 255) {
blue = 255;
}

out.r = red;
out.g = green;
out.b = blue;
} else {
out = in;
}
}
}




bye, Malcolm

[Malcolm]

O.k. here's a version of the script which does the blurring. it's my first implementation of a blur algorithm and it's very simple. it just averages one pixel (Cb and Cr) with it's 8 neighbours.
the part for the blurring looks awful, because i couldn't pack it in a method/function. so everything had to be inlined. :scared:
And at the moment, there's abug in it. along with the blurring, the color of the dark areas are shifted towards green.

i think the algorithm is worth publishing anyway, because it's comlpletely different to the one posted above. that's because you can't use scan() when you want to access more pixels at a time.

there are some things to be aware: src.w and src.h don't seem to work. so you have to fill in the width and height of your clip manually.
the blurring isn't strong enough. feel free to add another 16 neighbours. (that's x-2, y-2, x+2, y+2, ...)
and... IT'S LOW AS HELL!!! :devil: :D

Maybe, a completely different sort of blurring should be used (temporal for example).

Maybe one can achieve the wanted Result with a combination of existing filters:
- selecting dark areas (e.g. with Greyscale(), then Layer() & Mask())
- blurring (Blur(), SpatialSoften(), Convolution3D(), ...)
- assigning only to the Chrominance-Channels (Layer() ?)

Anyway, here's the script:

void run(bitmap dst, bitmap src) {
int x, y, w, h;

int Y, Cb, Cr, R, G, B;

int red, green, blue;

int blurY, blurCb, blurCr;
int finalY, finalCb, finalCr;

int threshhold;
threshhold = 64;

pixel32 newPix;

w = 640;
h = 304;

for (x=0; x<=w; x++) {
for (y=0; y<=h; y++) {

Y = ((77 * src[x, y].r + 150 * src[x, y].g + 29 * src[x, y].b) >> 8) + 16;

if (Y < threshhold) {
blurCb =
((-43 * src[x-1, y-1].r - 85 * src[x-1, y-1].g + 128 * src[x-1, y-1].b) >> 8) + 128 +
((-43 * src[x, y-1].r - 85 * src[x, y-1].g + 128 * src[x, y-1].b) >> 8) + 128 +
((-43 * src[x+1, y-1].r - 85 * src[x+1, y-1].g + 128 * src[x+1, y-1].b) >> 8) + 128 +
((-43 * src[x-1, y].r - 85 * src[x-1, y].g + 128 * src[x-1, y].b) >> 8) + 128 +
((-43 * src[x+1, y].r - 85 * src[x+1, y].g + 128 * src[x+1, y].b) >> 8) + 128 +
((-43 * src[x-1, y+1].r - 85 * src[x-1, y+1].g + 128 * src[x-1, y+1].b) >> 8) + 128 +
((-43 * src[x, y+1].r - 85 * src[x, y+1].g + 128 * src[x, y+1].b) >> 8) + 128 +
((-43 * src[x+1, y+1].r - 85 * src[x+1, y+1].g + 128 * src[x+1, y+1].b) >> 8) + 128 ;

blurCb = blurCb/8;

Cb = ((-43 * src[x, y].r - 85 * src[x, y].g + 128 * src[x, y].b) >> 8) + 128;

finalCb = ((blurCb * (1 - (Y / threshhold))) + (Cb * Y / threshhold)) / 2;

blurCr =
((128 * src[x-1, y-1].r - 107 * src[x-1, y-1].g - 21 * src[x-1, y-1].b) >> 8) + 128 +
((128 * src[x, y-1].r - 107 * src[x, y-1].g - 21 * src[x, y-1].b) >> 8) + 128 +
((128 * src[x+1, y-1].r - 107 * src[x+1, y-1].g - 21 * src[x+1, y-1].b) >> 8) + 128 +
((128 * src[x-1, y].r - 107 * src[x-1, y].g - 21 * src[x-1, y].b) >> 8) + 128 +
((128 * src[x+1, y].r - 107 * src[x+1, y].g - 21 * src[x+1, y].b) >> 8) + 128 +
((128 * src[x-1, y+1].r - 107 * src[x-1, y+1].g - 21 * src[x-1, y+1].b) >> 8) + 128 +
((128 * src[x, y+1].r - 107 * src[x, y+1].g - 21 * src[x, y+1].b) >> 8) + 128 +
((128 * src[x+1, y+1].r - 107 * src[x+1, y+1].g - 21 * src[x+1, y+1].b) >> 8) + 128 ;

blurCr = blurCr/8;

Cr = ((128 * src[x, y].r - 107 * src[x, y].g - 21 * src[x, y].b) >> 8) + 128;

finalCr = ((blurCr * (1 - (Y / threshhold))) + (Cr * Y / threshhold)) / 2;

red = (Y - 16)*298/256 + (finalCr-128)*516/256;
green = (Y - 16)*298/256 - (finalCr-128)*208/256 - (finalCb-128)*100/256;
blue = (Y - 16)*298/256 + (finalCb-128)*409/256;

if (red < 0) {
red = 0;
} else if (red > 255) {
red = 255;
}
if (green < 0) {
green = 0;
} else if (green > 255) {
green = 255;
}
if (blue < 0) {
blue = 0;
} else if (blue > 255) {
blue = 255;
}

newPix.r = red;
newPix.g = green;
newPix.b = blue;

dst[x, y] = newPix;
} else {
dst[x,y]= src[x,y];
}
}
}
}



bye, Malcolm

[Malcolm]

@SILICON

here's a script (avisynth, not filter-factory) which does exactly what you described in your original post.
i hadn't the time to test it with dvd-matrial. I also hadn't the time to measure how it influences compressibility. Would be nice, if someone will test this.

i've included 3 methods to modify the dark areas. the slowest by far is by using SpatialBlur(). A big radius is needed here to (i guess) deliver good results.
the second method is using TemporalBlur(). i don't know how good it is / how it influences compressibility.
the third, and fastest is to grey out the dark areas. the only drawback is, that if you apply this method to areas, which are not dark enough, a visible desaturation will occur.

here's the script:

# The Source
AviSource("T:\Temp\test.AVI")

# slowest, time heavily depends on parameter 'radius'
# the more blurring (big radius) the better
# overlay = SpatialSoften(5,255,255).ConvertToRGB32()

# second fastest
# overlay = TemporalSoften(5,255,255).ConvertToRGB32()

# fastest, but leads to visible desaturation in
# areas, which are not dark enough
overlay = BlankClip(last, $888888).ConvertToRGB32()

overlay = resetmask(overlay)
base = ConvertToRGB32(last)

# luma-range from 16 to 32 will be used as inverted mask
# you can use the gamma-parameter to further adjust the strength
mask_clip = levels(base,16,1,32,255,0)
overlay_clip = Mask(overlay, mask_clip)

# replacing the dark areas with the modified image
final = layer(base,overlay_clip,"add",255,0,0,0)

# only the chroma-channels are replaced
MergeChroma(ConvertToYUY2(base), ConvertToYUY2(final))


bye, Malcolm

[SILICON]

For every pixel you make:
- Change 8 pixels of RGB to YCbCr
- Make average for one pixel
- Change one pixel to RGB
- Test the pixel values.
For a frame size de 100x50 you make 100*50*(9+1) colorspace changes

I suggest:
- Change all frame of RGB to YCbCr
- Make average for one pixel
- Change one pixel to RGB
- Test the pixel values.
For a frame size de 100x50 you will make 100*50*(1+1) colorspace changes

Of course your filter will be a lot of faster if work only in YUV colorspace.

I suggest this algoritm:

int threshhold = 64;

pixel32 newPix;
int w = 640;
int h = 304;

int TempFrameY[720,576]; // I don't know how make it dinamic size.
int TempFrameCb[720,576];
int TempFrameCr[720,576];
int x, y;
int BlurCb, BlurCr

// Change all pixels to YUV
for (x=0, x<w, x++)
for (y=0, y<h, y++)
{
TempFrameCb(x,y)=((-43 * src[x, y].r - 85 * src[x, y].g + 128 * src[x, y].b) >> 8) + 128;
TempFrameCr(x,y)=((128 * src[x, y].r - 107 * src[x, y].g - 21 * src[x, y].b) >> 8) + 128;
TempFrameY(x,y)=((77 * src[x, y].r + 150 * src[x, y].g + 29 * src[x, y].b) >> 8) + 16;
if( TempFrameY(x,y) == 0) TempFrameY(x,y)++; // for not have "error, divided by 0"
}

// make the blur
for (x=1, x<w-1, x++) //skip the borders!!
for (y=1, y<h-1, y++)
{ //make the average in Cb and Cr
BlurCb=TempFrameCb[x-1,y-1]+TempFrameCb[x-1,y]
+TempFrameCb[x-1,y+1]+TempFrameCb[x,y-1]
+TempFrameCb[x,y+1]+TempFrameCb[x+1,y-1]
+TempFrameCb[x+1,y]+TempFrameCb[x+1,y+1];
BlurCb>>=3; //sum all neigbour pixels and divide for 8
BlurCr=TempFrameCr[x-1,y-1]+TempFrameCr[x-1,y]
+TempFrameCr[x-1,y+1]+TempFrameCr[x,y-1]
+TempFrameCr[x,y+1]+TempFrameCr[x+1,y-1]
+TempFrameCr[x+1,y]+TempFrameCr[x+1,y+1];
BlurCr>>=3; //sum all neigbour pixels and divide for 8


// for more Y, more influence of Original Pixel
// For more threshhold, less influence of Original Pixel
// linnear funtion
finalCb = (TempFrameCb[x,y] * TempFrameY[x,y] / threshhold)
+ (BlurCb * threshhold / TempFrameY[x,y]);
finalCr = (TempFrameCr[x,y] * TempFrameY[x,y] / threshhold))
+ (BlurCr * threshhold / TempFrameY[x,y]);

//Now change the pixel to RGB
red = (Y - 16)*298/256 + (finalCr-128)*516/256;
green = (Y - 16)*298/256 - (finalCr-128)*208/256 - (finalCb-128)*100/256;
blue = (Y - 16)*298/256 + (finalCb-128)*409/256;

if (red < 0) red = 0;
else if (red > 255) red = 255;
if (green < 0) green = 0;
else if (green > 255) green = 255;
if (blue < 0) blue = 0;
else if (blue > 255) blue = 255;

newPix.r = red;
newPix.g = green;
newPix.b = blue;

dst[x, y] = newPix;
}
// The end :-)


For gauss funtion (I think best metod) change:

finalCb = (TempFrameCb[x,y] * TempFrameY[x,y]^GAUSS / threshhold)
+ (BlurCb * threshhold / TempFrameY[x,y]^GAUSS);
finalCr = (TempFrameCr[x,y] * TempFrameY[x,y]^GAUSS / threshhold))
+ (BlurCr * threshhold / TempFrameY[x,y]^GAUSS);

Them: higger blur in Y low values, same blur than linnear funtion in the midle blur values and less blur in Y high values.
I have not sure about Gauss funtion (test it!!)