Composite Modes

[Bester]

Every paint program has a layer compositing mode called "overlay". I would like to
apply this in avisynth, but I don't know how to get there.

aka from here
http://img373.imageshack.us/img373/812/comporig3oh.jpg

to there
http://img373.imageshack.us/img373/4...laymode2sa.jpg

I found this on the net:
"Overlay mode applies two different techniques. If the pixel color value is lighter than middle gray, 128 value, then Screen mode is applied. Where the pixel color value is darker then middle gray, 128, Multiply mode is applied which makes the dark parts darker."

"Multiply compares the color values of the pixels in both clips and multiplies them together. Dark areas of the clips will remain dark, lighter areas will be darkened as a result. px = (pxlayer1 * pxlayer2) / 255"

"Screen compares the color values of the pixels in both clips and multiplies the inverse values of the pixels. Lighter parts of the image will remain the same. Darker parts of the image will be lightened by this process.
Wert=255-((255-pxlayer1)*(255-pxlayer2)/255)"

Any ideas how to achieve this??

[Didée]

AviSynth's "Overlay" command is what you're looking for. Look at the filter description in the AviSynth manual - the modes mode="softlight" and mode="hardlight" are said to exactly mimic the according transformations from Photoshop.

Personally, I don't like these transformations too much (they possibly create luma overflow, which then is calculated into chroma). What I use mostly are selfmade transformations through MaskTools' YV12Lut and YV12Lutxy commands.

Once I started my "replace all Overlay() transformations through YV12Lut/xy()" project, but didn't work on it since day 2 -- so it's still in the middle of somewhere, or nowhere ...

These are the extracted functions regarding "Hardlight":

#-----------------------------------------------------
function YV12Hardlight1(clip clp1, clip clp2, float "opacity", int "Cmode")
{
Cmode   = default( Cmode, 2)
opacity = default( opacity, 1.0)
opacity = opacity<0.0 ? 0.0 : opacity>1.0 ? 1.0 : opacity
OPAC = string( opacity )
HL1  =       "y 128 < "
HL1  = HL1 + "x x 128 / 128 y - abs * - "+OPAC+" * x 1 "+OPAC+" - * + "
HL1  = HL1 + "x 255 x - 128 / 128 y - abs * + "+OPAC+" * x 1 "+OPAC+" - * + "
HL1  = HL1 + "?"
return yv12lutxy(clp1,clp2,HL1,HL1,HL1,U=Cmode,V=Cmode)
}
function YV12Hardlight2(clip clp1, clip clp2, float "opacity", int "Cmode")
{
Cmode   = default( Cmode, 2)
opacity = default( opacity, 1.0)
opacity = opacity<0.0 ? 0.0 : opacity>1.0 ? 1.0 : opacity
OPAC = string( opacity )
HL1  =       "y 128 < "
HL1  = HL1 + "x x 16 < 16 x 16 - ? 128 / 128 y - abs * - "+OPAC+" * x 1 "+OPAC+" - * + "
HL1  = HL1 + "x x 235 > 235 235 x - ? 128 / 128 y - abs * + "+OPAC+" * x 1 "+OPAC+" - * + "
HL1  = HL1 + "?"
return yv12lutxy(clp1,clp2,HL1,HL1,HL1,U=Cmode,V=Cmode)
}
function YV12Hardlight3(clip clp1, clip clp2, float "opacity", int "Cmode")
{
Cmode   = default( Cmode, 2)
opacity = default( opacity, 1.0)
opacity = opacity<0.0 ? 0.0 : opacity>1.0 ? 1.0 : opacity
OPAC = string( opacity )
HL1  =       "y 128 < "
HL1  = HL1 + "x x 128 / 128 y - abs 128 / 1 2.0 / ^ 128 * * - "+OPAC+" * x 1 "+OPAC+" - * + "
HL1  = HL1 + "x 255 x - 128 / 128 y - abs 128 / 1 2.0 / ^ 128 * * + "+OPAC+" * x 1 "+OPAC+" - * + "
HL1  = HL1 + "?"
return yv12lutxy(clp1,clp2,HL1,HL1,HL1,U=Cmode,V=Cmode)
}
function YV12Hardlight4(clip clp1, clip clp2, float "opacity", int "Cmode")
{
Cmode   = default( Cmode, 2)
opacity = default( opacity, 1.0)
opacity = opacity<0.0 ? 0.0 : opacity>1.0 ? 1.0 : opacity
OPAC = string( opacity )
HL1  =       "y 128 < "
HL1  = HL1 + "x x 16 < 16 x 16 - ? 128 / 128 y - abs 128 / 1 1.4142 / ^ 128 * * - "+OPAC+" * x 1 "+OPAC+" - * + "
HL1  = HL1 + "x x 235 > 235 235 x - ? 128 / 128 y - abs 128 / 1 1.4142 / ^ 128 * * + "+OPAC+" * x 1 "+OPAC+" - * + "
HL1  = HL1 + "?"
return yv12lutxy(clp1,clp2,HL1,HL1,HL1,U=Cmode,V=Cmode)
}
#-----------------------------------------------------

Alas I didn't note the actual transformations done for the variants 1 - 4, so the user has to figure for itself

[Bester]

Your functions work pretty well.