Layer filter, level value question

[BlindWanderer]

In the docs it says

in RGB32 the alpha-channel of the overlay_clip is multiplied with level, so the resulting alpha = (alpha_mask * level) / 256. This means for full strength of operation BOTH alpha and level have to be 255.


But it seems to me that doesn't mathematically work.
if we substitute in values, "alpha = (alpha_mask * level) / 256" becomes "alpha = (255 * 255) / 256" to which alpha would equal "254.00390625" which would be truncated at 254. Considering that 254 is not equal to 255; then it could be said that where it previously states that using a value of 255 for the level will return the overlay, is in fact incorrect. For the math to work, level would have to be 256.

I think what the documentation should say:

in RGB32 the alpha-channel of the overlay_clip is multiplied with level, so the resulting alpha = (alpha_mask * level) / 256. This means for full strength of operation alpha has to be 255 and level has to be 256.


Of course I cannot verify that this is infact how the code works as I do not fully grok mmx assembly.

[IanB]

For the "Add" method the RGB32 arithmetic is done like this P1 += ( (P2-P1)*( (A2*level)>>8 ) )>>8and for YUY2 P1 += ( (P2-P1)*level )>>8So it looks like level=256 is required for 100% function.

But for RGB32 even with level=256 and alpha=255 there is still a slight loss. i.e 256*255>>8 = 255, 255*255>>8 = 254

Ah the penalty for fast MMX.

[sh0dan]

I have seen Avery use:
alpha += (alpha > 0);
Which is implementable in MMX. You could also use 127 as split, though I doubt you'll notice the difference between '>0' and '>127'.

[BlindWanderer]

Ah ok, so it's just the documentation thats wrong (just as I thought).

You see, i was working with a program that renders animated fractals to RGB32 with a clear background (via PNG). I wanted to set the background color but i like things perfect.

On that note, it would be cool if DevIL could handle the setting of the background color for PNG's with an alpha (with RGB24 output it just stripping the alpha).

As to hacking the range so it works for 255 as full, i think thats just wrong; corrupting the alpha in the process is wrong. Why does the function parameter have to be 0->255?