Overflow in function Overlay using mode add and subtract

[ErazorTT]

Hi folks,

I was trying to do some stuff when I realized that the function Overlay overflows quite "wierdly" in the modes add and subtract. Now it could of course be that this intendet. However, very ugly artifacts are the consequence (at least for what I was trying to use this function).

Try the following snippet to understand what I mean by "wierdly":
cl=ConvertToYUY2(last)
cl1=BlankClip(cl,color=$aa0000)
cl2=BlankClip(cl,color=$0000d0)
cl3=Overlay(cl1,cl2,mode="subtract")

Look at cl3 and at the YUV colors, especially at the U channel. The color will be 37,4,240. Now increase the number 0000d0 (Try 0000d1, 0000d2...). See how the U Channel decreases. At 0000d9 the U channel is finally at 0. However at the next number (0000da) the U channel warps around at starts off 255, which of course means that the color changes dramatically.

Now to the developers: Is this intended?
Wouldn't it be better to saturate?

Greetings,
erazortt

[IanB]

I never quite understood Sh0dan's intention with the Subtract and Difference modes in Overlay, nor the effect from YUY2 Subtract in Layer. I would guess they were supposed to result in a related effect to what would happen in RGB space, but I never really gave them that much thought, it's not something I ever had occasion to use. The functions actually implemented are what they are.

And yes I do think the result should be saturated.

One issue with UV saturation is Hue change. Hue is related to atan2(U, V) thus related to the ratio U/V. When one saturates either U or V then one probably should also scale the other channel to keep the Hue constant while sacrificing Saturation. This unfortunately is computationally quite expensive.

[ErazorTT]

Yes, indeed, the hue is an additional issue.
However, the hue is also scrued up at the overflow.