avoiding clipping with Overlay add/subtract modes?

[bredboi]

I've been experimenting with adding and subtracting to see if it can help improve certain operations.

for example, by denoising an image and then subtracting that from the original, you can process the denoised image further and then recombine the original grain/noise onto the processed image. this can potentially help with sharpening a grainy image as you avoid oversharpening the grain itself, and only sharpen the underlying image.

anyway, I started my experimenting in DaVinci Resolve Fusion and using its ChannelBooleans node to perform the subtraction and addition. this worked fine and recombining the grain worked as hoped.

but porting to avisynth, using the "add" and "subtract" modes in Overlay, I ran into the problem that recombining the grain onto the denoised image would result in less grain than the original.

upon further investigation, I think the issue is avisynth clipping negative pixel values, so the grain is only retained where it's brighter than the denoised pixel. Fusion has no problem keeping negative pixel values so the grain survives fully.

I've tried to add some initial offset to the source to avoid negative values with ColorYUV, which helps but risks clipping highlights so isn't ideal

I've tried converting to float with ConvertToFloat but that doesn't seem to work with Overlay's adding/subtracting, the denoised image goes through untouched.

I've tried using the "difference" mode in Overlay as it centres around grey but haven't been able to recombine it properly

any thoughts on how best to avoid clipping here?

[wonkey_monkey]

I've tried using the "difference" mode in Overlay as it centres around grey but haven't been able to recombine it properly

You could try somethng like:

Expr(denoised_clip, difference_noise_clip, "x y + 128 -")

This will add the clips and remove the bias in one operation.

[poisondeathray]

I've tried converting to float with ConvertToFloat but that doesn't seem to work with Overlay's adding/subtracting, the denoised image goes through untouched.


Use OverlayPlus, as internal Overlay looks to be buggy with float operations

AVS+ x64 r4289

w=blankclip(pixel_type="RGBPS", colors=[1,1,1])
g=blankclip(pixel_type="RGBPS", colors=[0.5,0.5,0.5])
lg=blankclip(pixel_type="RGBPS", colors=[0.25,0.25,0.25])

#overlay(w,w, mode="add") #1.0 wrong; ? clamp or return first value ?
#overlayplus(w,w, mode="add") #2.0 correct

#overlay(w,lg, mode="subtract") #1.0 wrong; return first value ?
#overlay(lg,w, mode="subtract") #0.25 wrong; return first value ?
#overlayplus(w,lg, mode="subtract") #0.75 correct
#overlayplus(lg,w, mode="subtract") #-0.75 correct

#overlay(w,g, mode="subtract") #1.0 wrong; return first value ?
#overlay(g,w, mode="subtract") #0.5 wrong; return first value ?
#overlayplus(w,g, mode="subtract") #0.5 correct
#overlayplus(g,w, mode="subtract") #-0.5 correct

[real.finder]

internal Overlay looks to be buggy with float operations

AVS+ x64 r4289

w=blankclip(pixel_type="RGBPS", colors=[1,1,1])
g=blankclip(pixel_type="RGBPS", colors=[0.5,0.5,0.5])
lg=blankclip(pixel_type="RGBPS", colors=[0.25,0.25,0.25])

#overlay(w,w, mode="add") #1.0 wrong; ? clamp or return first value ?

#overlay(w,lg, mode="subtract") #1.0 wrong; return first value ?
#overlay(lg,w, mode="subtract") #0.25 wrong; return first value ?

#overlay(w,g, mode="subtract") #1.0 wrong; return first value ?
#overlay(g,w, mode="subtract") #0.5 wrong; return first value ?



maybe it worth report it to avs+ devs

[poisondeathray]

maybe it worth report it to avs+ devs


https://github.com/AviSynth/AviSynthPlus/issues/461

[VideoMilk78]

for example, by denoising an image and then subtracting that from the original, you can process the denoised image further and then recombine the original grain/noise onto the processed image. this can potentially help with sharpening a grainy image as you avoid oversharpening the grain itself, and only sharpen the underlying image.

This has already been explored https://forum.doom9.org/showthread.php?t=182831

If you don't want to read through the whole script you could just
source=last
denoising=Source.denoisefilterofchoice().sharpenfilterofchoice
regrained=denoise.mrd_restoregrain(source)
return regrain


find mrd_restoregrain here
https://forum.doom9.org/showthread.php?t=186513
KeepGrain is a separate function and will not be useful for this purpose.

[bredboi]

Thanks all for the input!

OverlayPlus looks to work exactly as I hoped, I'll be continuing with that.

[pinterf]

Use OverlayPlus, as internal Overlay looks to be buggy with float operations

AVS+ x64 r4289

w=blankclip(pixel_type="RGBPS", colors=[1,1,1])
g=blankclip(pixel_type="RGBPS", colors=[0.5,0.5,0.5])
lg=blankclip(pixel_type="RGBPS", colors=[0.25,0.25,0.25])

#overlay(w,w, mode="add") #1.0 wrong; ? clamp or return first value ?
#overlayplus(w,w, mode="add") #2.0 correct

#overlay(w,lg, mode="subtract") #1.0 wrong; return first value ?
#overlay(lg,w, mode="subtract") #0.25 wrong; return first value ?
#overlayplus(w,lg, mode="subtract") #0.75 correct
#overlayplus(lg,w, mode="subtract") #-0.75 correct

#overlay(w,g, mode="subtract") #1.0 wrong; return first value ?
#overlay(g,w, mode="subtract") #0.5 wrong; return first value ?
#overlayplus(w,g, mode="subtract") #0.5 correct
#overlayplus(g,w, mode="subtract") #-0.5 correct


Aside from the fact that 32 bit float is only implemented for Overlay "blend" (and other operation types do not give a "not supported" error message), Overlay's "add" is something totally different from what one might expect; it is not a simple mathematical addition and was primarily designed for YUV.

First of all, sorry for not providing an "unsupported" error for other Overlay modes when input is 32 bit float.

Historically, Overlay converts RGB to YUV444 (except for "blend", where I stopped Overlay from doing this and implemented direct RGB handling).

In YUV "add" and "subtract" are not just per-channel math. The luma (Y) is added/subtracted, but if the result overflows (add, Y>max) or underflows (subtract, Y<0), the chroma (U/V) is pulled toward neutral (gray/white) to mimic how RGB overbright/underbright behaves visually.

http://avisynth.nl/index.php/Overlay

In RGB, adding two bright colors can result in "white" (all channels maxed).
In YUV, if we just add Y, U and V, we can get weird color shifts. "Overlay" compensates by blending U/V toward neutral when Y is out of range, making the result look more like RGB addition.

Future?
For RGB a simple per-channel add/subtract (with clamping for 8/16-bit, or no clamping for float) is what most users (and you all) expect.

The "magic" is only needed for YUV to avoid odd color artifacts. In RGB overbright naturally becomes white, so no special handling is needed.

Until I implement RGB directly (with float input as well), just use the OverlayPlus, which naturally handles addition and subtract for RGB as a simple math, without any historical burden.

[real.finder]

bredboi

in case you didn't notice there are new avs+ that has direct RGB support https://forum.doom9.org/showthread.php?p=2025273#post2025273