Help wanted creating customizable dirt mask

[VideoMilk78]

Hello, after developing the mRD_RestoreGrain mod, keepgrain, I've now set my sights on a better cleaning method.

The biggest issues that I've found is with the common cleaning filters, RemoveDirtMC and Spotless, is that they are very inflexible, especially RemoveDirtMC.

Originally I gravitated towards Despot because it had two very desirable parameters, the ability to only clean white or black spots, and customizable spot size. However after at least a month of trial and failure I deemed despot was not viable.

I was wondering if it would be possible to take a difference mask between the source clip and a cleaned clip and have customizable parameters that decide whether or not spots of a certain size or color are restored/replaced with the cleaned clip.

Thanks in advance.

[Selur]

Something like https://github.com/Sunflower-Dolls/Vapoursynth-areafilter for Vapoursynth could probably be used.

[VideoMilk78]

Thanks selur, that might for the size aspect of restoring however I don't see how it would work for color?

[Selur]

Filtering by luma can be easily done through a luma/limit mask using mt_lut

[VideoMilk78]

Still deciding on how to do the size relation options. How would I do the luma filtering to only show dark spots and light spots? I've been working on some tests but I'm struggling to wrap my brain around how I would go about taking the removed dirt and filtering out lighter spots or darker spots.

[Selur]

How would I do the luma filtering to only show dark spots and light spots?
can't you do something like

src = last
filtered = src.Sharpen(0.5) # Sharpen only as example filter
darkmask = mt_lut(src, "x 50 < 255 0 ?", U=1, V=1) # everything below 50 is white, the rest is black
brightmask = mt_lut(src, "x 200 > 255 0 ?", U=1, V=1) # everything above 200 is white, the rest is black
combimask = mt_logic(darkmask, brightmask, "max") # dark (below 50) are white, bright (above 200) are white, mid range is black
mt_merge(src, filtered, combimask) # filter effect is only keept for dark and bright areas

?

Cu Selur

[VideoMilk78]

I meant restoring white or dark spots not both.
So far this has yielded quite good results, RWhite restores white spots while RBlack restores dark spots.

src=FFVideoSource("test.mov").robocrop()

filtered =src.removedirtmc(30)

dark = mt_lut(src, "x 100 < 255 0 ?", U=1, V=1)
light = mt_lut(src, "x 100 > 255 0 ?", U=1, V=1)

rwhite=overlayplus(filtered,src,mask=light,chroma=false)
rblack=overlayplus(filtered,src,mask=dark,chroma=false)

I've seen MT_Inpand used to reduce mask size (more uses of MT_Inpand -> smaller spots become invisible in the mask) however I want more of a width/height definition for what to restore or not, still figuring out how to do this.

EDIT: this works horribly for most scenes and only worked well for the one scene I was testing unfortunately.

[Selur]

Here as functions (note they assume 8bit 0-255 values):

# Pixels with luma above the threshold become white in the mask.
# Only those areas receive the filtered image.
# clipA = unfiltered, clipB= filtered
function filterBrightOnly(clip clipA, clip clipB, int threshold=200)
{
# mask: bright pixels are white (255), others black (0)
brightmask = mt_lut(clipA, "x " + string(threshold) + " > 255 0 ?", U=1, V=1)

# merge filtered clip only where mask is white
return mt_merge(clipA, clipB, brightmask)
}



# Pixels with luma below the threshold become white in the mask.
# Only those areas receive the filtered image.
# clipA = unfiltered, clipB= filtered
function filterDarkOnly(clip clipA, clip clipB, int threshold)
{
# mask: dark pixels are white (255), others black (0)
darkmask = mt_lut(clipA, "x " + string(threshold) + " < 255 0 ?", U=1, V=1)

# merge filtered clip only where mask is white
return mt_merge(clipA, clipB, darkmask)
}



Cu Selur

Ps.: not sure a simple luma mask is the right approach, but good luck :)

[VideoMilk78]

I came up with this janky function
function DespotMask(clip clean,clip source,int "Width",int "Height",int "Sign",int "Dilate",int "Blur") {
Sign=Default(Sign,0)
SpotWidth=Default(Width,500)
SpotHeight=Default(Height,500)
Dilate=default(dilate,4)
Blur=default(Blur,dilate/4)
Despot=source.Despot(mthres=100,p1=8,p2=4,Sign=Sign,Pwidth=SpotWidth,pheight=SpotHeight,color=false,fitluma=false,seg=0)
DMask=mt_lutxy(source, Despot, expr="x y - abs 0 > 255 0 ?", u=1, v=1).ExpandMask(Dilate,Blur)
Return OverlayPlus(source,clean,Mask=DMask,Chroma=True)
}

It takes the useful detection settings of Despot and allows you to use them with the cleaning filter of your choice.

I really wonder how Despot does the color filtering