[film transfer] Remove dirt but keep grain? [Archive]

chmars

22nd May 2021, 15:08

As many recommanded, I tried Spotless. Impressing!
Here is a part of dozen of old and dirty 8MM reels I have to transfer.
As you can see, lot to tweak not to loose too many details, in the sea, for example.
But promising.

Click on the picture to download the video:

https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/MnyhRNfkoDukWpb/download (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/TUZCy6e2ANtyVqj/download)

But something went wrong with Koka's RestoreGrain trick.

What do I have to adapt not to loose brightness?

source1= AviSource("D:\458_graded-trim_for_tests-v210.avi")

BlkSz = 8
OLap = 2
Pel = 1
Tm=true

spotless_source1=SpotLess(source1,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm)

rd = spotless_source1
ori = source1
out = rd.mRD_RestoreGrain(ori)

StackVertical(subtitle(source1,"original",size=68,align=2),subtitle(spotless_source1,"spotless",size=68,align=2)\
,subtitle(out,"spotless+restoregrain",size=68,align=2))

#------------------------------------------Koka,s RestoreGrain

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 20.0))

expr = " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

#-------------------------------------------StainlessS Spotless

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips.
Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
Bblur = Default(bblur,0.0) # Default OFF
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (bBlur<=0.0 ? c : c.blur(bblur)).MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2)
sup_rend = (bBlur<=0.0) ? sup : c.MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2,levels=1) # Only 1 Level required where not MAnalyse-ing.
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

StainlessS

22nd May 2021, 15:35

Suggest

tm = False # many think its better option
pel = 2 # or 4, 1 is least accurate, maybe 1 for TRUE HD content.

Also add Bblur=0.6 to spotless, or use the beta [about last post in Spotless thread].
I suspect that kolak restore grain thing is 8 bit only.
Maybe DogWay added some HBD restore grain thing to his recent library [or real.finder].

chmars

22nd May 2021, 22:22

Thanks
Suggest

tm = False # many think its better option
pel = 2 # or 4, 1 is least accurate, maybe 1 for TRUE HD content.

Also add Bblur=0.6 to spotless, or use the beta [about last post in Spotless thread].
ok

I suspect that kolak restore grain thing is 8 bit only.

True

Maybe DogWay added some HBD restore grain thing to his recent library [or real.finder].
Found this:
https://github.com/Dogway/Avisynth-Scripts/blob/master/Mods/GrainFactory3mod.avsi
I'll try it.

StainlessS

22nd May 2021, 23:54

I think that Grainfactory thingy is just for adding grain, rather than recovering filtered out grain [I could be wrong].

I dont like grain, and so never have the urge to add any back.

Perhaps someone could advise if any HBD version of mRD_RestoreGrain() exists.

kedautinh12

23rd May 2021, 00:23

Can ask @Dogway or @real.finder for HBD ver

real.finder

23rd May 2021, 04:43

HBD mRD_RestoreGrain
Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {
sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 20.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

kedautinh12

23rd May 2021, 05:21

Thank real.finder

kedautinh12

23rd May 2021, 05:34

I wonder can use mRD_RestoreGrain with another plugins/scripts ex: despot().mRD_RestoreGrain()??

chmars

23rd May 2021, 09:56

Thank you real.finder, great.
On first look, outputs a compromise between source and Spotless. Grains greatly comes back, some dirt also do, of course.
Have to play with parameters though.

kedautinh12

23rd May 2021, 11:09

If you find out better results, can you give me scripts and pictures??

chmars

24th May 2021, 12:02

These tools are impressing!
Here, a test.

https://nmldqjct.preview.infomaniak.website/shared_files/index.php/apps/files_sharing/ajax/publicpreview.php?x=1881&y=571&a=true&file=458_graded_spotless_restoregrain_8bit.00.jpeg&t=9vqEYjKdLB3Kuzq&scalingup=0 (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/BcvtxV9PEwGrVIu/download)

0. Original
1. Comparison of Spotless, RemoveDirtMC,
2. Combination of both,
3. Sharpen,
4. Grain restored.
5. Original

I wasn't able to decide between Spotless and RemoveDirtMC, because one is removing dirts that the other one wouldn't remove.
So I tried to concatenate both, just to see.
Still a bit of dirt back with grain restoration but for me, the compromise is very acceptable.

Here, the avi file used to build this test (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/citK3NKEazfcsr4/download)
Here, the resulting compressed file (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/BcvtxV9PEwGrVIu/download) (same as picture click)
Here, the resulting lossless compressed file (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/EzGGOAVgKn77Xty/download)

Script in the next post

EDIT: I didn't try yet to play with RestoreGrain parameters.

Strange practice but it's not so bad, what do you think?
Is there a way to stay in 4:2:2 10 bit with unsharpmask or what is the best conversion to feed it?
.

chmars

24th May 2021, 12:03

/*
24/05/2021 - Experiment by Chmars, thank to solutions given by doom9 forum users, to remove dirt but keep grain.
Discussion here: https://forum.doom9.org/showthread.php?p=1943385#post1943385

Much more experimented people made scripts for footge restoring, please read them here: https://forum.doom9.org/showthread.php?t=144271
! Multithread, memory and other stuff as used here might be totally wrong !

Tools used:
StainlessS Spotless 1.06, please find and use original and updated version here: https://forum.doom9.org/showthread.php?t=181777
Real.finder Restoregrain, same page
RemoveDirtMC and unsharpmask usages borrowed from VideoFred /Johnmeyer found on the above footage restoring thread.

#~ Artifacts when sharpening after restoring grain, hence sharpening before restoring grain.

As this script will be used at the end of the workflow, I thought it was not a problem to convert all to 8bits 4:2:0 as I didn't find other way with unsharpmask.
...But if you know a way to keep 10bit 4:2:2, please share.
*/

SetMemoryMax(2000)
threads = 16
SetFilterMTMode("DEFAULT_MT_MODE", 2)
Loadplugin("D:\somepath\RemoveDirt.dll")
Loadplugin("D:\somepath\RgTools.dll")
Loadplugin("D:\somepath\warpsharp.dll")

source1= AviSource("D:\davinci_eric_folder\rushes_cathy_new_system\458_graded-trim-crop_for_tests.avi")
source1= KillAudio(source1)
source1 = ConvertTo8bit(source1)
source1 = ConvertToYV12(source1)

BlkSz = 12
OLap = 4
Pel = 2
Tm=false
Bblur = 0.6
ThSAD = 5000
ThSAD2 = 5000
RadT = 2

Mytext="BlkSz=12,Olap=4,Pel=2,Tm=False,Bblur=0.6,Thsad2=Thsad=5000,RadT=2,Dirt_str=60,Us_str=120, Us_rad=5, Us_th=1, Rest_gr_str2=20"

dirt_strength=60

Usharp_strength= 120
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#~ Sharp_Strength = 0.9

txt_sz = 40

spotless_out=SpotLess(source1,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,thsad2=thsad2,RadT=RadT)
rd_out = RemoveDirtMC(source1,dirt_strength,false)
spotless_Rd = RemoveDirtMC(spotless_out,dirt_strength,false)

#Sharpen----------------------------------------------------------------------
spotless_Rd_Sh=spotless_Rd.unsharpmask(Usharp_strength,Usharp_radius,Usharp_th)
#~ spotless_Rd_Sh =spotless_Rd_Sh.sharpen(Sharp_Strength)

# Restore grain -------------------------------------------------------------
rd = spotless_Rd_Sh
orig = source1
spotless_Rd_Sh_Rg = mRD_RestoreGrain(rd, orig)
#----------------------------------------------------------------------------------

group = StackHorizontal(\
subtitle(source1,"Original",size = txt_sz,align=2),\
subtitle(spotless_out,"SpotlesS",size= txt_sz,align=2),\
subtitle(rd_out,"RemoveDirtMC",size= txt_sz,align=2),\
subtitle(spotless_Rd,"SpotlesS+RDMC",size = txt_sz,align=2),\
subtitle(spotless_Rd_Sh,"SlesS+RDMC+Sharp",size = txt_sz,align=2),\
subtitle(spotless_Rd_Sh_Rg,"Sl+RDMC+Sh+RGrain",size = txt_sz,align=2),\
subtitle(source1,"Original",size = txt_sz,align=2))

subtitle(group,Mytext,size = txt_sz,spc=4,align=7)

#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {
sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 20.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

/*
SpotLess v1.06. temporal spot/noise remover, by StainlessS @ Doom9. https://forum.doom9.org/showthread.php?t=181777
Original idea from Didée post :- https://forum.doom9.org/showthread.php?p=1402690#post1402690
Req:- Pinterf MvTools2(), Pinterf Medianblur2() with MedianBlurTemporal rather than the MedianBlurT() from Didée post.
With appropriate plugins, will be AVS+ colorspace and x64 compatible.
Fine using Avs+ Prefetch() so long as current Pinterf plugins, and Frame Accurate source. RequestLinear() following the source filter might suffice as frame accurate source filter.

NOT FOR cartoon/anime, live video only, sorry.

v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
v1.02, Add some stuff.
v1.03, Frame size auto config of args removed (found instances where caused problems). Added Glob and bBlur args.
v1.04, Script breaking changes, I guess is more flexible if can also set ThSAD, inserted ThSAD 3rd arg. RadT now default 1, was 2, dont over denoise unless requested.
v1.05, Additional checks on args.
v1.06, Glob Default true, Almost always better.

SpotLess(clip c,int "RadT"=1,int "ThSAD"=10000,int "ThSAD2"=ThSAD,int "pel"=2,bool "chroma"=true, int "BlkSz"=8,Int "Olap"=BlkSz/2,
\ bool "tm"=true,Bool "glob"=True,Float "bBlur"=0.0)

RadT, 1 or more, Default 1. Removes Spots on up to RadT [Temporal Radius] consecutive frames.
RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

ThSAD, Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences].
ThSAD and ThSAD2 suggested absolute minimum of maybe about 400.
ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
in the result MCompensated frame].
Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
or some other mixture.
A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
middle pixel value.
I've just tested real bad double frame, full frame luma and chroma corruption, with below line:
SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
as we still had the temporal median filter to fall back on and pick the middle pixel value.

From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

ThSAD2, Default ThSAD, 0 < ThSAD2 < 16320(8*8*255), Lower removes fewer spots, but less chance of blurring.
ThSAD2 sets the SAD [Sum of Absolute Differences] threshold for most distant frame from current_frame at distance RadT, with those frames that are distances in-between 1 and RadT
acquiring a SAD threshold linearly interpolated between the two.
From mvtools2 docs:
ThSAD2:
Defines the SAD soft threshold for the furthest frames at current_frame +- RadT.
The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame)
and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
EDIT: Although I have said that SAD threshold being too high could result in blurred frames, that is really taken from above "risk of bluring" line from mvtools docs,
however, that warning says "temporal denoising filter", which might suggest pixel averaging, whereas we are using pixel median. I'm not sure that blurring would be the result
of having too high a SAD threshold.

Pel, Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)

Chroma, Default True. MAnalyse chroma arg. If true, use chroma in block matching when creating vectors. Maybe use False if source B&W or color noise.

BlkSz, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]

OLap, Default half BlkSz, Block overlap.

Tm, TrueMotion Default True. Some folk swear truemotion=false is better.

Glob, Default True (True v1.06, was same as Tm, true almost always better), Allow set MAnalyse(global) independently of TrueMotion.
From MvTools2 docs for MAnalyse,
global
Estimate global motion (at every level) and use it as an additional predictor.
Only pan shift is estimated (no zoom and rotation).
Use false to disable, use true to enable.

bBlur, Default 0.0 [OFF]. If used, Suggest about 0.6, where MAnalyse create vectors is performed on denoised (blurred) super clip.
*/

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,False) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
Bblur = Default(bblur,0.0) # Default OFF
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (bBlur<=0.0 ? c : c.blur(bblur)).MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2)
sup_rend = (bBlur<=0.0) ? sup : c.MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2,levels=1) # Only 1 Level required where not MAnalyse-ing.
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}
function RemoveDirtMC(clip,int "limit", bool "_grey")
{
_grey=default(_grey, false)
limit = default(limit,6)

# Alternative way to do pre-filtering.
# prefiltered = fft3dfilter(clip,sigma=1,sigma2=2,sigma3=3,sigma4=5,bw=64,bh=64)

prefiltered = RemoveGrain(clip,2)
superfilt = MSuper(prefiltered, hpad=32, vpad=32,pel=2)

super=MSuper(clip, hpad=32, vpad=32,pel=2)

bvec = MAnalyse(superfilt,isb=true, blksize=16, overlap=2,delta=1, truemotion=true)
fvec = MAnalyse(superfilt,isb=false, blksize=16, overlap=2,delta=1, truemotion=true)

# Increase thSAD if moving objects are being removed
bvec_re = Mrecalculate(super,bvec,blksize=8, overlap=0,thSAD=100)
fvec_re = Mrecalculate(super,fvec,blksize=8, overlap=0,thSAD=100)

backw = MFlow(clip,super,bvec_re)
forw = MFlow(clip,super,fvec_re)

clp=interleave(forw,clip,backw)
clp=clp.RemoveDirt(limit,_grey)
clp=clp.SelectEvery(3,1)
return clp
}
function RemoveDirt(clip input, int "limit", bool "_grey")
{
clensed=input.Clense(grey=_grey, cache=4)
alt=input.RemoveGrain(2)
return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,dmode=2,debug=false,noise=limit,noisy=4, grey=_grey)

# Alternative settings
# return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=4,cthreshold=6, gmthreshold=40,dist=1,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=true)
# return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,tolerance= 12,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=false)
}
Prefetch(threads)

real.finder

24th May 2021, 14:12

Is there a way to stay in 4:2:2 10 bit with unsharpmask or what is the best conversion to feed it?
.

https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/UnsharpMask_avsi.avsi (there are dogway edit of it too if you like)

also you may like RemoveDirtMC_SE instead of RemoveDirtMC https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/RemoveDirtMC_SE.avsi

chmars

24th May 2021, 15:43

https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/UnsharpMask_avsi.avsi (there are dogway edit of it too if you like)

also you may like RemoveDirtMC_SE instead of RemoveDirtMC https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/RemoveDirtMC_SE.avsi
Thanks.
Trying UnsharpMask_avsi.avsi. Txt file in plugins... ok.
But Avspmod says "no function spasses". Google, doom search, wiki...no track of something called spasses.

real.finder

24th May 2021, 15:45

Thanks.
Trying UnsharpMask_avsi.avsi. Txt file in plugins... ok.
But Avspmod says "no function spasses". Google, doom search, wiki...no track of something called spasses.

https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/Zs_RF_Shared.avsi

StainlessS

24th May 2021, 15:51

Must be called "Zs_RF_Shared.avsi" in plugins, ie not a *.txt file.

EDIT: I mean, "UnsharpMask.avsi"
and not "UnsharpMask_avsi.avsi"

EDIT: And also, turn off that brain dead Windows Explorer (file manager) Folder options,
"Hide Extensions for known file Types".

Maybe you created a text file, and changed extensions to .avsi, but it is still really a .txt file
due to that crazy option being selected.
(Right click and select Properties to see real extension, even with that messed up option selected)

real.finder

24th May 2021, 16:00

EDIT: And also, turn off that brain dead Windows Explorer (file manager) Folder options,
"Hide Extensions for known file Types".

Maybe you created a text file, and changed extensions to .avsi, but it is still really a .txt file
due to that crazy option being selected.
(Right click and select Properties to see real extension, even with that messed up option selected)

or, just right click on "Raw"

https://i.postimg.cc/ryNBGF85/Untitled.png (https://postimages.org/)

then save as :D

chmars

24th May 2021, 16:12

Tanks, no, no worry, I meant "text type avsi file (actually text file with avsi ext)"
I desactivate all pseudo-simplifying assisting options, even Windows if It wouldn't have avisynth!
I never tried right click in Github. Cool.
RemoveDirtMC_SE working with 4:2:2/10b so far :cool:
EDIT: RemoveDirtMC_SE too.

StainlessS

24th May 2021, 16:36

or, just right click on "Raw"
then save as :D
Damn, I knew there must be an easier method than copy/paste filename from page into new text file name, and then
Go RAW, and then CTRL A, copy all, and paste into text file.

Thats why R.F. gets the big bucks :)

EDIT: SomeScript.avsi.txt problems have happened before on-site.

real.finder

24th May 2021, 17:01

EDIT: SomeScript.avsi.txt problems have happened before on-site.

yes, in pastebin

kedautinh12

24th May 2021, 17:14

Thanks for your script chmars

chmars

24th May 2021, 20:48

Thanks for your script chmars

Verify by yourself though. I noticed my OLap = 4 ( with Blksz 12 ) was too low. It is making stairs in some shadowed 45° diagonals.
Very rarely however. Only when something appears from one frame to the next one. Like the cigarette thing maybe, in the Spotless thread.
One might think it is a detail but I saw it from far, without searching.
As I have big dirt spots, I believe I have to keep Blksz high. So, a better value for OLap is 6.

https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/hGUUXVRghJXH7mj/download

RemoveDirtMC_SE is efficient but seems to me "eating more grain" than Spotless even when changing repmode and clmode.
I might keep Spotless only, without restoring grain (and some dirt neither).

I also notice how uggly-noisy can grain be (1st pic, right leg)

StainlessS

24th May 2021, 21:08

Ideally BlkSz should be a fair bit bigger than spots, spot=8x8 then blksz=16x16, or bigger.
You want blocks to be able to match good [non spot] pixels in a block, and to outnumber the spot pixels.
and maybe Olap at about half blksz.

chmars

24th May 2021, 22:22

Hmm, ok, so if I go higher with BlkSz, I will not necessarily increase the stair effect.
You want blocks to be able to match good [non spot] pixels in a block
Are there a defined blocks of pixels in images, blocks that BlkSz has to override/match to?:o

johnmeyer

24th May 2021, 23:34

StainlessS

25th May 2021, 00:09

If your spots are on only single frames (and as film, I'm guessin' that is true) then suggest RadT=1.
(Use RadT= number of consecutive frames spots can 'infect').

Also, as HD frames, maybe at least blkSz=16, Olap=16/2, or
BlkSz=32, OLap=32/2.

Also, I suggest the beta Spotless, about last post in the Spotless Thread.
[bBlur =0.6 by default, or can use dc=MyPrefilteredDenoisedClip)
Also, I may change default TM=false, many peoples favorite.

https://forum.doom9.org/showthread.php?p=1939724#post1939724

GMJCZP

25th May 2021, 04:07

I would not want to cause a headache to chmars but it would be interesting to test Killerspots to see what its performance is here, it is not a competition because SpotLess is working wonderfully, what happens is that I have only exhaustively tested KillerSpots in anime.

StainlessS

25th May 2021, 04:18

Cant hurt to try out KillerSpots.
[see GMJCZP Flea Market, in his sig above]

chmars

25th May 2021, 08:25

chmars,

Those are really useful, and very well-done comparisons. I guess I need to roll up my sleeves and try substituting SpotlessS (my spelling) for RemoveDirtMC in my various film restoration scripts.

I have backed away from using too much remove grain, mostly because I often end up with "gooey" video where everything looks one frame is melting into another, like a Salivdor Dali film.
...

Thaank you, yes, I noticed Spotlesss seems to be more adapted to restoration as the result is effective in dirt removing but still, the output is sensibly less "digital" than RdMC.

...
You can see this in my attempt to restore Polavision film. Because of the massive chemical stains that can only be removed digitally, I had the RemoveDirtMC settings turned up to five times the normal values. At roughly the thirty second mark (the toy wagon on the sidewalk) you'll see it in the street behind the boy. The results at the 1:10 mark are even worse. Look at the wall behind the boy pushing the toy car.
https://www.youtube.com/watch?v=ud1XNIoh1P8

I remember this video from "the long long thread". Very headache footage!
Nevertheless, considering the source, result is good. The phenomenon you describe is also very noticeable in the last scene, like fixed steady grain.
I sometimes note the sharpening. Did you use more sharpen than unsharpmask?
Or maybe it is because of youtube compression.

Therefore, the other test you need to do before to compare SpotlessS with RemoveDirtMC is to look at the results in motion. Individual frames are useful, but with temporal filters you often don't spot filter artifacts until you put everything in motion.
Sure, I did take count of motion in the previous post with Spotless, RDMC compared (https://forum.doom9.org/showthread.php?p=1943433#post1943433).
The pant image triple comparison was taken from the same footage and, effectively, the phenomenon is barely noticeable when the video is running.

If your spots are on only single frames (and as film, I'm guessin' that is true) then suggest RadT=1.
(Use RadT= number of consecutive frames spots can 'infect').

Oh Yes, thanks, forgot to put it back. I tried RadT=2 for these long spots that seemed to be like liquid/chemical, during maybe more than one frame.

Also, as HD frames, maybe at least blkSz=16, Olap=16/2, or
BlkSz=32, OLap=32/2.
Also, I suggest the beta Spotless, about last post in the Spotless Thread.
https://forum.doom9.org/showthread.php?p=1939724#post1939724[/url]
Ok, excellent, I'll try.
I would not want to cause a headache to chmars but it would be interesting to test Killerspots to see what its performance is here, it is not a competition because SpotLess is working wonderfully, what happens is that I have only exhaustively tested KillerSpots in anime.Cant hurt to try out KillerSpots.
[see GMJCZP [B]Flea Market, in his sig above]
Thank you for the suggestion. If I may contradict: headache hurts. :)
I maybe keep it for later but wanna tweak Spotless some more before to test something else.
It is for me very very difficult to pull an objective opinion from such comparisons.
I mean, if you want to know the "truth", it is a lot of work, notes taking with rigor and severity. Hard to get a reliable result. Each filter has lots of params. I am afraid to get lost finally.
Also, I'll assume some irrational reason to stay with Spotless ;)

StainlessS

25th May 2021, 14:45

If I may contradict: headache hurts. :)
Yes indeed :)

KillerSpots is near to Spotless(RadT=1), but also has an Advanced mode, which limits pixel changes. [but that may also prevent total spot removal - might be ok on 'transparent spots'].

RadT=1, does median on 3 frames [2 * RadT + 1], so if 1 frame spots, then 2 frames without spots outnumber spot frames.
So, RadT=ConsecutiveSpotFrames is easy to remember.

chmars

25th May 2021, 21:33

Compared Spotless 1.06 and 1.07, same parameters, including Bblur 0.6.
No differences.
I decided to search for code differences.
I should have done before, not after!
The difference is the ability toprovide a prefiltered clip (HasDC).

Some weeks ago, I didn't know Avisynth exists. (and believed my life was fullfilled).
Diving one more step deeper to discover what is a prefiltered clip is becoming dangerous :p

But I noticed other things:

If your spots are on only single frames (and as film, I'm guessin' that is true) then suggest RadT=1.
Yesno. Few spots (liquid/chem traces alike) last longer than 1 frame. Helps in that case. But RadT increased is slower I guess. Up to one decide if it's worth it.

Spotless is perfect and I am totally happy with its use and can live with it as it is and enjoy all others aspects of life.

But, as professional in perfectionism-maniacism:
I am trying to find a way to get details back. Like background moving water, trees.
I tried to reduce ThSAD until these details came back. But spots came back far before details. Didn't find the use of reducing it, keepin'it at 10000 ATM.
Then, BlkSz. As details are small, smaller than hairs, spots, there is no use of reducing BlkSz, I believe.

Some ideas to conserve details into water, trees, grass...:
-Ability to set a limit in the amount of removed spots by certain surface (eg: max 3 spots each 100x100 pixel), or (eg: 5 small spots, 2 medium spots, 1 big spot)
-ability to define a minimum color difference between spots and near environment, difference underneath which, spot is not removed (eg: don't remove a dark-blue wave in blue water)
and/or
-ability to select colors where the spot removal is less active (ex: don't remove somuch spots into blue-green water, green trees...)

Also, I finally found a ball in the footage, to see how it disappears while flying.
-We could tell Spotless "simply" to not remove light objects (grey, white...). After all, 90% of dirt is dark.

Well...Maybe these ideas have been discussed 1000 times in dirt removal threads, I apologize if none of them are new, because me, I am (new)

EDIT: Or, we could try to restore more or less grain, depending on the color. Not bad, maybe, as remove grain doesn't bring so much dirt back.

johnmeyer

25th May 2021, 23:23

... We could tell Spotless "simply" to not remove light objects (grey, white...). After all, 90% of dirt is dark.The "only white" or "only dark" setting in Despot were quite useful, back when that was all that was available. Video dropouts tend to be near-white and lacking color and, as chmars says, print film dirt tends to be near-black, and also lacking in any color. Adding chroma and luma threshold settings could make a big difference in reducing false positives such as removing buttons on shirts, something that happens quite often with most spot removal algorithms.

joka

26th May 2021, 07:27

It is always a compromise between removing dirt and keeping moving object / removing grain and keeping details+sharpness.

What are the relevant characteristics to decide (remove vs. keep)?

- black / white dirt
- size of dirt

StainlessS

26th May 2021, 13:16

DeSpot detects spots, and can selectively choose color of spot to remove.
Indeed, with blur=false [or whatever the blur arg is], it only affects those pixels where spots
are identified, does not do frame global changes [only does (TEMPORAL I think) blur if blur arg is true].
So, if multiple passes to remove different size/color spots, then dont ever use the blur arg=true as it will
spread spots temporally, for the next pass, making impossible to detect/remove next color/size spots.
Only blur [if absolutely sure] on final pass.

Spotless does not detect spots at all, and so white/black spot detect aint ever gonna happen.
Spotless creates mutliple candidate frames via MC, and for each pixel finds the median corresponding pixel
of those candidate frames, thats it. Spotless frame has global changes, not just where there are spots.

EDIT: If a block from a candidate frame is too diiferent to the source frame block (by ThSAD,ThSAD2), then it uses the source frame pixels instead <in the median thing>, (which could be good if source pixel is a good pixel, or bad if source pixel is a bad pixel. [So setting ThSAD/ThSAD2 too low would slant towards
using the source pixel <good or bad>, and if set high, then relies more on median getting it right.]
I would tend towards having ThSAD,ThSAD2 a bit high, and BlkSz sufficiently big so that spot pixels are outnumbered by good pixel matches in SAD, and let the median thing do its stuff.

EDIT: Despot arg might be tblur, or blurt [or similar].

chmars

27th May 2021, 09:54

Too bad. I was very satisfied with this grain/spots/sharpness compromise:

https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/rM8k1niecdzpZ7z/download (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/AaXChavgYw6Mql4/download)
Video (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/AaXChavgYw6Mql4/download)

But I had some balloons and fast moving people on a new old footage and some ended throwing nothing in the air with no hands -not very convenient. (I knew it was the risk though)

So I am gonna follow these advices:
The "only white" or "only dark" setting in Despot were quite useful, back when that was all that was available.
...
... interesting to test Killerspots...
Cant hurt to try out KillerSpots.
DeSpot detects spots, and can selectively choose color of spot to remove.

...

Maybe I begin with Despot as there is a possibility to restrict color or darkness of the spot.

EDIT: @John, what do you mean by: " back when that was all that was available"?

Sharc

27th May 2021, 10:17

But I had some balloons and fast moving people on a new old footage and some ended throwing nothing in the air with no hands -not very convenient. (I knew it was the risk though)

Was it persistent (i.e. spanning a couple of frames) or only a single frame? Would people notice when watching real-time?

chmars

27th May 2021, 11:09

-not very convenient.
(Ment: playing ball with no ball, not the filter)

Was it persistent (i.e. spanning a couple of frames) or only a single frame? Would people notice when watching real-time?

(bad graded, sorry: )
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/j4kJnzXQADSCKpx/download (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/UYnpov4jMtHG1va/download)
Video (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/UYnpov4jMtHG1va/download)

I guess it is subjective. Of course 80% will not notice it. But for one who does it for job, it is delicate, I guess.
.

johnmeyer

28th May 2021, 01:39

EDIT: @John, what do you mean by: " back when that was all that was available"?It was first released eighteen years ago, before most of the other transient removal plugins developed.

kedautinh12

28th May 2021, 02:13

(Ment: playing ball with no ball, not the filter)

(bad graded, sorry: )
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/j4kJnzXQADSCKpx/download (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/UYnpov4jMtHG1va/download)
Video (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/UYnpov4jMtHG1va/download)

I guess it is subjective. Of course 80% will not notice it. But for one who does it for job, it is delicate, I guess.
.

If spotless detroy many details, you can use SpotRemover with less detail detroyed

chmars

28th May 2021, 21:35

Hi,

@kedautinh12, thanks, here it is.
@John ok, I misunderstood, thought color option wasn't available anymore.

This is my new game of 7 differences :)
This is a new test I made, still to find the good path to:
->remove dirt, sharpen, keep some grain.

This time, as kindly advised by members, I tried SpotRemover:
-Original
-Spotless + sharpening + restore grain
-Spotremover (alone, to not distort results)

Spotless works still as good as before.

For SpotRemover, I think I didn't find the good settings to remove dirt as good. No moving objects were removed though.

One difference, where?
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/J0d5I6NfKs37dG5/download (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/3uHbifQDt0Ma3RA)
The bottom left ballon.

Spotless likes to eat balloons, hands but not so much trains, neither moving logos:
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/apps/files_sharing/ajax/publicpreview.php?x=1920&y=465&a=true&file=459_s2_002-XAVCI-4K-BLD_5_0-trim_crop-spotless_sharp_restoregrain-SpotRemover_comp.pic2.jpeg&t=ZJG5sgEHBvPNAVA&scalingup=0 (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/lCk8EI9iTENqaWN/download)

Rarely tough, trains logos and train chimneys are too temptating:
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/apps/files_sharing/ajax/publicpreview.php?x=1920&y=465&a=true&file=459_s2_002-XAVCI-4K-BLD_5_0-trim_crop-spotless_sharp_restoregrain-SpotRemover_comp.pic3.jpeg&t=SjSDR0GgbIZB6K3&scalingup=0
(https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/FrCbs938lQI8N5D/download)

Test video here (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/luESRWEyUSbB7bm/download)
Source video here (EDIT: original, avi, huge) (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/U2MK5Q1woX7NF5f/download)

I was very interested to look at the test frame by frame.
I noticed that Spotless is really eating only really fast moving objects. Some disappear, some become squared. (baloons, people in the train, chimneys, road signal behind the train)

But in terms of spots removing , I couldn't get good results as easily with the others cleaners I tested. (RemoveDirtMC, Despot).
I don't say I made extensive tests neither I understood all the parameters but I however read, tried and spend time to change parameters. Perhaps, they might work better for experienced users but for noobs like me, more cryptic.
Killerspots, I didn't try. The temporal dimension of Spotless makes me prefer it.

If no one has a suggestion I will not resist to test, atm, my conclusion is that Spotless is the easiest + more efficient solution. Its output combined to unsharpmask and restoring grain gives a result which is, in my taste, a perfect compromise between digital cleaning and analogic feeling, just splendid (when i'll be able to grade it correctly...)!

About eaten objects: It does not happen so often. So I will put both unfiltered and filtered videos on the timeline and apply transparency to bring back moving objects when necessary.

I want to thank members for the plenty of suggestions, and time taken for explanations, which permit to make my way a bit and learn.

chmars

28th May 2021, 21:46

I Forgot the code used for the test:
Can't exclude I missed something about SpotRemover...

source1= AviSource("D:\somepath\459_s2_002-XAVCI-4K-BLD_5_0-trim_crop-for_tests-graded.avi")
source1= KillAudio(source1)
source1 = crop(source1,700,0,-500,0)

source1_8bit = ConvertBits(source1, bits=8) #I dont'know much about formats, maybe this is not ideal...
source1YV12 = ConvertToYV12 (source1_8bit) #I dont'know much about formats, maybe this is not ideal...

txt_sz = 36

BlkSz =32
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Remove spots-----------------------------------------------------------------
spotless_107=SpotLess107(source1,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
SpotRemoved = SpotRemover(source1YV12,Spot=128, RadT=1, Pel=Pel, BlkSz=32, DeGrain=false, Chroma=true, PreFilter=true, mask=false, Tm = true,thsad=thsad, Glob = true)
#----------------------------------------------------------------------------------

SpotRemoved = ConvertBits(SpotRemoved, bits=10) #I dont'know much about formats, maybe this is not ideal...
SpotRemoved = ConvertToYUV422(SpotRemoved) #I dont'know much about formats, maybe this is not ideal...

#Sharpen----------------------------------------------------------------------
spotless_107_shp=spotless_107.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
SpotRemoved_shp=SpotRemoved.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#----------------------------------------------------------------------------------

# Restore grain -------------------------------------------------------------
spotless_107_shp_Rg = mRD_RestoreGrain(spotless_107_shp, source1)
SpotRemoved_shp_Rg = mRD_RestoreGrain(SpotRemoved_shp, source1)
#----------------------------------------------------------------------------------

group = StackHorizontal(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(spotless_107_shp_Rg,"spotless_107_shp_Rg",size = txt_sz,align=2),\
subtitle(SpotRemoved,"SpotRemover",size = txt_sz,align=2))

return(group)#.AddBorders(1000, 0, 500, 0, color=$000000)

#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

Function SpotLess107(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess107: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

Function SpotRemover(clip c, int "Spot", bool "DeGrain", int "RadT", int "ThSAD", int "ThSAD2", int "Pel", bool "Chroma",
\ int "BlkSz", Int "Olap", bool "Tm", bool "Glob", bool "PreFilter", bool "Mask", string "Denoiser") {
Spot = Default(Spot, 32) # Spot size
DeGrain = Default(DeGrain, false) # MDeGrain in MvTools
RadT = Default(RadT, 1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD, 1000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2, 130) # SAD threshold at radius RadT.
Pel = Default(pel, 1) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma, true) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz, 8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, (Blksz/2)) # Default half of BlkSz. #################################################################
Tm = Default(tm, true) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob, true) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
PreFilter = Default(PreFilter, true)
Mask = Default(Mask, false)
Denoiser = Default(Denoiser, Undefined)

pad = Max(BlkSz, 8)

sup = c.mt_lut(expr="x 20 1 x 127.5 / - 21 ^ pi x * 255 / cos 21 ^ - * 1.05 * -", Y=3, U=2, V=2)
\ .RemoveGrain(mode=17).MSuper(hpad=pad, vpad=pad, pel=pel, sharp=2, chroma=Chroma)
sup_rend = MSuper(c, hpad=pad, vpad=pad, pel=pel, sharp=2, levels=1, chroma=Chroma) # Only 1 Level required where not MAnalyse-ing.

c1 = PreFilter ? Eval("""
MultiVec = MAnalyse(sup, multi=true, delta=RadT, blksize=BlkSz*2, overlap=OLap*2, search=5, dct=0, chroma=Chroma, truemotion=Tm, global=Glob)
c0 = MCompensate(c, sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
c0 = RadT == 1 ? Clense(c0) : MedianBlurTemporal(c0, radiusY=0, radiusU=0, radiusV=0, temporalradius=RadT)
c0 = SelectEvery(c0, RadT*2+1, RadT) # Return middle frame
return Repair(c0, c.RemoveDust().Sharpen(0.1), mode=17)
""") : c

bvec1 = MAnalyse(sup, isb=true, blksize=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0,
\ chroma=Chroma, truemotion=Tm, global=Glob) # backward vectors
fvec1 = MAnalyse(sup, isb=false, blksize=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0,
\ chroma=Chroma, truemotion=Tm, global=Glob) # forward vectors
backw1 = MFlow(c1, sup_rend, bvec1)
forw1 = MFlow(c1, sup_rend, fvec1)

c2 = Defined(Denoiser) ? Interleave(backw1, c1, forw1).Eval(Denoiser) : Interleave(backw1, c1, forw1)

msk = Mask ? Eval("""
sadb = MMask(c, bvec1, ml=50, kind=1, gamma=1, thscd1=ThSAD)#.mt_binarize(15, upper=true) # mask SAD backward
sadf = MMask(c, fvec1, ml=50, kind=1, gamma=1, thscd1=ThSAD)#.mt_binarize(15, upper=true) # mask SAD forward
msad = mt_logic(sadf, sadb, "and", chroma="-127.5")#.mt_expand() # combined mask SAD
return Interleave(msad, msad, msad)
""") : Undefined

filtered = c2.DeSpot(p1=6, p2=6, pwidth=Spot, pheight=Spot, mthres=16, mwidth=Spot/2, mheight=Spot/2,
\ blur=1, motpn=true, seg=2, color=true, ranked=true, show=0, extmask=msk)

restore = c2.RemoveDust().Sharpen(0.1)

c3 = RestoreMotionBlocks(filtered, restore, alternative=restore, pthreshold=10, cthreshold=10, gmthreshold=160,
\ dist=3, dmode=2, noise=3, noisy=9, grey=false, debug=false, show=false)

SelectEvery(c3, 3, 1)

DeGrain ? MDeGrain1(last, sup_rend, bvec1, fvec1, plane=4, limit=255.0, thSad=400) : last
}

function RemoveDust(clip input, int "mode") {
mode = Default(mode, 17)

clensed = Clense(input)
rep = Repair(clensed, input, mode=mode)

RemoveGrain(rep)
}

zorr

30th May 2021, 02:12

I liked your idea of restricting the filtering to darkish spots. So here's a modded script that does that, based on Spotless (and this same trick will work with any other degrain filter as well):

source1= AviSource("459_s2_002-XAVCI-4K-BLD_5_0-trim_crop-for_tests.avi")
source1= KillAudio(source1)
source1 = crop(source1,700,0,-500,0)

source1_8bit = ConvertBits(source1, bits=8) #I dont'know much about formats, maybe this is not ideal...
source1YV12 = ConvertToYV12 (source1_8bit) #I dont'know much about formats, maybe this is not ideal...

txt_sz = 36

BlkSz =32
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Remove spots-----------------------------------------------------------------
spotless_107=SpotLess107(source1,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
SpotRemoved = SpotRemover(source1YV12,Spot=128, RadT=1, Pel=Pel, BlkSz=32, DeGrain=false, Chroma=true, PreFilter=true, mask=false, Tm = true,thsad=thsad, Glob = true)
#----------------------------------------------------------------------------------

SpotRemoved = ConvertBits(SpotRemoved, bits=10) #I dont'know much about formats, maybe this is not ideal...
SpotRemoved = ConvertToYUV422(SpotRemoved) #I dont'know much about formats, maybe this is not ideal...

#Sharpen----------------------------------------------------------------------
spotless_107_shp=spotless_107.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
SpotRemoved_shp=SpotRemoved.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#----------------------------------------------------------------------------------

# Restore grain -------------------------------------------------------------
spotless_107_shp_Rg = mRD_RestoreGrain(spotless_107_shp, source1)
SpotRemoved_shp_Rg = mRD_RestoreGrain(SpotRemoved_shp, source1)
#----------------------------------------------------------------------------------

# diff ----------------------------------------------------------------------

diff = Grayscale(mt_makediff(spotless_107_shp, source1))

#----------------------------------------------------------------------------

# white mask ----------------------------------------------------------------

threshold = 0.9 # adjust threshold to change how bright objects can get removed
whitemask = mt_lut(diff, expr="x range_half "+String(threshold)+" * > 255 scalef 0 ?")
whitemask_orig = whitemask

# fill small mask holes -----------------------------------------------------

filltimes = 1 # increase filltimes to remove larger holes from the mask
whitemask = whitemask.mt_invert()
whitemask_inverted = whitemask
for (i = 1, filltimes) {
whitemask = whitemask.mt_inpand()
}
whitemask = mt_hysteresis(whitemask, whitemask_inverted)
whitemask = whitemask.mt_invert(chroma="-127")

# enlarge mask (optional) ---------------------------------------------------

# experiment with mt_inpand and/or mt_deflate to restore more of the removed object
#whitemask = whitemask.mt_inpand(chroma="-127")
#whitemask = whitemask.mt_deflate(chroma="-127")

#----------------------------------------------------------------------------

# whitemask overlay ---------------------------------------------------------

overlay = Overlay(source1, spotless_107_shp, mask=whitemask)

#----------------------------------------------------------------------------

group = StackHorizontal(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(spotless_107_shp,"spotless_107_shp",size = txt_sz,align=2),\
subtitle(overlay,"overlay",size = txt_sz,align=2),\
subtitle(whitemask,"filled whitemask",size = txt_sz,align=2),\
subtitle(whitemask_orig,"orig. whitemask",size = txt_sz,align=2),\
subtitle(diff,"diff",size = txt_sz,align=2))
#subtitle(SpotRemoved,"SpotRemover",size = txt_sz,align=2))
#subtitle(spotless_107_shp_Rg,"spotless_107_shp_Rg",size = txt_sz,align=2),\

return(group)#.AddBorders(1000, 0, 500, 0, color=$000000)

#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

Function SpotLess107(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess107: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

Function SpotRemover(clip c, int "Spot", bool "DeGrain", int "RadT", int "ThSAD", int "ThSAD2", int "Pel", bool "Chroma",
\ int "BlkSz", Int "Olap", bool "Tm", bool "Glob", bool "PreFilter", bool "Mask", string "Denoiser") {
Spot = Default(Spot, 32) # Spot size
DeGrain = Default(DeGrain, false) # MDeGrain in MvTools
RadT = Default(RadT, 1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD, 1000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2, 130) # SAD threshold at radius RadT.
Pel = Default(pel, 1) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma, true) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz, 8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, (Blksz/2)) # Default half of BlkSz. #################################################################
Tm = Default(tm, true) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob, true) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
PreFilter = Default(PreFilter, true)
Mask = Default(Mask, false)
Denoiser = Default(Denoiser, Undefined)

pad = Max(BlkSz, 8)

sup = c.mt_lut(expr="x 20 1 x 127.5 / - 21 ^ pi x * 255 / cos 21 ^ - * 1.05 * -", Y=3, U=2, V=2)
\ .RemoveGrain(mode=17).MSuper(hpad=pad, vpad=pad, pel=pel, sharp=2, chroma=Chroma)
sup_rend = MSuper(c, hpad=pad, vpad=pad, pel=pel, sharp=2, levels=1, chroma=Chroma) # Only 1 Level required where not MAnalyse-ing.

c1 = PreFilter ? Eval("""
MultiVec = MAnalyse(sup, multi=true, delta=RadT, blksize=BlkSz*2, overlap=OLap*2, search=5, dct=0, chroma=Chroma, truemotion=Tm, global=Glob)
c0 = MCompensate(c, sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
c0 = RadT == 1 ? Clense(c0) : MedianBlurTemporal(c0, radiusY=0, radiusU=0, radiusV=0, temporalradius=RadT)
c0 = SelectEvery(c0, RadT*2+1, RadT) # Return middle frame
return Repair(c0, c.RemoveDust().Sharpen(0.1), mode=17)
""") : c

bvec1 = MAnalyse(sup, isb=true, blksize=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0,
\ chroma=Chroma, truemotion=Tm, global=Glob) # backward vectors
fvec1 = MAnalyse(sup, isb=false, blksize=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0,
\ chroma=Chroma, truemotion=Tm, global=Glob) # forward vectors
backw1 = MFlow(c1, sup_rend, bvec1)
forw1 = MFlow(c1, sup_rend, fvec1)

c2 = Defined(Denoiser) ? Interleave(backw1, c1, forw1).Eval(Denoiser) : Interleave(backw1, c1, forw1)

msk = Mask ? Eval("""
sadb = MMask(c, bvec1, ml=50, kind=1, gamma=1, thscd1=ThSAD)#.mt_binarize(15, upper=true) # mask SAD backward
sadf = MMask(c, fvec1, ml=50, kind=1, gamma=1, thscd1=ThSAD)#.mt_binarize(15, upper=true) # mask SAD forward
msad = mt_logic(sadf, sadb, "and", chroma="-127.5")#.mt_expand() # combined mask SAD
return Interleave(msad, msad, msad)
""") : Undefined

filtered = c2.DeSpot(p1=6, p2=6, pwidth=Spot, pheight=Spot, mthres=16, mwidth=Spot/2, mheight=Spot/2,
\ blur=1, motpn=true, seg=2, color=true, ranked=true, show=0, extmask=msk)

restore = c2.RemoveDust().Sharpen(0.1)

c3 = RestoreMotionBlocks(filtered, restore, alternative=restore, pthreshold=10, cthreshold=10, gmthreshold=160,
\ dist=3, dmode=2, noise=3, noisy=9, grey=false, debug=false, show=false)

SelectEvery(c3, 3, 1)

DeGrain ? MDeGrain1(last, sup_rend, bvec1, fvec1, plane=4, limit=255.0, thSad=400) : last
}

function RemoveDust(clip input, int "mode") {
mode = Default(mode, 17)

clensed = Clense(input)
rep = Repair(clensed, input, mode=mode)

RemoveGrain(rep)
}

Basically we just compare the filtered image to the original and if something too white (and large enough) was removed we take that part from the original clip.

You can adjust how bright objects need to be in order to be restored and some other minor adjustments, there are comments in the code.
The script displays the mask that is used to restore missing features, it's easy to spot the differences from that (there's a lot of stuff in the train scene).

Here's the train frame where Spotless ate a logo.
https://i.postimg.cc/pXfjBX2f/chmars-00.png

It also restores the missing balls at the beach scene.
https://i.postimg.cc/Gm9ynPFQ/chmars-01.png

kedautinh12

30th May 2021, 03:33

Thanks

johnmeyer

30th May 2021, 03:55

Zorr, that looks really good.

chmars

30th May 2021, 07:57

Whaaa, excellent, Thanks!
What a knowlage you all have folks!

EDIT: This time I really can mark this thread as [solved], isn't it? I wouldn't have dreamt better.

zorr

30th May 2021, 21:41

I was a bit surprised myself how well that turned out. :)

About that mask, I would probably go with filltimes = 2 and then add this code for the optional mask enlargement part:

inpandtimes = 6
for (i = 1, inpandtimes) {
whitemask = whitemask.mt_inpand(chroma="-127")
}
maskblur = 3
if (maskblur > 0) {
whitemask = whitemask.FastBlur(maskblur)
}

That makes the balls fit in better, also the balls have a small shadow which is darker than what is being restored so by enlarging the mask we can restore at least some of that shadow as well. The blur makes sure that we don't get any harsh edges. Here's a gif comparison new / old mask:

https://i.postimg.cc/x83FPwJQ/chmars-mask.gif

I like to compare before/after frames using Interleave, in this case it was

return StackHorizontal(Interleave(source1, overlay), Interleave(whitemask, whitemask))

There could be situations where Spotless removes a moving object which has both bright and dark areas, in that case this script will return (mostly) just the bright areas which could look funny. If you find such a problem I could take a look at how to improve the masking to handle such cases as well.

kedautinh12

31st May 2021, 00:12

Thank zorr, you're great

Arx1meD

31st May 2021, 09:51

zorr, I experimented with masks just like you. The fast moving objects restored, but the spots also returned.

GMJCZP

31st May 2021, 15:04

zorr, great job!

I'd like to test my Killerspots script with this video to see what improvements could be made to it.

chmars

31st May 2021, 17:19

It's really really cool, Zorr!

Spotless + your script is a perfect conjunction.
I tested it for 1-2 hours on footage (not yet with last blur improvement though). Impressive!

Spotless removes the dirt (middle up left), the stick and a part of arm and finger.
Zorr Script brings the stick back but not the arm+finger.
EDIT: changed pic with anim. Gif, graded for the eyes, no sharpening at all, no grain restoration.
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/2XJ5OQ8DmDmwQoG/download

EDIT: here the values used

threshold = 1.0
filltimes = 2
No inpandtimes

I tweaked the threshold until I could recover the arm without bringing back the spot...
Like every Chrismas, wonderment feeds new dreams... :o

Doing this, I noticed light grains will come back too, on the whole image. Normal. EDIT: But super nice details eg. water surface too.
EDIT: Not sure about that phenomenon. Forget this next paragraph!
What could be done though (easy to say, I couldn't) would be to define a minimum size of compensated spots. This would avoid the return of grain/noise.
Yes yes, I said "keep grain" earlyer, "I like grain"...
I do for sure. But I wander if it is good to mix the grain return from two processes. Specially if discriminated (only light grain) on one output. This makes steps of the process unclear and hard to dose/manage.

I wrote "ZorrBack" on the picture as I had to describe what was illustrated. :D
If you dislike, I won't use that name any more.

There could be situations where Spotless removes a moving object which has both bright and dark areas, in that case this script will return (mostly) just the bright areas which could look funny. If you find such a problem I could take a look at how to improve the masking to handle such cases as well.

Actually, this is also the case of this arm, a bit. Or?
Btw, thanks too for the interleave trick! Tried to make an animated gif from Vdub but...well, you know...
.

kedautinh12

31st May 2021, 17:35

If you find out the scripts tweaked the threshold until I could recover the arm without bringing back the spot... you can share me, thanks

chmars

31st May 2021, 22:00

Here a detail of the same frame as post #60.
Graded, no sharpening at all, no grain restoration.

Original frame alternating with Spotless only (left part in post #60):
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/9D74XqZr8RpbQrl/download

Spotless+Zorr alternating with Spotless only (middle part in post #60):
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/iFWFlPgWDOy0wpb/download

Original frame alternating with Spotless+Zorr (right part in post #60):
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/EnkQqYfiLdIsX1z/download

Note how details are nicely coming back into the water.
.

chmars

31st May 2021, 23:10

If you find out the scripts tweaked the threshold until I could recover the arm without bringing back the spot... you can share me, thanks

Not sure it's about threshold. Tried every value possible.
I didn't measure the arm's brightness but seems not as dark as the spot.
However, I couldn't separate both.

Here is the short avi file (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/XxkOwIytdJvJDI1/download) if you wish to test.
The script above in this thread, no changes.

zorr

31st May 2021, 23:18

zorr, I experimented with masks just like you. The fast moving objects restored, but the spots also returned.

What kind of spots returned? If you mean bright (or brighter than average) spots then that's what the script does, it restores those. And you should not go overboard with the mask, it should only contain the moving objects and not all little spots of white, those are just mostly noise which Spotless graciously removed. :)

zorr

1st June 2021, 01:13

It's really really cool, Zorr!

Spotless + your script is a perfect conjunction.
I tested it for 1-2 hours on footage (not yet with last blur improvement though). Impressive!

Thanks for testing it, your animations are really helpful!

Zorr Script brings the stick back but not the arm+finger.

The arm+finger are mostly the same brightness or a little bit darker than the background. You can bring back that arm completely with threshold 1.4 but then your mask is all black and you have arrived at the original source with no noise removed. :)

I think I can fix this by looking at the color channels as well, the difference is much clearer there. I'll do some experimentation and report back. Thanks for the test video.

EDIT: here the values used

threshold = 1.0
filltimes = 2
No inpandtimes

threshold 1.0 is already pretty hardcore, take a look at the mask. :)

https://i.postimg.cc/9XTDgDRf/threshold1-0.png

What could be done though (easy to say, I couldn't) would be to define a minimum size of compensated spots. This would avoid the return of grain/noise.

The script already has that feature but I didn't explain it well. The filltimes is what determines how large spots will remain in the mask. Crank it up and you will very quickly lose the "grain". But it doesn't work with threshold 1.0 because then you pretty much have only one "spot" filling the whole frame. See the example below, this one is using threshold 0.9.

https://i.postimg.cc/28zK9Bcd/filltimes.gif

chmars

1st June 2021, 09:29

...
I think I can fix this by looking at the color channels as well, the difference is much clearer there. I'll do some experimentation and report back. Thanks for the test video.
...

Cool. But keep in mind I am nitpicking. Dont wanna make people loose time.
These scripts do a really great job and fulfill 98% of the task IMO. (Regrettably), no client has a chance to see that a blurred arm is missing on one frame, I believe.

threshold 1.0 is already pretty hardcore, take a look at the mask. :)

I know, yes, seen it during my tests (btw the grayscale diff is really excellent to detect/identify removed objects vs removed spots ).
I finally consider that th=1 was the good value for this scene.
It really brings back nice details, like into the water. I said "white grain", it's just more life, actually (in my taste, of course).
EDIT: example: middle zoomed pic: we can see It brings back onto the water, the reflection of the clear lines from the stone. Isn't that brilliant?

... The filltimes is what determines how large spots will remain in the mask.
... But it doesn't work with threshold 1.0 because then you pretty much have only one "spot" filling the whole frame. See the example below, this one is using threshold 0.9.

Ok, [Hit my forehead] of course, thank you for explaining more.

EDIT: Side question: Missing something: how can the variable whitemask be different than the other variable same-named whitemask?
.

Arx1meD

1st June 2021, 10:53

What kind of spots returned? If you mean bright (or brighter than average) spots then that's what the script does, it restores those. And you should not go overboard with the mask, it should only contain the moving objects and not all little spots of white, those are just mostly noise which Spotless graciously removed. :)

Yes. Light spots have returned.
I could not test your masks on moving dark objects. I don't have clips with moving dark objects.
Will dark moving objects be repaired?

chmars

1st June 2021, 13:58

Yes. Light spots have returned.
I could not test your masks on moving dark objects. I don't have clips with moving dark objects.
Will dark moving objects be repaired?

(Zorr, please correct me if I'm wrong.)
The principle of Zorr's script is to search for whatSpotless has removed. Then It can bring back everything, or only a part of it.
The criteria of decision is lightness.
The threshold sets the limit in terms of brightness, from this limit up to white, pixels will be retained to be brought back.

In addition, you can adjust the variable "filltimes" to set the size of the pixel groups you want to keep.
Then, there is "inpandtimes" which allows you to blur the brought back groups reintegrate them more discretely.

So, depending on how you set the threshold, dark objects will return. However, if they are lighter than the spots you want to remove, you have a chance.

zorr

4th June 2021, 01:18

It took some tinkering but the arm has been saved! :D

New script to play with below:

#source1= AviSource("459_s2_002-XAVCI-4K-BLD_5_0-trim_crop-for_tests.avi")
#source1 = crop(source1,700,0,-500,0)

source1= FFVideoSource("ZorrBack_test.avi")
source1 = crop(source1,450,50,1352,-40)

source1= KillAudio(source1)

source1_8bit = ConvertBits(source1, bits=8) #I dont'know much about formats, maybe this is not ideal...
source1YV12 = ConvertToYV12 (source1_8bit) #I dont'know much about formats, maybe this is not ideal...

txt_sz = 36

BlkSz =32
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Remove spots-----------------------------------------------------------------
spotless_107=SpotLess107(source1,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
#SpotRemoved = SpotRemover(source1YV12,Spot=128, RadT=1, Pel=Pel, BlkSz=32, DeGrain=false, Chroma=true, PreFilter=true, mask=false, Tm = true,thsad=thsad, Glob = true)
#----------------------------------------------------------------------------------

#SpotRemoved = ConvertBits(SpotRemoved, bits=10) #I dont'know much about formats, maybe this is not ideal...
#SpotRemoved = ConvertToYUV422(SpotRemoved) #I dont'know much about formats, maybe this is not ideal...

#Sharpen----------------------------------------------------------------------
spotless_107_shp=spotless_107.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#SpotRemoved_shp=SpotRemoved.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#----------------------------------------------------------------------------------

# Restore grain -------------------------------------------------------------
spotless_107_shp_Rg = mRD_RestoreGrain(spotless_107_shp, source1)
#SpotRemoved_shp_Rg = mRD_RestoreGrain(SpotRemoved_shp, source1)
#----------------------------------------------------------------------------------

# Delta Restore -------------------------------------------------------------

src = source1
filtered = spotless_107_shp

# extra clipping (if needed)
#crop_left = 150
#crop_right = 250
#crop_left = 750
#crop_right = 250
#src = src.Crop(crop_left, 0, -crop_right, 0)
#filtered = filtered.Crop(crop_left, 0, -crop_right, 0)

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(src, filtered, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
suppression_mask = LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)

# suppression mask removes too dark areas from chroma mask
suppressed_chroma_mask = mt_lutxy(chroma_mask, suppression_mask, expr="x y -")

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(src, filtered, delta=5, spotSize=6)
chroma_override = ExpandMask(chroma_override, 3, blur=4)

# combine luma, chroma and chroma override masks
combined_mask = mt_logic(luma_mask, suppressed_chroma_mask, "or")
combined_mask = mt_logic(combined_mask, chroma_override, "or")

# TEST - luma and chroma override only
#combined_mask = mt_logic(luma_mask, chroma_override, "or")

# combined mask without suppression and chroma override - for testing
combined_no_suppr = mt_logic(luma_mask, chroma_mask, "or")

# restore ---------------------------------------------------------------------------
delta_restore = Overlay(filtered, src, mask=combined_mask)
restore_nosuppression = Overlay(filtered, src, mask=combined_no_suppr)

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(src)
yellow = BlankClip(src, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(src, color=$40f75b)
gray = BlankClip(src, color=$c0c0c0)
red = BlankClip(src, color=$9e2424)
maskviz = Overlay(black, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = Overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = Overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = Overlay(maskviz, gray, mask=luma_mask) # luma mask - gray

comp = Overlay(delta_restore, maskviz, opacity=0.4)

stack = StackHorizontal(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

# scaling (if needed)
stack = stack.BilinearResize(stack.width/2, stack.height/2)

return stack

inter = Interleave(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

return inter

function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

video = Interleave(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(overlay,"delta restore",size = txt_sz,align=2),\
subtitle(restore_nosuppression,"no suppression",size = txt_sz,align=2),\
subtitle(spotless_107_shp,"spotless_107_shp",size = txt_sz,align=2)\
)

masks = Interleave(\
subtitle(BlankClip(source1),"",size = txt_sz,align=2),\
subtitle(combined_mask,"combined_mask",size = txt_sz,align=2),\
subtitle(combined_no_suppr,"combined_no_suppr",size = txt_sz,align=2),\
subtitle(BlankClip(spotless_107_shp),"",size = txt_sz,align=2)\
)

return StackHorizontal(video, masks)

group = StackHorizontal(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(spotless_107_shp,"spotless_107_shp",size = txt_sz,align=2),\
subtitle(overlay,"delta restore",size = txt_sz,align=2),\
subtitle(combined_mask,"combined mask",size = txt_sz,align=2),\
subtitle(luma_mask,"luma delta mask",size = txt_sz,align=2),\
subtitle(suppression_mask,"suppression mask",size = txt_sz,align=2),\
subtitle(suppressed_chroma_mask,"suppressed chroma mask",size = txt_sz,align=2),\
subtitle(chroma_mask,"chroma delta mask",size = txt_sz,align=2),\
subtitle(diff,"diff",size = txt_sz,align=2))
#subtitle(chroma_mask,"chroma mask",size = txt_sz,align=2),\
#subtitle(whitemask,"whitemask",size = txt_sz,align=2),\
#subtitle(whitemask,"filled whitemask",size = txt_sz,align=2),\
#subtitle(uv,"UV",size = txt_sz,align=2))
#subtitle(whitemask_orig,"orig. whitemask",size = txt_sz,align=2),\
#subtitle(SpotRemoved,"SpotRemover",size = txt_sz,align=2))
#subtitle(spotless_107_shp_Rg,"spotless_107_shp_Rg",size = txt_sz,align=2),\

return(group)#.AddBorders(1000, 0, 500, 0, color=$000000)

#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff ----------------------------------------------------------------------
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
mask = mask.FastBlur(blur)
}

return mask
}

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

Function SpotLess107(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess107: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

Explanations in the next post because of character limitations...

zorr

4th June 2021, 01:22

I segmented to code into a couple of functions and decided to give the whole machinery a name "Delta Restore".

LumaDeltaMask() is basically what the previous script did - compares two clips and selects mask where either clip is lighter or darker. It doesn't do the optional mask expansion and/or blur, that part has been moved to a separate function ExpandMask(). You can choose the direction of the comparison, "<" for src brightness < filtered brightness and ">" for the opposite direction. Parameter threshold is now called brightness and its value has been switched around - now larger brightness value means brighter and smaller brightness means darker (makes sense right). For example threshold 0.9 is now brightness 1.1. (conversion formula: 2.0 - threshold).
ChromaDeltaMask() has a similar idea to LumaDeltaMask but this time we look at differences in the chroma channels. There's no clear way to say "darker" or "brighter" so it just measures the difference in the two channels and combines them into one value using euclidean distance sqrt(u*u + v*v). Selection is made where distance is large enough, controlled by parameter delta.
ExpandMask() does the mask expansion and/or blur, can be called if needed after mask has been created with LumaDeltaMask() or ChromaDeltaMask().
RemoveSmallSpots() removes too small spots from the mask. It's called by both LumaDeltaMask() and ChromaDeltaMask(). The parameter filltimes has been replaced with more understandable "spotSize", the filter removes all spots which are smaller than that.

What needs to be done to save the arm is to create two masks - luma mask like before and a new chroma mask. The arm in particular is only detectable in the chroma channels. We combine these two masks with "or" operation (additive) to select pixels from the original clip where either mask has selections.

All good? It's not bad but we can do better. The chroma delta and spotSize has to be set quite high (delta=5, spotSize=6) in order not to bring back spots which also have chroma differences. We can improve things if we use a couple more masks. We create another LumaDeltaMask called "suppression mask" where source is a lot darker than filtered and remove those areas from the chroma mask. The chroma mask can now use more sensitive values (delta=3, spotSize=3) and will no longer select the dark spots. But turns out that our little suppression mask will also remove parts of the arm - it has areas where source is a lot darker. To fix that we create another ChromaDeltaMask "chroma override mask" with large delta and spotSize (just like the original chroma mask) and this one is not suppressed. Now we get smaller chroma deltas (which are not too dark) and make sure large chroma deltas are always selected even when they are darker.

A couple of examples: this one shows the arm and the different masks used to salvage it. The masks are color coded:

gray - luma mask
yellow - chroma mask (sensitive)
red - suppression mask
green - chroma override mask

https://i.postimg.cc/DyRLTR1g/delta-restore-hand.png

We can see that the arm is mostly restored by the chroma override mask. I used luma mask brightness 1.1 here but it's ok to use 1.0 which will bring even more details back.

Here's another example showing the sensitive chroma mask (yellow) in action. It has restored a small hand that was undetected by luma mask.

https://i.postimg.cc/4ymHC1dq/delta-restore-hand2.png

And finally this shows why we need a suppression mask. There's a dark spot which was detected by the sensitive chroma mask. Without the suppression mask (red overlay) the spot would have been restored.

https://i.postimg.cc/NM05Tx1F/delta-restore-suppression.png

StainlessS

4th June 2021, 01:39

Nifty :)

kedautinh12

4th June 2021, 02:14

Wow, thanks

Arx1meD

4th June 2021, 07:22

zorr, really great job!
Will you make your own script like post-processing after cleaning spots?

chmars

4th June 2021, 08:13

What a big work!
Gonna test that on a terrible new footage that made me sweat yesterday.

chmars

4th June 2021, 18:01

Found some hours to test on relevant footage parts, I can say: Brilliant

I tested "out of the box" as provided, in exception of:
-Spotless BlkSz = 24 (48 was too much, not useful, eating too much)
-Unsharpmask applied before spotless (less pixelisation/artefacts in eaten parts, same cleaning efficiency (different but same success average)).
As Spotless softens/degrains the image, the DeltaRestored parts are more grainy since coming from source. With unsharpmask applied, this difference tends to disapear.

Remarks:
Spotless cleans very well.
DeltaR restores most of the eaten parts:
-heads
-arms
-birds
-glass reflections...

Sometimes doesn't work on
-birds dark parts
-some rare spots (too chromatic?)

I repeat, I didn't tweak params: result is very good already. Dunno if some more hours tweaking would help, time runs...

Maybe, for the birds wings + other unrestored parts:
There is one difference between wanted parts and dirt:
Dirt is always sharp.
Unrestored parts of the images had unsharp edges.
Could this difference be used to determine if dirt/not dirt?

Here are extracts.
Sorry for those who dislike grain, lots of 1950-60 8mm to do...
I reordered the Interleave to faster locate spots on the picture.

Dirt away, leg back
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/E8EAeX9hnH5KIiE/download

Leg, hand, retored and remaining dirt...or bird?
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/3TLsI3nFO3bVgu3/download

Even birds while flying.
This seems nothing but don't miss it:
An Italian 1955 square is not itself any more without all the birds taking off at the same time.
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/yEZLqgaWh4UA8P2/download

https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/WNM9VpYGnEkud3E/download

Same for this tramway. Some reflections, you say. But these details make the image alive, bring some kind of beauty. We can see it well in the original movie.
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/rrsf38O5VAlpgjq/download

Here is a partially restored fronthead. Only the bright part is back, not the hair.
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/04qLjGPNefrqsX1/download

Here, a spot is restored.
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/VsNZMrS4ahSGy17/download

Links:
source (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/4VMlbuB16GpZVko/download),
full scenes comparison result (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/zJgYDe92nbYwwjG/download).

(realizing I mixed up the order of the frames. Give me a simple trick, (Linux), to make webm from tiff, avi, whatever and I'll do better next time ;))

johnmeyer

4th June 2021, 21:06

I have a small film transfer project coming up next week and am eager to try this new approach to dirt removal. I'll report on my results when I'm finished.

If it works, it will save me a huge amount of time when cleaning the old football films that a collector sends to me. Because of RemoveDirtMC's tendency to remove footballs while in flight, I've had to do a second restoration, but without dirt removal, and then cut to that during pass, punts, and kicks, while the ball was in the air. This could therefore save me perhaps as much as an hour for each of those projects, and also result in a much better final result since all the frames will be cleaned.

chmars

4th June 2021, 23:58

Howevwer, I noticed it is still good to run two scripts to prepare the grading in the NLE.
To put one underneath the other in the timeline.
Both with same crop, trim, everything, but one with [sharpening+Spotless+DeltaRestoreOne] and one without.

In the case of the man with missing hair (pic6), it becomes easy to just delete a part of the head to let appear the original from the clip laying underneath.

But when DeltaRestore will discriminate between sharp and unsharp dark removed "spots", this will become unusefull .:D
Anyways, with or without, thanks again for these scripts, you are AviSaints!

zorr

5th June 2021, 01:56

-Spotless BlkSz = 24 (48 was too much, not useful, eating too much)
-Unsharpmask applied before spotless

I applied these changes to the latest script.

There is one difference between wanted parts and dirt:
Dirt is always sharp.
Unrestored parts of the images had unsharp edges.
Could this difference be used to determine if dirt/not dirt?

Challenge accepted. :D

#source1= AviSource("459_s2_002-XAVCI-4K-BLD_5_0-trim_crop-for_tests.avi")
#source1 = crop(source1,700,0,-500,0)

#source1= FFVideoSource("ZorrBack_test.avi")
#source1 = crop(source1,450,50,1352,-40)

source1= FFVideoSource("464_-trim_crop_grd_cut.avi")

source1= KillAudio(source1)

source1_8bit = ConvertBits(source1, bits=8) #I dont'know much about formats, maybe this is not ideal...
source1YV12 = ConvertToYV12 (source1_8bit) #I dont'know much about formats, maybe this is not ideal...

txt_sz = 36

BlkSz = 24
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Sharpen----------------------------------------------------------------------
spotless_107_shp=source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#SpotRemoved_shp=SpotRemoved.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#----------------------------------------------------------------------------------

#Remove spots-----------------------------------------------------------------
spotless_107=SpotLess107(spotless_107_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
#SpotRemoved = SpotRemover(source1YV12,Spot=128, RadT=1, Pel=Pel, BlkSz=32, DeGrain=false, Chroma=true, PreFilter=true, mask=false, Tm = true,thsad=thsad, Glob = true)
#----------------------------------------------------------------------------------

#SpotRemoved = ConvertBits(SpotRemoved, bits=10) #I dont'know much about formats, maybe this is not ideal...
#SpotRemoved = ConvertToYUV422(SpotRemoved) #I dont'know much about formats, maybe this is not ideal...

# Restore grain -------------------------------------------------------------
#spotless_107_shp_Rg = mRD_RestoreGrain(spotless_107_shp, source1)
#SpotRemoved_shp_Rg = mRD_RestoreGrain(SpotRemoved_shp, source1)
#----------------------------------------------------------------------------------

# Delta Restore -------------------------------------------------------------

src = source1
filtered = spotless_107

# extra clipping (if needed)
#crop_left = 150
#crop_right = 250
#crop_left = 750
#crop_right = 250
#crop_left = 30
#crop_right = 1070
#src = src.Crop(crop_left, 0, -crop_right, 0)
#filtered = filtered.Crop(crop_left, 0, -crop_right, 0)

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(src, filtered, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
suppression_mask = LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)

# suppression mask removes too dark areas from chroma mask
suppressed_chroma_mask = mt_lutxy(chroma_mask, suppression_mask, expr="x y -")

# edge mask - detects sharp edges
edgeMask = scharr(src).mt_binarize(52)
#edgeMask = src.vsTEdgeMask(threshY=51, type=5, link=0, y=3, u=2, v=2)
#return edgeMask

# TODO: don't apply edge mask at the edges of the frame (to fix cases like frame 276)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(src, filtered, delta=5, spotSize=6) # orig. spotSize 6

# NOTE: using blur causes weird bug (example: frame 147)
#chroma_override = ExpandMask(chroma_override, 3, blur=4)

# combine luma, chroma and chroma override masks
combined_mask = mt_logic(luma_mask, suppressed_chroma_mask, "or")
combined_mask = mt_logic(combined_mask, chroma_override, "or")

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-127")
combined_mask = mt_lutxy(combined_mask, edgeAreaMask, expr="x y -")

# TEST - luma and chroma override only
#combined_mask = mt_logic(luma_mask, chroma_override, "or")

# combined mask without suppression and chroma override - for testing
combined_no_suppr = mt_logic(luma_mask, chroma_mask, "or")

# restore ---------------------------------------------------------------------------
delta_restore = Overlay(filtered, src, mask=combined_mask)
restore_nosuppression = Overlay(filtered, src, mask=combined_no_suppr)

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(src)
yellow = BlankClip(src, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(src, color=$40f75b)
gray = BlankClip(src, color=$c0c0c0)
red = BlankClip(src, color=$9e2424)
blue = BlankClip(src, color=$0d3b7a) #24579e 243c9e
maskviz = Overlay(black, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = Overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = Overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = Overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = Overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue

comp = Overlay(delta_restore, maskviz, opacity=0.4)

#return StackHorizontal(src, filtered, delta_restore, edgeMask, maskviz)

stack = StackHorizontal(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(edgeMask, "edge mask", txt_sz),\
sub(diff, "diff", txt_sz)\
)

# scaling (if needed)
stack = stack.BilinearResize(stack.width/2 + 1, stack.height/2)

return stack

inter = Interleave(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

return inter

function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

video = Interleave(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(overlay,"delta restore",size = txt_sz,align=2),\
subtitle(restore_nosuppression,"no suppression",size = txt_sz,align=2),\
subtitle(spotless_107_shp,"spotless_107_shp",size = txt_sz,align=2)\
)

masks = Interleave(\
subtitle(BlankClip(source1),"",size = txt_sz,align=2),\
subtitle(combined_mask,"combined_mask",size = txt_sz,align=2),\
subtitle(combined_no_suppr,"combined_no_suppr",size = txt_sz,align=2),\
subtitle(BlankClip(spotless_107_shp),"",size = txt_sz,align=2)\
)

return StackHorizontal(video, masks)

group = StackHorizontal(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(spotless_107_shp,"spotless_107_shp",size = txt_sz,align=2),\
subtitle(overlay,"delta restore",size = txt_sz,align=2),\
subtitle(combined_mask,"combined mask",size = txt_sz,align=2),\
subtitle(luma_mask,"luma delta mask",size = txt_sz,align=2),\
subtitle(suppression_mask,"suppression mask",size = txt_sz,align=2),\
subtitle(suppressed_chroma_mask,"suppressed chroma mask",size = txt_sz,align=2),\
subtitle(chroma_mask,"chroma delta mask",size = txt_sz,align=2),\
subtitle(diff,"diff",size = txt_sz,align=2))
#subtitle(chroma_mask,"chroma mask",size = txt_sz,align=2),\
#subtitle(whitemask,"whitemask",size = txt_sz,align=2),\
#subtitle(whitemask,"filled whitemask",size = txt_sz,align=2),\
#subtitle(uv,"UV",size = txt_sz,align=2))
#subtitle(whitemask_orig,"orig. whitemask",size = txt_sz,align=2),\
#subtitle(SpotRemoved,"SpotRemover",size = txt_sz,align=2))
#subtitle(spotless_107_shp_Rg,"spotless_107_shp_Rg",size = txt_sz,align=2),\

return(group)#.AddBorders(1000, 0, 500, 0, color=$000000)

#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff ----------------------------------------------------------------------
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
mask = mask.FastBlur(blur)
}

return mask
}

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

Function SpotLess107(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess107: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

Details in the next message.

zorr

5th June 2021, 02:37

This version has a new mask type - edge mask. It's a standard edge detection combined with my favourite MaskTools filter mt_hysteresis. Basically if there are sharp edges anywhere within an isolated mask area then those areas will be cut out so that part will be taken from the filtered version.

I tested a couple of different edge detection filters, two different modes of a recent vsTEdgeMask and my own Scharr edge detector. I think the former is faster but I didn't make measurements.

What I did measure was the proper threshold to use in the edge mask, how hard edges should be considered to be a spot and not some real feature. It's a surprisingly specific value and for the latest test video only Scharr managed to handle all the test cases. The most problematic frames are 9 (shoe should be restored), 155 (highlight should be restored), 190 (large spot should be removed) and 204 (bird should be restored). There's also frame 276 which cannot be fixed with the current version but is fixable. Scharr handles all cases when binarized with threshold 52. vsTEdgeMask type 5 was close, the bird needed threshY > 51 but the spot at frame 190 needs threshY <= 51. Type 3 was also close with threshY 29.

Some test notes below:

# using vsTEdgeMask type 2: threshY min 30 - max 29
# frame 9 - shoe with threshY <= 27 -> threshY > 27
# frame 20 - small dot with threshY <= 30
# frame 90 - medium spot with threshY <= 31
# frame 121 - large spot with threshY <= 48
# frame 132 - small spot with threshY <= 46
# frame 147 - large spot with threshY <= 49
# frame 155 - a highlight with threshY <= 25 -> threshY > 25
# frame 183 - a large spot with threshY <= 37
# frame 190 - a large spot with threshY <= 29
# frame 204 - bird with threshY <= 29 -> threshY > 29
# frame 229 - a spot with threshY <= 62
# frame 241 - small spot with threshY <= 56
# frame 260 - small spot with threshY <= 42
# frame 265 - medium dark spot with threshY <= 55
# frame 276 - large chroma delta at the bottom with threshY <= 42 -> threshY > 42

# using vsTEdgeMask type 5: threshY min 52 - max 51
# frame 9 - shoe with threshY <= 48 -> threshY > 48
# frame 155 - a highlight with threshY <= 46 -> threshY > 46
# frame 190 - a large spot with threshY <= 51
# frame 204 - bird with threshY <= 51 -> threshY > 51
# frame 276 - large chroma delta at the bottom with threshY <= 71 -> threshY > 71

# using scharr: binarize min 52 - max 52
# frame 9 - shoe with binarize <= 48 -> threshY > 48
# frame 155 - a highlight with binarize <= 46 -> threshY > 46
# frame 190 - a large spot with binarize <= 52
# frame 204 - bird with binarize <= 51 -> threshY > 51
# frame 276 - large chroma delta at the bottom with binarize <= 72 -> threshY > 72

Let's look at some example frames:

Here's a large spot which was restored earlier. The mask visualization now includes the edge mask (or rather the hysterized area grown from the edge mask) as blue (luckily there were still some colors left). Whenever that blue color appears it means edge mask was used to cut out a feature which would have been restored from the original clip without the edge mask.

https://i.postimg.cc/FKM4nKt3/delta-restore-edge-mask-1.png

Here's another one. In this one we can also see that some part of a wing was not restored.

https://i.postimg.cc/zBTrm15M/delta-restore-edge-mask-2.png

I didn't yet try to restore the dark areas which would fix this wing and the guy's missing hair. Perhaps we don't need as many masks if we can rely on the edge mask.

There are many blue areas around the frame edges, that's because those contain sharp... edges. :) The problem with frame 276 is caused by that. The edge mask should not be applied near the edges. I'll try that in the next version.

I took out the blur (and expansion) of the chroma override mask, it looks like there's a bug in FastBlur which causes a weird glitch in the edge mask (try to enable it and look at frame 147).

patul

5th June 2021, 03:48

Hi zorr,

Great work there. I'm experimenting with your script, while I don't have old material to begin with, hence I used this (https://www.youtube.com/watch?v=eB91_OHB5Rg) video as a sample, which probably is not an ideal sample because it might have been through some heavy processing prior upload, but I use it anyway.

I noticed some dirt is not removed by SpotLess, and some dirt reappear on Delta Restore. Is there any hints that you can provide please? Thanks in advance

chmars

5th June 2021, 09:52

Seems challenge brightly fullfilled ;)
Just copied it, testing.
EDIT: Your explanations are vital, thanks for this effort too.

Side note, my error:
Minor importance but introduces confusion.
Names are wrong, it reflected as if a not sharpened version was used.

spotless_107_shp=source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
Should be
source1_shp=source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
.

And then,
spotless_107=SpotLess107(spotless_107_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
should bespotless_107=SpotLess107(source1_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)

chmars

5th June 2021, 13:14

Can confirm: returning spots are away!
Lots of steps... Added thread summary in post #1 to recap.

I didn't yet try to restore the dark areas which would fix this wing and the guy's missing hair. Perhaps we don't need as many masks if we can rely on the edge mask.

I was looking... maybe by combining suppression_mask and edgeMask?

There are many blue areas around the frame edges, that's because those contain sharp... edges. :) The problem with frame 276 is caused by that. The edge mask should not be applied near the edges. I'll try that in the next version.

I plan to fill the perforations white holes with black anyway.

Going to test it all on a new 15meter footage, before grading with Resolve (Slog3, very bright-uncontrasted). Will see...
I prefer to do it before grading. So I can verify all at once, avoiding having to look a second time to the whole movie.
Maybe, I'll add some functions at script end to simulate grading, so I can check if it works that way.

EDIT: As expected, Spotless seems ok but Data Restore works less good on ungraded Slog3.
So I will kind of grade the source with Autolevels to build the masks, then apply the masks on spotlessed original + original.

zorr

5th June 2021, 21:49

zorr, really great job!
Will you make your own script like post-processing after cleaning spots?

No, unless I come up with something worth contributing. I haven't thought about post-processing much (and that's quite a wide definition). Did you have something specific in mind?

zorr

5th June 2021, 21:53

I have a small film transfer project coming up next week and am eager to try this new approach to dirt removal. I'll report on my results when I'm finished.

That would be interesting, I'd love hear how it goes. Perhaps it needs some adjustment or even a new approach. The method is quite general in principle so I think you have better than 50% chance of having useful results. And if not then we can learn something new.

zorr

5th June 2021, 22:04

I used this (https://www.youtube.com/watch?v=eB91_OHB5Rg) video
...
I noticed some dirt is not removed by SpotLess, and some dirt reappear on Delta Restore. Is there any hints that you can provide please? Thanks in advance

I took a quick look at the video. It has many duplicate/repeating frames and noise removal algorithms don't work well with those. So the first step should be getting rid of the duplicate frames, I believe the correct term for that is decimation. There are people on this forum who are much more knowledgeable than me on that subject so I can't give you detailed instructions.

I didn't try Delta Restore on your clip but it seems to have softer look, even the dirt spots are much softer than the material chmars has shown so the edge mask is probably not going to work very well on that. You may have to adjust the suppression mask and/or the chroma override mask to prevent dark spots from reappearing.

zorr

6th June 2021, 01:57

Names are wrong, it reflected as if a not sharpened version was used.

Thanks, I corrected the error.

I've worked on the "dark side" and it's almost finished but it's getting late so I will write a comprehensive report tomorrow.

Meanwhile have you considered removing the vertical streaks? Spotless can't touch them because they last many consecutive frames but there are other tools. I fiddled a bit with DeScratch (http://avisynth.nl/index.php/DeScratch) and got some promising results.

https://i.postimg.cc/85brv8Rv/descratch.gif

Here it was applied after Spotless but I think it might work even better before it. If DeScratch leaves behind a lone streak but manages to remove it from the previous and next frame then Spotless can remove the remaining one.

kedautinh12

6th June 2021, 03:43

Thanks, Waiting for result

chmars

6th June 2021, 10:07

...
I think you have better than 50% chance of having useful results. And if not then we can learn something new.
...

I'say 90%- > Used it yesterday on see/boat/coast 5mn 8mm footage.
Very efficient, even on Slog washed out source (pseudo-graded to feed Sl (Spotless) +Dr (DeltaRestore)).
Wave splashing water and hands removed by Sl were restored well by Dr.

...
I've worked on the "dark side" and it's almost finished but it's getting late so I will write a comprehensive report tomorrow.

Cool. What do you think of writing Dr as a function with version number as like Sl?
So we could put the whole cleaned script in post #1. It would also be easier in cases of updates, I guess.
I'll add the pseudo-grading thing for washed out footages too.

Meanwhile have you considered removing the vertical streaks? Spotless can't touch them because they last many consecutive frames but there are other tools. I fiddled a bit with DeScratch (http://avisynth.nl/index.php/DeScratch) and got some promising results.

Yes, was thinking of it. Just discovered Avisynth - which is beautiful but hurts a bit sometimes - short ago. My head is not very good at multutasking :). I'll try it then could add it to the general script in #1, btw.

Arx1meD

6th June 2021, 10:30

No, unless I come up with something worth contributing. I haven't thought about post-processing much (and that's quite a wide definition). Did you have something specific in mind?I mean to combine all your work (RemoveSmallSpots, LumaDeltaMask, ChromaDeltaMask, ExpandMask) into one script. For example, something similar to the Repair (http://avisynth.nl/index.php/RgTools/Repair) filter in RgTools.

chmars

6th June 2021, 23:18

Here is the updated - cleaned script.
The script is not functional yet - Don't try it.
Zorr, here is the idea of using a levelled clip for Spotless and Delta Restore and then apply masks to the original clip when necessary (Slog).

I'll pm you the DR I turned into a function.
This Avoids confusions between versions and mostly, it's yours, dunno if it suits you, I don't wanna "walk on your grass".
But as it was some work to transpose variables, check the whole thing..., it could help perhaps.

Part 1
/*
SpoDel (Spotless+Delta Restore)

Script intended to clean and sharpen digitalized film.
Can optionally restore grain after cleaning.
Sharpening and denoising could be hardly improved. Please see VideoFred's and JohnMeyer's scripts and posts fore more knowlege. (VideoFred's sharp-blur-sharp-blur-sharp trick). Please share emprovements!

Functions: StainlessS, Zorr, real.finder
Script: chmars - 06/06/2021 - some parts copied here and there, thank to solutions given by doom9 forum users.
Thread: https://forum.doom9.org/showthread.php?t=182831
Spotless thread, last versions: https://forum.doom9.org/showthread.php?t=181777
Lots of informaions on film capture/restoration: https://forum.doom9.org/showthread.php?t=144271

Script tested on HD 4:2:2 10bits sources with Avisynth+370_x64

How it works:
Spotless removes dirt spots that are not present in neibourg frames.
But sometimes, fast moving objects are removed too (balloons, birds, hands, feet)
Hence is Delta Restore needed. It brings back details (but not spots) that are part of the image.
As Spotless tends to soften/degrain the image, parts brought back from the original image by Delta Restore are more grainy
Solution for a discreete replacement of these parts are either to sharpen the Spotless input, or degrain Delta Restore output, before to apply it to Spotless output.
Washed out, Slog rushes:
Spotless seems to work as well with washed out images (old footage, Slog rushes), but Delta Restore needs colors and contrast to funtion properly.
In this case, a more contrasted clip is generated to feed Delta Restore and generate masks only. But final result will not be graded.

*/

SetMemoryMax(8000) # Dunno if optimal...
threads = 16 # Dunno if optimal...
### Check speed, memory consumption and CPU usage with AVSMeter64 ###
SetFilterMTMode("DEFAULT_MT_MODE", 2) # Dunno if optimal...

# VARIABLES #################################################################################################
########################################################################################################################
########################################################################################################################
# Here, define what you want

# Path to file
video_file = "X:\some_path\some_file"

play_speed = 25 # This will not change your frames/fields, neither their count
txt_sz = 36 #Test size if infos displayed
Sharpen_it = True # Default True. Sharpen clip or not (recomanded, see intro)
Slog = True # Default Falde. If True, Generate a leveled clip if or wahed-out or Slog rushes

#Grainitude, cancels sharpen
Rgr = False # Set True Restores grain after passed through Spotless. Might restore some dust too. 5,10 = good compromise grain/dirt. Dunno how that works.
val1 = 10
val2 = 20 # 5,10 = good compromise grain/dirt. Dunno how that works exactly...

# Sharpen variables ----------------------------------------------------------------------------------------------------------------------------------------------------------
Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

# Spotless variables, to remove dirt and spots. -------------------------------------------------------------------------------------------------------------------------
# See Function script for more advanced settings
BlkSz =24 #Spot Size, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]
Pel = 1 # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)
Tm=false #TrueMotion Default True. Some folk swear truemotion=false is better.
Bblur = 0.6 #Default 0.6. If used, Suggest about 0.6, where MAnalyse create vectors is performed on denoised (blurred) super clip for better motion analysis.
ThSAD = 10000 # [SAD, Sum of Absolute (pixelwise) Differences] Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1)
RadT = 1 #Removes Spots on up to RadT [Temporal Radius] consecutive frames. RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
#Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

#Delta Restore variables, restore removed parts
LuDeMaDir = ">" # ">" Removed parts to restore are: "<"=brighter or ">"=darker
LuDeMaBr = 1.1 # 1.1 LumaDeltaMask Brightness
LuDeMaSz = 3 # 3 LumaDeltaMask SpotSize
ChroDeMaDe = 3 # 3 ChromaDeltaMask Delta
ChroDeMaSz = 3 # 3 ChromaDeltaMask SpotSize
SLuDeMaDir = "<" # "<"
SupLuDeMaBr = 0.9 # 0.9 Suppression LumaDeltaMask brightness threshold
SupLuDeMaSz = 2 #2 Suppression LumaDeltaMask SpotSize
Ov_ChroDeMaDe = 5 # 5 Override ChromaDeltaMask Delta # chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
Ov_ChroDeMaSz = 6 # 6 Override ChromaDeltaMask SpotSize
EdMaShp = 52 #52 bigger = sharper

# Levels correction -------------------------------------------------------------------------------------------------------------------------------------------------------------
#In case of washed-out footage or Slog captures
# Needed to generate dark and saturated enough clips, to feed the filters whith what they need.
# Will NOT apply to final result
gamma = 1.0
input_low = 80 # What level is considered as black in input
input_high = 240 # Same for white
output_low = 0 # Minimum level wanted for output black pixels
output_high = 255 # Same for white

#Denoise----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
# If needed, removes grain
#Not deeply tested, please test and report
#~ denoising_strength= 200 #denoising level of first denoiser: MDegrain() # high reduces colors
#~ block_size= 4 #block size of MVDegrain
#~ block_size_v= 4
#~ block_over= 2 #block overlapping of MVDegrainMulti()
#~ dirt_strength=10 #sets amount of dirt removal (big spots)

# / END OF VARIABLES ######################################################################################################
########################################################################################################################
########################################################################################################################

chmars

6th June 2021, 23:18

Part 2 of 2

source1 = FFVideoSource(vid_file)
# Set play speed (if needed)
source1 = assumefps(source1,play_speed)

# Optional
## ------------------------------------------------------------------------------------
# Keep only from frame n to frame m
# source1 = trim(source1, 2,0)
# Resize (speeds-up for tests?)
# source1 = source1.BilinearResize(source.width/2 , source.height/2)

# Leveled clip
## ------------------------------------------------------------------------------------
# If Slog = True,
# Make a decently contrasted clip to feed the filters.
# Else, do nothing

#EDIT:
#The next line was crap.
#lvl_src = Slog ? Autolevels(source1, sceneChgThresh=255, gamma = gamma, midpoint = 0.5, input_low = input_low, input_high = input_high, output_low = output_low, output_high = output_high) : source1
#Has to be replaced by, at least:
lvl_src = Slog ? source1.convertbits(8).Autolevels(filterradius = 3, sceneChgThresh = 20, autogamma = true, midpoint = 0.5, autolevel = true, border=500, input_high = 230 ).convertbits(16) : source1
#/EDIT END

# If needed, verify levels
h = Histogram(source1,"classic")
h_lvl = Slog ? Histogram(lvl_src,"classic") : NOP
#~ return(StackHorizontal(h, h_lvl)) # Un comment to see clips with histograms

#Sharpening
# -----------------------------------------------------------------------------------
# Best results found by sharpening before (not after) cleaning.
source1_shp = Sharpen_it ? source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th) : source1
# If Using Slog rushes, also sharpen the leveled clip
source1_lvl_shp = Sharpen_it ? lvl_src.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th) : source1_shp

#Remove spots, Spotless, version 1.07
# ------------------------------------------------------------------------------------
filtered_orig = SpotLess(source1_shp,blksz=BlkSz,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
lvl_filtered = Slog ? SpotLess(source1_lvl_shp,blksz=BlkSz,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT) : filtered_orig
#~ return(filtered_orig)
# Restore removed objects seen as dirt
# ------------------------------------------------------------------------------------
restored = DeltaRestore(

#To see if Delta Restore becomes a function or not

# Restore grain - cancels sharpen
#------------------------------------------------------------------------------------
restored_2 = Rgr ? mRD_RestoreGrain(restored, source1,val1, val2) : restored #5,10 = good compromise grain/dirt

#~ return(Interleave(source1, restored_2, restored))
return(restored_2)

###############################################################################################################################
###############################################################################################################################
###############################################################################################################################
###############################################################################################################################
# Delta Restore -------------------------------------------------------------

...

# Delta Restore sub-functions
###############################################################################################################################
###############################################################################################################################

function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff ----------------------------------------------------------------------
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
mask = mask.FastBlur(blur)
}

return mask
}

# RestoreGrain Function
###############################################################################################################################
###############################################################################################################################

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") { #https://forum.doom9.org/showthread.php?p=1943371#post1943371

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 20.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

# Spotless Function
###############################################################################################################################
###############################################################################################################################
# Version 1.07
Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess107: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

Prefetch(threads)

chmars

6th June 2021, 23:31

Little off topic: I didn't find a way to automate light levels. Is that so hard to do? So, I used autolevels in "manual mode"...

EDIT: I might have been tired... and just didn't RTFM
Autolevels line use should be, at least:
lvl_src = Slog ? source1.convertbits(8).Autolevels(filterradius = 3, sceneChgThresh = 20, autogamma = true, midpoint = 0.5, autolevel = true, border=500, input_high = 230 ).convertbits(16) : source1
As it needs 8bits and needs to be told to use autogamma and autolevels.
On my source, borders contain always darker and lighter parts than the image. Therefore border=500 has to be used to not calculate these areas.

zorr

7th June 2021, 22:13

What do you think of writing Dr as a function with version number as like Sl?

I mean to combine all your work (RemoveSmallSpots, LumaDeltaMask, ChromaDeltaMask, ExpandMask) into one script.

Yes that's a good idea and I will reform the script into one or several functions. The sub-functions can be useful as is but the "main" code needs to be wrapped into something as well. I think it may be more than one function (maybe several variations, each a bit more extensive) to keep the number of parameters sensible.

I'll add the pseudo-grading thing for washed out footages too.

Easiest way is to have one function return the final mask and then other to apply it, then you can use a different clip to create the masks.

zorr

7th June 2021, 22:30

WORK IN PROGRESS - dark mask

I have implemented another mask variation - the dark mask (spooky!). It's not working very well at this point as it seems to be impossible to detect the correct dark areas to restore without also restoring the spots here and there. At least with the current approach which uses edge sharpness, spot size and brightness criterias.

I thought it would help to include a limit to the brightness (parameter limitBrightness in LumaDeltaMask), for example to select all areas where brightness is < 0.97 but only if most of the brightness is > 0.66. It works for some of the spots but in order to restore the guy's hair the limit has to be set too low to be effective. I might remove this parameter later.

The blurring functionality is back - I switched to vcmohan's GBlur.

Also edge detection can now be restricted to areas not close the the frame borders, there's function Padding() for that. I had to set the left padding quite high, perhaps a better implementation is needed (or some preprocessing to remove sharp features near the borders).

Latest script here (didn't implement as a function yet or include chmars' latest changes, sorry)

#source1= AviSource("459_s2_002-XAVCI-4K-BLD_5_0-trim_crop-for_tests.avi")
#source1 = crop(source1,700,0,-500,0)

#source1= FFVideoSource("ZorrBack_test.avi")
#source1 = crop(source1,450,50,1352,-40)

source1= FFVideoSource("464_-trim_crop_grd_cut.avi")

source1= KillAudio(source1)

source1_8bit = ConvertBits(source1, bits=8) #I dont'know much about formats, maybe this is not ideal...
source1YV12 = ConvertToYV12 (source1_8bit) #I dont'know much about formats, maybe this is not ideal...

txt_sz = 36

BlkSz = 24
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Sharpen----------------------------------------------------------------------
source1_shp=source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#SpotRemoved_shp=SpotRemoved.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#----------------------------------------------------------------------------------

#Remove spots-----------------------------------------------------------------
spotless_107=SpotLess107(source1_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
#SpotRemoved = SpotRemover(source1YV12,Spot=128, RadT=1, Pel=Pel, BlkSz=32, DeGrain=false, Chroma=true, PreFilter=true, mask=false, Tm = true,thsad=thsad, Glob = true)
#----------------------------------------------------------------------------------

#SpotRemoved = ConvertBits(SpotRemoved, bits=10) #I dont'know much about formats, maybe this is not ideal...
#SpotRemoved = ConvertToYUV422(SpotRemoved) #I dont'know much about formats, maybe this is not ideal...

# Restore grain -------------------------------------------------------------
#spotless_107_shp_Rg = mRD_RestoreGrain(spotless_107_shp, source1)
#SpotRemoved_shp_Rg = mRD_RestoreGrain(SpotRemoved_shp, source1)
#----------------------------------------------------------------------------------

# Delta Restore -------------------------------------------------------------

src = source1
filtered = spotless_107

# extra clipping (if needed)
#crop_left = 800
#crop_right = 280
#src = src.Crop(crop_left, 0, -crop_right, 0)
#filtered = filtered.Crop(crop_left, 0, -crop_right, 0)

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3)
luma_mask = ExpandMask(luma_mask, 3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(src, filtered, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
suppression_mask = LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)

# suppression mask removes too dark areas from chroma mask
suppressed_chroma_mask = mt_lutxy(chroma_mask, suppression_mask, expr="x y -")

# edge mask - detects sharp edges
edges = scharr(src)

# don't detect edges near the frame edges
edges = edges.Padding(60, 14, 14, 10)

edgeMask = edges.mt_binarize(52)

# dark mask - selects where source brightness < filtered brightness but not below limitBrightness
dark_mask = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
darkEdgeMask = edges.mt_binarize(44) # 40
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_mask, chroma="-127")
dark_mask = mt_lutxy(dark_mask, darkEdgeAreaMask, expr="x y -")
dark_mask = ExpandMask(dark_mask, 3, blur=4)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(src, filtered, delta=5, spotSize=6) # orig. spotSize 6
chroma_override = ExpandMask(chroma_override, 3, blur=2)

# combine luma, chroma and chroma override masks
combined_mask = mt_logic(luma_mask, suppressed_chroma_mask, "or")
combined_mask = mt_logic(combined_mask, chroma_override, "or")

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-127")
combined_mask = mt_lutxy(combined_mask, edgeAreaMask, expr="x y -")

# add dark areas (these are already checked for edges)
combined_mask = mt_logic(combined_mask, dark_mask, "or")

# TEST - luma and chroma override only
#combined_mask = mt_logic(luma_mask, chroma_override, "or")

# combined mask without suppression and chroma override - for testing
combined_no_suppr = mt_logic(luma_mask, chroma_mask, "or")

# restore ---------------------------------------------------------------------------
delta_restore = Overlay(filtered, src, mask=combined_mask)
restore_nosuppression = Overlay(filtered, src, mask=combined_no_suppr)

edgeAreaMask = mt_logic(edgeAreaMask, darkEdgeAreaMask, "or")
#edgeAreaMask = darkEdgeAreaMask

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(src)
yellow = BlankClip(src, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(src, color=$40f75b)
gray = BlankClip(src, color=$d0d0d0)
darkgray = BlankClip(src, color=$909090)
orange = BlankClip(src, color=$a36b02)
purple = BlankClip(src, color=$4e0091)
red = BlankClip(src, color=$9e2424)
blue = BlankClip(src, color=$0d3b7a) #24579e 243c9e
maskviz = Overlay(black, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = Overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = Overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = Overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = Overlay(maskviz, purple, mask=dark_mask) # dark mask - dark gray
maskviz = Overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue

comp = Overlay(delta_restore, maskviz, opacity=0.45)

#return Interleave(StackHorizontal(src, comp), StackHorizontal(delta_restore, maskviz))

#return StackHorizontal(src, filtered, delta_restore, edgeMask, maskviz)

stack = StackHorizontal(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(edgeMask, "edge mask", txt_sz),\
sub(diff, "diff", txt_sz)\
)

# scaling (if needed)
stack = stack.BilinearResize(stack.width/2 + 1, stack.height/2)

return stack

inter = Interleave(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

return inter

function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------

function Padding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness suppression (part 2)
if (limitBrightness != brightness) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff ----------------------------------------------------------------------
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
# mask = mask.ConvertBits(16)
mask = mask.GBlur(rad=6, sd=blur, u=true)
# mask = mask.ConvertBits(10)
}

return mask
}

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

Function SpotLess107(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess107: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

zorr

7th June 2021, 22:54

Some examples of dark mask in action:

Let's look at the happy cases first. The main goal was to restore the guy's hair and yes, we can do that! The dark mask is shown as purple in the mask visualization.

https://i.postimg.cc/MZ5vN8P4/dark-mask-4-frame-219.png

Here's another, the dark mask restores the missing features of a bird.

https://i.postimg.cc/SN4Jtxfc/dark-mask-1.png

But the dark mask also restores spots, here's a typical example:

https://i.postimg.cc/VsqNMcPv/dark-mask-3-frame-79.png

There's also at least one frame where the edge mask suppresses the dark mask because a sharp-edged spot is located over the dark mask:

https://i.postimg.cc/9fsfPmfR/dark-mask-2.png

I have another approach for the dark mask that I'm going to try next. Most of the time when it's a real feature the diff has both darker and lighter deltas. So if we only select those dark areas which have bright areas in close proximity we might have better success in detecting only what we want.

There also could be a better way to detect the sharpness of a feature. Now it's considered sharp if there is an edge anywhere within the area. Perhaps it works better if we only look at the sharpness of the edge pixels of the area.

chmars

8th June 2021, 09:42

(or some preprocessing to remove sharp features near the borders).

Yes, I just had dark scenes where these perforations disturb sight.
Don't want to crop the image but let's fill them.
I borrowed some code of yours. If we fill them with something like this? No sharp edges any more:

source1=FFVideoSource("464-13_63.mov").KillAudio()
perfo = LumaDeltaMask(source1, ">", brightness=1.0, spotSize=100)
fill_color = BlankClip(source1,color=$26282d)
# enlarge mask (optional) ---------------------------------------------------
expandtimes = 6
for (i = 1, expandtimes) {
perfo = perfo.mt_expand( chroma="-127")
}
maskblur = 6
if (maskblur > 0) {
perfo = perfo.FastBlur(maskblur)
}

filled = Overlay(source1,fill_color,mask=perfo)
return(filled)

#########################################################################################################################

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, string direction, float "brightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)

gray = BlankClip(source, color=$c0c0c0)
# diff
diff = Grayscale(mt_makediff(source, gray ))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

return mask
}

Have TODO:
-find the way to copy the color which is between the holes, as it sometimes shortly changes like in the example below.
-maybe limit horizontal blur.
It will never be 100% but I think could be discreete enough.
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/KZs5b4MCjsHb8Et/download

EDIT: Made new thread not to mix all:
https://forum.doom9.org/showthread.php?t=182930

...Didn't implement as a function yet or include chmars' latest changes, sorry)
No problem, later.

It's great, you are getting closer with the dark mask.

I wander if we shouldn't look earlier in the process.
If we could talk to Spotless in terms of surface instead of block side size (n*n), I believe this would help.
I mean: we specify a spot size to remove. But the big spots are rather long but have a small surface (liker hairs).
A dirt spot as big as the birds or guy's hair is rare.
So if we could specify [max length] AND [max surface] for spots to be removed, wouldn't this help a lot?

kedautinh12

8th June 2021, 09:53

I think Padding() is a normal name, maybe same function names in the future. I think you need rename for your personal. Ex: zorr_Padding() or z_Padding()

zorr

10th June 2021, 01:47

Yes, I just had dark scenes where these perforations disturb sight.
Don't want to crop the image but let's fill them.

I noticed you made another thread for this subject, so let's keep that discussion there.

It's great, you are getting closer with the dark mask.

Yes I am actually. The idea I described in my previous post seems to work fairly well. :D And I also added a limit based on the absolute brightness. No results to show yet, have to get some sleep...

I mean: we specify a spot size to remove. But the big spots are rather long but have a small surface (liker hairs).
A dirt spot as big as the birds or guy's hair is rare.
So if we could specify [max length] AND [max surface] for spots to be removed, wouldn't this help a lot?

Spotless mostly doesn't remove those long hair-like things so Delta Restore cannot help there. It would be simple to specify the spot in terms of width and height instead of size but I'm not sure how many cases there are where that would make an improvement. Surface area is trickier... it's possible to calculate the total area of the mask but not per spot, at least without writing an actual plugin (I shouldn't say it's not possible because it probably is but it would be *very* slow to execute such code).

I think Padding() is a normal name, maybe same function names in the future. I think you need rename for your personal. Ex: zorr_Padding() or z_Padding()

Thanks for the hint, I'll rename it or perhaps it's going to be an internal function not visible outside Delta Restore.

chmars

10th June 2021, 08:12

I noticed you made another thread for this subject, so let's keep that discussion there.
Yes, thought it was better to sort the questions. But still, it will help with borders, I hope.

Spotless mostly doesn't remove those long hair-like things

I ment, for the ones it removes (almost all "long hairs" on my footages are away) thanks to it.

Surface area is trickier...I shouldn't say it's not possible because it probably is but it would be *very* slow to execute such code).

Hm, I understand, ok. Anyway very far from what I could achieve in present knowlage...

No results to show yet, have to get some sleep...

Hey, don't forget Spotelss + Delta Restore are already brilliant and working in a huge majority of cases!
I am verifying another footage made with them, I found no fault. Footage with so many birds, guys and kids like the previous one seems rare.

Sharc

10th June 2021, 08:24

Spotless mostly doesn't remove those long hair-like things ....

Perhaps this may help:
turnleft() #some angle
spotless()
turnright()

zorr

10th June 2021, 23:33

Delta Restore - Dark Mask complete

I think this version of dark mask is good enough. I had to leave Spotless out of this code, doesn't fit into single post otherwise. You can copy it from the previous version.

source1= FFVideoSource("464_-trim_crop_grd_cut.avi")
source1= KillAudio(source1)

source1_8bit = ConvertBits(source1, bits=8) #I dont'know much about formats, maybe this is not ideal...
source1YV12 = ConvertToYV12 (source1_8bit) #I dont'know much about formats, maybe this is not ideal...

txt_sz = 36

BlkSz = 24
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Sharpen----------------------------------------------------------------------
source1_shp=source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#SpotRemoved_shp=SpotRemoved.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#----------------------------------------------------------------------------------

#Remove spots-----------------------------------------------------------------
spotless_107=SpotLess107(source1_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
#SpotRemoved = SpotRemover(source1YV12,Spot=128, RadT=1, Pel=Pel, BlkSz=32, DeGrain=false, Chroma=true, PreFilter=true, mask=false, Tm = true,thsad=thsad, Glob = true)
#----------------------------------------------------------------------------------

#SpotRemoved = ConvertBits(SpotRemoved, bits=10) #I dont'know much about formats, maybe this is not ideal...
#SpotRemoved = ConvertToYUV422(SpotRemoved) #I dont'know much about formats, maybe this is not ideal...

# Restore grain -------------------------------------------------------------
#spotless_107_shp_Rg = mRD_RestoreGrain(spotless_107_shp, source1)
#SpotRemoved_shp_Rg = mRD_RestoreGrain(SpotRemoved_shp, source1)
#----------------------------------------------------------------------------------

# Delta Restore -------------------------------------------------------------

src = source1
filtered = spotless_107

# extra clipping (if needed)
#crop_left = 800
#crop_right = 280
#src = src.Crop(crop_left, 0, -crop_right, 0)
#filtered = filtered.Crop(crop_left, 0, -crop_right, 0)

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3)
luma_mask = ExpandMask(luma_mask, 3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(src, filtered, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
suppression_mask = LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)

# suppression mask removes too dark areas from chroma mask
suppressed_chroma_mask = mt_lutxy(chroma_mask, suppression_mask, expr="x y -")

# edge mask - detects sharp edges
edges = scharr(src)

# don't detect edges near the frame edges
edges = edges.ZPadding(60, 14, 14, 10)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(src, filtered, delta=5, spotSize=6) # orig. spotSize 6
chroma_override = ExpandMask(chroma_override, 3, blur=2)

# new approach for dark mask:
# select both darker and lighter mask and keep dark mask if it is near lighter mask or overlaps chroma override mask
dark = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
light = LumaDeltaMask(src, filtered, ">", brightness=1.04, spotSize=5)
light = light.ZPadding(60, 14, 14, 10)

dark_near_light = isNearMask(dark, light, 28)
expanded = IsNearMask(dark, light, 28, expanded=true) # for debugging
dark_final = ExpandMask(dark_near_light, 3, blur=4)

# secondary dark mask - doesn't need to be near lighter area but has stricter limits
dark2 = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.69, limitAbsBrightness=0.12, spotSize=5)
dark2 = ExpandMask(dark2, 2, blur=2)
dark_final = mt_logic(dark_final, dark2, "or")

# edge test for dark areas
darkEdgeMask = edges.mt_binarize(54) # at least 54 needed for frame 222
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_final, chroma="-127")
dark_final = mt_lutxy(dark_final, darkEdgeAreaMask, expr="x y -")

# combine luma, chroma and chroma override masks
combined_mask = mt_logic(luma_mask, suppressed_chroma_mask, "or")
combined_mask = mt_logic(combined_mask, chroma_override, "or")

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeMask = edges.mt_binarize(52)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-127")
combined_mask = mt_lutxy(combined_mask, edgeAreaMask, expr="x y -")

# add dark areas (no edge test for those)
combined_mask = mt_logic(combined_mask, dark_final, "or")

# restore ---------------------------------------------------------------------------
delta_restore = Overlay(filtered, src, mask=combined_mask)

edgeAreaMask = mt_logic(edgeAreaMask, darkEdgeAreaMask, "or")
#edgeAreaMask = darkEdgeAreaMask

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(src)
yellow = BlankClip(src, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(src, color=$40f75b)
gray = BlankClip(src, color=$d0d0d0)
darkgray = BlankClip(src, color=$909090)
orange = BlankClip(src, color=$a36b02)
purple = BlankClip(src, color=$4e0091)
purple_light = BlankClip(src, color=$bc6eff)
purple_dark = BlankClip(src, color=$340061)
red = BlankClip(src, color=$9e2424)
blue = BlankClip(src, color=$0d3b7a) #24579e 243c9e

# maskviz for dark mask
maskviz = Overlay(black, purple, mask=expanded)
maskviz = Overlay(maskviz, purple_dark, mask=dark)
maskviz = Overlay(maskviz, purple_light, mask=light)
maskviz = Overlay(maskviz, black, mask=dark_near_light)

# only show dark mask?
showDarkMaskOnly = false
if (!showDarkMaskOnly) {
maskviz = Overlay(maskviz, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = Overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = Overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = Overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = Overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue
}

# mask overlay
comp = Overlay(delta_restore, maskviz, opacity=0.45)
comp = Overlay(comp, purple_light, mask=light, opacity=0.2)
comp = Overlay(comp, black, mask=dark, opacity=0.2)
comp = Overlay(comp, black, mask=dark_final, opacity=0.6)

maskviz = Overlay(maskviz, black, mask=dark_final, opacity=0.6)

#return Interleave(StackHorizontal(src, comp), StackHorizontal(delta_restore, maskviz))
#return StackHorizontal(src, filtered, delta_restore, edgeMask, maskviz)

stack = StackHorizontal(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(edgeMask, "edge mask", txt_sz),\
sub(diff, "diff", txt_sz)\
)

# scaling (if needed)
stack = stack.BilinearResize(stack.width/2 + 1, stack.height/2)

return stack

inter = Interleave(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

return inter

function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

video = Interleave(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(overlay,"delta restore",size = txt_sz,align=2),\
subtitle(restore_nosuppression,"no suppression",size = txt_sz,align=2),\
subtitle(spotless_107_shp,"spotless_107_shp",size = txt_sz,align=2)\
)

masks = Interleave(\
subtitle(BlankClip(source1),"",size = txt_sz,align=2),\
subtitle(combined_mask,"combined_mask",size = txt_sz,align=2),\
subtitle(BlankClip(spotless_107_shp),"",size = txt_sz,align=2)\
)

return StackHorizontal(video, masks)

group = StackHorizontal(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(spotless_107_shp,"spotless_107_shp",size = txt_sz,align=2),\
subtitle(overlay,"delta restore",size = txt_sz,align=2),\
subtitle(combined_mask,"combined mask",size = txt_sz,align=2),\
subtitle(luma_mask,"luma delta mask",size = txt_sz,align=2),\
subtitle(suppression_mask,"suppression mask",size = txt_sz,align=2),\
subtitle(suppressed_chroma_mask,"suppressed chroma mask",size = txt_sz,align=2),\
subtitle(chroma_mask,"chroma delta mask",size = txt_sz,align=2),\
subtitle(diff,"diff",size = txt_sz,align=2))
#subtitle(chroma_mask,"chroma mask",size = txt_sz,align=2),\
#subtitle(whitemask,"whitemask",size = txt_sz,align=2),\
#subtitle(whitemask,"filled whitemask",size = txt_sz,align=2),\
#subtitle(uv,"UV",size = txt_sz,align=2))
#subtitle(whitemask_orig,"orig. whitemask",size = txt_sz,align=2),\
#subtitle(SpotRemoved,"SpotRemover",size = txt_sz,align=2))
#subtitle(spotless_107_shp_Rg,"spotless_107_shp_Rg",size = txt_sz,align=2),\

return(group)#.AddBorders(1000, 0, 500, 0, color=$000000)

#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------

function OverlapsMask(clip mask, clip otherMask) {
return mt_logic(mask, mt_hysteresis(otherMask, mask), "and")
}

function IsNearMask(clip mask, clip otherMask, int maxDistance, bool "expanded") {
expanded = Default(expanded, false)

expand_amount = maxDistance / 2
mask_expanded = ExpandMask(mask, expand_amount)
other_expanded = ExpandMask(otherMask, maxDistance - expand_amount)
common = mt_logic(mask_expanded, other_expanded, "or")
if (expanded) {
return common
}
other_near = mt_hysteresis(otherMask, common)
return mt_logic(mask, other_near, "and")
}

function ZPadding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", float "limitAbsBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)
limitAbsBrightness = Default(limitAbsBrightness, 0.0)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# limitAbsBrightness suppression
if (limitAbsBrightness > 0.0) {
mask = mt_lutxy(mask, source, expr="y range_max "+String(limitAbsBrightness)+" * < 0 x ?")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness or limitAbsBrightness suppression (part 2)
if (limitBrightness != brightness || limitAbsBrightness > 0.0) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff ----------------------------------------------------------------------
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
# mask = mask.ConvertBits(16)
mask = mask.GBlur(rad=6, sd=blur, u=true)
# mask = mask.ConvertBits(10)
}

return mask
}

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

zorr

11th June 2021, 00:03

Some dark mask examples: below you can see a typical case where there are dark deltas near light deltas (easy to see in the diff image). The way dark mask works is that it tests if there are lighter areas nearby and if there are it restores the dark area. How near they have to be can be adjusted with the argument maxDistance of IsNearMask() function. The dark mask does not restore the lighter area itself, it's already handled by the luma mask. The visualization is only showing the purple-tinted dark mask related masks (you can adjust that with the variable showDarkMaskOnly). The allowed maximum distance between the lighter and darker mask is shown as well, if the "edge" of darker and lighter masks overlap then they are close enough.

https://i.postimg.cc/mZjt9L5T/dark-mask-v2-120.png

Not all dark areas have light deltas nearby so there is a secondary dark mask which doesn't have to be near the lighter areas. That one is a bit more conservative on how dark areas it will select. There's a new limit parameter limitAbsBrightness in LumaDeltaMask() which rejects mask if most of the brightness is below that value. Note that this is not the diff / delta brightness but the brightness of the source clip under the mask location. This removes dark spots which are located over dark features (like in the shadows of a building), those don't have a large delta but the spot is so dark that it is easy to detect. Such spots occur at frames 79, 108, 190 and 227 and can be dealt with limitAbsBrightness=0.12.

https://i.postimg.cc/ZKDnffmg/dark-mask-v2-137.png

Let's see how the guy's hair is doing:

https://i.postimg.cc/Gm826khp/dark-mask-v2.gif

Delta Restore has brought back all the missing details. The gif is now showing all the masks, you can see for example the green chroma override mask.

The dark mask selects quite many unnecessary, sometimes large features. However those are mostly harmless as the change is so tiny. You can see some of those in the above gif, here's another example of a larger feature:

https://i.postimg.cc/qM7gXPPy/dark-mask-v2-129.png

It does select some faint spots as well, but that's the price you have to pay in order to restore the dark features. I still think it's a good compromise and improves the end result.

[EDIT] Here's an example of a faint spot which is restored by the dark mask.

https://i.postimg.cc/pLzg4T8j/dark-mask-v2-199.png

kedautinh12

11th June 2021, 00:11

Thanks

kedautinh12

11th June 2021, 02:50

I think your script need use with DeScratch for remove some black line dirty

johnmeyer

11th June 2021, 15:09

I think your script need use with DeScratch for remove some black line dirtyRemoving movie film scratches is extremely difficult to do automatically without extensive user interaction with each frame. Nothing works well. The problem is that scratches persist in the same place from one frame to the next and therefore look like they are part of the scene. In "Hollywood" restorations they have a team of restorationists who paint each scratch and then use motion tracking software to track that scratch as it wanders slightly back and forth from one frame to the next, often persisting for many minutes.

The better way to remove scratches is during the transfer from film to video using a wet gate transfer system.

chmars

11th June 2021, 20:23

I still think it's a good compromise and improves the end result.

This seems very beautiful. Lots of sweat I guess.
I'll try that on the next footage, next week. Coming back asap to give a feed back.

zorr

11th June 2021, 22:14

I think your script need use with DeScratch for remove some black line dirty

Yes I think so too. You can see an example of DeScratch in this post (https://forum.doom9.org/showthread.php?p=1944417#post1944417).

That was just a quick test and it already removes like 90% of the scratches. It's never going to work as well as restorationists doing it frame by frame but for an easily applied filter... not bad at all.

chmars

11th June 2021, 22:34

I tried too, works well on graded footage.
But you have to rotate the image as the scratches are as vertical as possible (in case the camera is not 100% at capturing time).
So you can lower the orientation parameter "maxangle" to avoid corrections on unwanted objects.

Descratch is much less efficient with Slog3 though, contrast is too low.

Arx1meD

14th June 2021, 09:43

My version of the script to restore moving objects. Unfortunately some spots also returned. Not as complex as the zorr version.

# v1.1
# Needed plugins: MaskTools2, RgTools

function Restore_Objects(clip source, clip filtered, float "threshold_dark", float "threshold_light", int "Y", int "U", int "V") {
dark = String(Default(threshold_dark, 20.0)) # Threshold for dark objects. Range: 0...127
light = String(Default(threshold_light, 20.0)) # Threshold for light objects. The lower the thresholds, the more objects will be restored.
Y = Default(Y, 3)
U = Default(U, 2)
V = Default(V, 2)

m1 = mt_lut(source.ColorYUV(autogain=true), expr="x "+dark+" > x 255 "+light+" - < 0 255 ? 255 ?", Y=Y, U=U, V=V, scale_inputs="allf", use_expr=2).mt_expand().mt_expand().mt_expand(chroma="-128")
m2 = mt_lutxy(source, filtered, expr="x y * 255 /", Y=Y, U=U, V=V, scale_inputs="allf", use_expr=2).mt_binarize(threshold=128, chroma="-128")
mask_1 = mt_logic(m1, m2, "max").mt_expand().mt_expand(chroma="-128").Blur(1.58)#.mt_convolution(horizontal="3 1 3", vertical="3 1 3", Y=Y, U=U, V=V)

r = mt_merge(source.RemoveDust(), filtered, mask_1, Y=Y, U=U, V=V)

m3 = mt_lutxy(source, r, expr="x y - abs", Y=Y, U=U, V=V, scale_inputs="allf", use_expr=2).mt_binarize(threshold=5).mt_inpand(chroma="-128", thY=128, thC=128)
m4 = m3.mt_inpand().mt_inpand(chroma="-128")
mask_2 = mt_logic(m3, m4, "xor").Blur(1.58)#.mt_convolution(horizontal="3 1 3", vertical="3 1 3", Y=Y, U=U, V=V)

mt_merge(Repair(filtered, r, 17), filtered.RemoveDust(), mask_2, Y=Y, U=U, V=V)
}

function RemoveDust(clip input, int "mode") {
mode = Default(mode, 17)

clensed = Clense(input)
rep = Repair(clensed, input, mode=mode)

RemoveGrain(rep, mode=10)
}

Comparison script:

source = DSS2("C:\464_-trim_crop_grd_cut.avi").ConvertBits(16)
filtered = source.SpotLess(RadT=1, Pel=1, BlkSz=16, Chroma=true, Tm=true)
Restore_Objects = Restore_Objects(source, filtered, threshold_dark=30, threshold_light=30, Y=3, U=2, V=2)

StackHorizontal(source.AddBorders(0, 0, 0, 40).Subtitle("Source", align=2, size=30),
\ filtered.AddBorders(0, 0, 0, 40).Subtitle("Filtered: SpotLess(RadT=1, Pel=1, BlkSz=16, Chroma=true, Tm=true)", align=2, size=30),
\ Restore_Objects.AddBorders(0, 0, 0, 40).Subtitle("Restore_Objects(source, filtered, threshold_dark=30, threshold_light=30, Y=3, U=2, V=2)", align=2, size=30))

Comparison screenshots:
https://thumbs2.imgbox.com/d4/e1/6sREjVoT_t.jpg (https://imgbox.com/6sREjVoT) https://thumbs2.imgbox.com/cf/ed/J0L9nTyR_t.jpg (https://imgbox.com/J0L9nTyR) https://thumbs2.imgbox.com/50/8b/c4iYlYhB_t.jpg (https://imgbox.com/c4iYlYhB) https://thumbs2.imgbox.com/af/17/5Yd473hG_t.jpg (https://imgbox.com/5Yd473hG) https://thumbs2.imgbox.com/51/b3/x1VdS9tF_t.jpg (https://imgbox.com/x1VdS9tF) https://thumbs2.imgbox.com/1e/13/1QzVj147_t.jpg (https://imgbox.com/1QzVj147)

kedautinh12

14th June 2021, 10:56

Thanks

chmars

15th June 2021, 13:52

kedautinh12

15th June 2021, 15:36

I think it's same cause it's same from vcmohan

johnmeyer

15th June 2021, 16:50

Here's a good clip to test whether you can mask out a white ball being thrown. When using RemoveDirtMC or Spotless with sufficiently strong settings needed to remove the dirt, the ball disappears in flight.

Dirt Removal "Torture" Test Clip (https://www.mediafire.com/file/nbvg27jotdthkv1/Dirt_Removal_Test_Clip_-_Thrown_Ball.mp4)

The clip is 16 fps progressive.

I was hoping to be able to use the software posted in this thread, but didn't have the time to turn the masking software into a function so I could import it into my existing film restoration script. Spotless is no problem.

The results posted here do look intriguing, but I need to get this project out the door so I did my usual workaround, namely doing a second render with the dirt removal turned way down, and then using that version during the time the ball is in flight.

Arx1meD

15th June 2021, 19:43

johnmeyer, try my version of SpotLess_Mod. This is the best I can do to keep as much detail as possible!
Expect some spots to remain.

Function SpotLess_Mod(clip c, bool "DeGrain", int "RadT", int "ThSAD", int "ThSAD2", int "Pel", bool "Chroma",
\ int "BlkSz", Int "Olap", bool "Tm", bool "Glob", string "Denoiser") {
DeGrain = Default(DeGrain, false) # MDeGrain in MvTools
RadT = Default(RadT, 1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD, 10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2, ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel, 1) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma, true) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz, 8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm, true) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob, true) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
Denoiser = Default(Denoiser, Undefined)

pad = Max(BlkSz, 8)

sup = c.Pre_Clip(blur=1.0, contrast=0, brightness=0).MSuper(hpad=pad, vpad=pad, pel=pel, sharp=2, chroma=Chroma)
sup_rend = MSuper(c, hpad=pad, vpad=pad, pel=pel, sharp=2, levels=1, chroma=Chroma) # Only 1 Level required where not MAnalyse-ing.

MultiVec = MAnalyse(sup, multi=true, delta=RadT, blksize=BlkSz, overlap=OLap, chroma=Chroma, truemotion=Tm, global=Glob, search=5, dct=7, divide=0, plevel=0)
#~ MultiVec = Mrecalculate(sup, MultiVec, tr=RadT, blksize=4, overlap=2, search=5, dct=7, thSAD=150, chroma=Chroma) # bed vectors recalculate

c0 = MCompensate(c, sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
c0 = RadT == 1 ? Clense(c0) : MedianBlurTemporal(c0, radiusY=0, radiusU=0, radiusV=0, temporalradius=RadT)
c0 = SelectEvery(c0, RadT*2+1, RadT) # Return middle frame
c1 = Restore_Objects(c, c0, threshold_dark=30, threshold_light=30, Y=3, U=2, V=2)

clensed = Clense(c, cache=4)
# sbegin = ForwardClense(c, cache=-1)
# send = BackwardClense(c, cache=-1)
# alt=Repair(SCSelect(c, sbegin, send, clensed, debug=true), c, mode=1)
restore = Repair(clensed, c, mode=1)

c3 = RestoreMotionBlocks(clensed, restore, neighbour=c, alternative=restore, gmthreshold=70, dist=1, dmode=2, noise=10, noisy=12, grey=false, debug=false, show=false)

mask_c3 = mt_lutxy(c0, c3, expr="x y - abs", Y=3, U=2, V=2, scale_inputs="allf", use_expr=2).mt_binarize(threshold=7)
\ .mt_inpand().mt_expand().mt_expand().mt_expand().Blur(1.58)#.mt_convolution(horizontal="3 1 3", vertical="3 1 3", Y=3, U=2, V=2, chroma="-128")

clear = mt_merge(c1, c, mask_c3, Y=3, U=2, V=2)

Defined(Denoiser) || DeGrain ? Eval("""
sup2 = MSuper(clear, hpad=pad, vpad=pad, pel=pel, sharp=2, chroma=Chroma)
sup_rend2 = MSuper(clear, hpad=pad, vpad=pad, pel=pel, sharp=2, chroma=Chroma)
bvec1 = MAnalyse(sup2, isb=true, blksize=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0,
\ chroma=Chroma, truemotion=Tm, global=Glob) # backward vectors
fvec1 = MAnalyse(sup2, isb=false, blksize=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0,
\ chroma=Chroma, truemotion=Tm, global=Glob) # forward vectors
# bvec1 = Mrecalculate(sup2, bvec1, blksize=4, overlap=2, search=5, dct=7, thSAD=150, chroma=Chroma) # bed vectors recalculate
# fvec1 = Mrecalculate(sup2, fvec1, blksize=4, overlap=2, search=5, dct=7, thSAD=150, chroma=Chroma) # bed vectors recalculate
backw1 = MFlow(clear, sup_rend2, bvec1)
forw1 = MFlow(clear, sup_rend2, fvec1)
DN = Defined(Denoiser) ? Interleave(backw1, clear, forw1).Eval(Denoiser).SelectEvery(3, 1) : clear
return DeGrain ? MDeGrain1(DN, sup_rend2, bvec1, fvec1, plane=4, limit=255.0, thSad=400) : DN
""") : clear
}

function RemoveDust(clip input, int "mode") {
mode = Default(mode, 17)

clensed = Clense(input)
rep = Repair(clensed, input, mode=mode)

RemoveGrain(rep, mode=10)
}

function Pre_Clip(clip clp, float "blur", float "contrast", float "brightness"){
blur = Default(blur, 1.0)
contrast = Default(contrast, 1.2)
brightness = Default(brightness, 10.0)

v = clp.mt_lut(expr="x 20 1 x 127.5 / - 21 ^ pi x * 255 / cos 21 ^ - * 1.05 * -", Y=3, U=2, V=2, scale_inputs="allf", use_expr=2)

mask_1 = clp.Tweak(sat=0).Blur(blur).mt_edge(mode="min/max", thY1=0, thY2=255, Y=3, U=2, V=2)

pre_1 = v.Tweak(cont=contrast, bright=brightness)
pre_2 = mt_merge(v, pre_1, mask_1, Y=3, U=2, V=2)
pre_3 = pre_2.Tweak(sat=0).mt_invert().Blur(1.58).Blur(1.58).Blur(1.58)
pre_4 = mt_lutxy(pre_2, pre_3, expr="x 127.5 > y 255 x - 127.5 / * x 255 x - - + y x 127.5 / * ? ", scale_inputs="allf", use_expr=2)

return pre_4
}

# v1.1
function Restore_Objects(clip source, clip filtered, float "threshold_dark", float "threshold_light", int "Y", int "U", int "V") {
dark = String(Default(threshold_dark, 20.0)) # Threshold for dark objects. Range: 0...127
light = String(Default(threshold_light, 20.0)) # Threshold for light objects. The lower the thresholds, the more objects will be restored.
Y = Default(Y, 3)
U = Default(U, 2)
V = Default(V, 2)

m1 = mt_lut(source.ColorYUV(autogain=true), expr="x "+dark+" > x 255 "+light+" - < 0 255 ? 255 ?", Y=Y, U=U, V=V, scale_inputs="allf", use_expr=2).mt_expand().mt_expand().mt_expand(chroma="-128")
m2 = mt_lutxy(source, filtered, expr="x y * 255 /", Y=Y, U=U, V=V, scale_inputs="allf", use_expr=2).mt_binarize(threshold=128, chroma="-128")
mask_1 = mt_logic(m1, m2, "max").mt_expand().mt_expand(chroma="-128").Blur(1.58)#.mt_convolution(horizontal="3 1 3", vertical="3 1 3", Y=Y, U=U, V=V)

r = mt_merge(source.RemoveDust(), filtered, mask_1, Y=Y, U=U, V=V)

m3 = mt_lutxy(source, r, expr="x y - abs", Y=Y, U=U, V=V, scale_inputs="allf", use_expr=2).mt_binarize(threshold=5).mt_inpand(chroma="-128", thY=128, thC=128)
m4 = m3.mt_inpand().mt_inpand(chroma="-128")
mask_2 = mt_logic(m3, m4, "xor").Blur(1.58)#.mt_convolution(horizontal="3 1 3", vertical="3 1 3", Y=Y, U=U, V=V)

mt_merge(Repair(filtered, r, 17), filtered.RemoveDust(), mask_2, Y=Y, U=U, V=V)
}

SpotLess_Mod(RadT=1, Pel=1, BlkSz=8, Chroma=true, DeGrain=false, Denoiser=Undefined)

zorr

15th June 2021, 21:07

I have seen Gblur is used in the last version.
For the tests, I replaced it with the avs+ Gblur from manyPlus (http://www.avisynth.nl/users/vcmohan/manyPlus/manyPlus.html)by vcmohan (https://forum.doom9.org/showthread.php?t=174162) instead of the standard 32bit Gblur (http://avisynth.nl/index.php/GBlur).
Is it ok to do so?

Yes, that's the version I used myself. I had to find a blur function that can deal with HBD, there are not many of those. FastBlur would be great but it has some glitches and I wasn't able to create a simple test scenario to report the bug.

chmars

16th June 2021, 05:56

Hmm, too bad... How bad are the glitches? Often?
Because Gblur makes the script 3x slower (well, it's said in the name :) ) than fastblur (fps reported while using Vdub, looked on the ~25 first frames).

Zorr, if you allow, I can post the script version I am using where I turned Delta Restore into a function (updated to last version).
I know it works but can't report precise facts yet as I haven't graded the last footage yet, this allowing to see precisely what happens.

EDIT: But... Maybe bad question but, Is it really important to build the masks in HBD? Couldn't we build them in LBD and apply them in HBD?

chmars

16th June 2021, 06:54

Couldn't resist. Quick and dirty test.
Seems promising.
Had to resize the video to avoid changing all the size parameters. Forgot to resize it back.

https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/8h2CZrBTueygcxZ/download (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/FwKmbrUZyMob1JB/download)
EDIT: (video under the picture)

chmars

16th June 2021, 22:32

After two hours examining the results of the last Delta Restore version used on my footage, I can share some feedback.

For the recall, this footage is old, captured in Slog with some overexposed scenes. So, generally very washed/light.
For this reason, I used Autolevels (In the right manner this time, corrected posts 82 + 83 (https://forum.doom9.org/showthread.php?p=1944383#post1944383) ) to generate a clip for the masks calculation.
Then, the masks are applied on the original clip.
I don't post example images as I don't want to generate too fast conclusions on some pics only. I prefer to share my impressions. There are much parameters and results can vary a lot, also depending on the images used. I kept the defaults parameters.

As said, Spotless strongly removes spots, hairs, dirt. But sometimes, also moving objects.
Delta Restore brings back these objects:
-Light objects always come back as well as light/dark objects.
-Dark only objects come back but also some medium spots and an important proportion of present hairs. All small spots are away. Rare large spots sometimes come back, sometimes not.

However, the resulting images are much cleaner than the source.
Feeling: "There are some spots sometimes, maybe 10-15% remaining, but image is much better" And people parts/objects are back.

Spotless + Delta Restore in this last flavour are perfect for a not-too-much-time-consuming use, with a good result and the security of not deleting important parts. It is well adapted to a systematic treatment, making sure that arms, balloons, feet, birds will not be deleted.

For a very perfect result, I don't know if it is preferable to: either adapt parameters to every type of images and dirt, or use the previous version of Delta Restore without dark masks and use some transparency to bring back (rare) dark only disappeared objects.
EDIT: This version has many changes and work on the masks. Maybe use this version and lower the impact of one of the dark masks?

Delta Restore also has many nice ways to visualize what is happening with all the coloured overlays.

zorr

17th June 2021, 00:13

I tried to come up with a more user friendly version of Delta Restore. I don't think making one huge function with a gazillion parameters is the way to go, Delta Restore works better as a toolbox of functions which can be adapted to almost any situation. However I still wanted to make a single function that takes all the different masks, combines them and creates the final result.

Here's the previous version written in the new style. Some lines are there only to create the debug visualization, I need to create another function for that.

#source1= AviSource("459_s2_002-XAVCI-4K-BLD_5_0-trim_crop-for_tests.avi")
#source1 = crop(source1,700,0,-500,0)

#source1= FFVideoSource("ZorrBack_test.avi")
#source1 = crop(source1,450,50,1352,-40)

source1= FFVideoSource("464_-trim_crop_grd_cut.avi")
source1= KillAudio(source1)

source1_8bit = ConvertBits(source1, bits=8) #I dont'know much about formats, maybe this is not ideal...
source1YV12 = ConvertToYV12 (source1_8bit) #I dont'know much about formats, maybe this is not ideal...

txt_sz = 36

BlkSz = 24
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Sharpen----------------------------------------------------------------------
source1_shp=source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#SpotRemoved_shp=SpotRemoved.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#----------------------------------------------------------------------------------

#Remove spots-----------------------------------------------------------------
spotless_107=SpotLess107(source1_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
#SpotRemoved = SpotRemover(source1YV12,Spot=128, RadT=1, Pel=Pel, BlkSz=32, DeGrain=false, Chroma=true, PreFilter=true, mask=false, Tm = true,thsad=thsad, Glob = true)
#----------------------------------------------------------------------------------

# Delta Restore -------------------------------------------------------------

src = source1
filtered = spotless_107

# extra clipping (if needed)
crop_left = 20
crop_right = 1120
#src = src.Crop(crop_left, 0, -crop_right, 0)
#filtered = filtered.Crop(crop_left, 0, -crop_right, 0)

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(src, filtered, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
# suppression mask removes too dark areas from chroma mask
suppression_mask = LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)
suppressed_chroma_mask = chroma_mask.SuppressWith(suppression_mask)

# edge mask - detects sharp edges
# don't detect edges near the frame edges
edges = scharr(src).ZPadding(60, 14, 14, 10)
edgeMask = edges.mt_binarize(52)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(src, filtered, delta=5, spotSize=6).ExpandMask(3, blur=2)

# dark mask: select both darker and lighter mask and keep dark mask if it is near lighter mask or overlaps chroma override mask
dark = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
light = LumaDeltaMask(src, filtered, ">", brightness=1.04, spotSize=5).ZPadding(60, 14, 14, 10)
dark_near_light = dark.MustBeNear(light, 28).ExpandMask(3, blur=4)
expanded = dark.MustBeNear(light, 28, expanded=true) # for debugging

# secondary dark mask - doesn't need to be near lighter area but has stricter limits
dark2 = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.69, limitAbsBrightness=0.12, spotSize=5).ExpandMask(2, blur=2)

# edge test for dark areas
darkEdgeMask = edges.mt_binarize(54) # at least 54 needed for frame 222
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_near_light.AddTo(dark2), chroma="-128") # for debug

dark_final = DarkDeltaMask(dark_near_light, secondary=dark2, edgeMask=darkEdgeMask)

combined_mask = luma_mask.AddTo(suppressed_chroma_mask).AddTo(chroma_override)

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-128") # for debug

delta_restore = DeltaRestore(filtered, src, luma=luma_mask, chroma=suppressed_chroma_mask, chromaOverride=chroma_override, edges=edgeMask, dark=dark_final)

edgeAreaMask = mt_logic(edgeAreaMask, darkEdgeAreaMask, "or")
#edgeAreaMask = darkEdgeAreaMask

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(src)
yellow = BlankClip(src, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(src, color=$40f75b)
gray = BlankClip(src, color=$d0d0d0)
darkgray = BlankClip(src, color=$909090)
orange = BlankClip(src, color=$a36b02)
purple = BlankClip(src, color=$4e0091)
purple_light = BlankClip(src, color=$bc6eff)
purple_dark = BlankClip(src, color=$340061)
red = BlankClip(src, color=$9e2424)
blue = BlankClip(src, color=$0d3b7a) #24579e 243c9e

# maskviz for dark mask
maskviz = Overlay(black, purple, mask=expanded)
maskviz = Overlay(maskviz, purple_dark, mask=dark)
maskviz = Overlay(maskviz, purple_light, mask=light)
maskviz = Overlay(maskviz, black, mask=dark_near_light)

# only show dark mask?
showDarkMaskOnly = false
if (!showDarkMaskOnly) {
maskviz = Overlay(maskviz, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = Overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = Overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = Overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = Overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue
}

# mask overlay
comp = Overlay(delta_restore, maskviz, opacity=0.45)
comp = Overlay(comp, purple_light, mask=light, opacity=0.2)
comp = Overlay(comp, black, mask=dark, opacity=0.2)
comp = Overlay(comp, black, mask=dark_final, opacity=0.6)

maskviz = Overlay(maskviz, black, mask=dark_final, opacity=0.6)

stack = StackHorizontal(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(edgeMask, "edge mask", txt_sz),\
sub(diff, "diff", txt_sz)\
)

# scaling (if needed)
stack = stack.BilinearResize(stack.width/2 + 1, stack.height/2)

return stack

inter = Interleave(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

return inter

function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------

function DeltaRestore(clip filtered, clip source, clip "luma", clip "chroma", clip "chromaOverride", clip "edges", clip "dark") {
Assert(luma.Defined || chroma.Defined || chromaOverride.Defined || dark.Defined, "DeltaRestore:must define at least one of luma, chroma, chromaOverride, dark")

mask = Default(luma, BlankClip(filtered))
if (chroma.Defined) {
mask = mask.AddTo(chroma)
}
if (chromaOverride.Defined) {
mask = mask.AddTo(chromaOverride)
}
if (edges.Defined) {
mask = mask.MustNotOverlap(edges)
}
if (dark.Defined) {
mask = mask.AddTo(dark)
}
return Overlay(filtered, source, mask=mask)
}

function DarkDeltaMask(clip mask, clip "secondary", clip "edgeMask") {
if (secondary.Defined) {
mask = mask.AddTo(secondary)
}
if (edgeMask.Defined) {
mask = mask.MustNotOverlap(edgeMask)
}
return mask
}

function AddTo(clip mask, clip other) {
return mt_logic(mask, other, "or")
}

function SuppressWith(clip mask, clip suppressionMask) {
return mt_lutxy(mask, suppressionMask, expr="x y -")
}

function MustNotOverlap(clip mask, clip other) {
other = mt_hysteresis(other, mask, chroma="-128")
return mt_lutxy(mask, other, expr="x y -")
}

function MustOverlap(clip mask, clip otherMask) {
return mt_logic(mask, mt_hysteresis(otherMask, mask), "and")
}

function MustBeNear(clip mask, clip otherMask, int maxDistance, bool "expanded") {
expanded = Default(expanded, false)

expand_amount = maxDistance / 2
mask_expanded = ExpandMask(mask, expand_amount)
other_expanded = ExpandMask(otherMask, maxDistance - expand_amount)
common = mt_logic(mask_expanded, other_expanded, "or")
if (expanded) {
return common
}
other_near = mt_hysteresis(otherMask, common)
return mt_logic(mask, other_near, "and")
}

function ZPadding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", float "limitAbsBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)
limitAbsBrightness = Default(limitAbsBrightness, 0.0)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# limitAbsBrightness suppression
if (limitAbsBrightness > 0.0) {
mask = mt_lutxy(mask, source, expr="y range_max "+String(limitAbsBrightness)+" * < 0 x ?")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness or limitAbsBrightness suppression (part 2)
if (limitBrightness != brightness || limitAbsBrightness > 0.0) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
mask = mask.GBlur(rad=6, sd=blur, u=true)
}
return mask
}

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

zorr

17th June 2021, 00:21

If you don't need the debug visualization you can now call Delta Restore as one function call (although the parameters themselves are sometimes function calls). But it's easier to read and every setting is there in one place. Also maybe you don't need all the features, you can select which masks you want to use. You only need to provide one of luma, chroma, chromaOverride or dark masks. Some examples:

# example call with luma mask only
restored = DeltaRestore(filtered, src, luma = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4))

# example call with luma and chroma only
restored = DeltaRestore(filtered, src, \
luma = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4), \
chroma = ChromaDeltaMask(src, filtered, delta=3, spotSize=3) \
)

# example call with luma and suppressed chroma only
restored = DeltaRestore(filtered, src, \
luma = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4), \
chroma = ChromaDeltaMask(src, filtered, delta=3, spotSize=3).SuppressWith(LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)) \
)

# example call with luma, suppressed chroma and chroma override only
restored = DeltaRestore(filtered, src, \
luma = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4), \
chroma = ChromaDeltaMask(src, filtered, delta=3, spotSize=3).SuppressWith(LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)), \
chromaOverride = ChromaDeltaMask(src, filtered, delta=5, spotSize=6).ExpandMask(3, blur=2) \
)

# example call with luma, suppressed chroma, chroma override and edge mask only
restored = DeltaRestore(filtered, src, \
luma = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4), \
chroma = ChromaDeltaMask(src, filtered, delta=3, spotSize=3).SuppressWith(LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)), \
chromaOverride = ChromaDeltaMask(src, filtered, delta=5, spotSize=6).ExpandMask(3, blur=2), \
edges = src.scharr().ZPadding(60, 14, 14, 10).mt_binarize(52) \
)

# example call with luma, suppressed chroma, chroma override, edge mask and a simple dark mask only
restored = DeltaRestore(filtered, src, \
luma = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4), \
chroma = ChromaDeltaMask(src, filtered, delta=3, spotSize=3).SuppressWith(LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)), \
chromaOverride = ChromaDeltaMask(src, filtered, delta=5, spotSize=6).ExpandMask(3, blur=2), \
edges = src.scharr().ZPadding(60, 14, 14, 10).mt_binarize(52), \
dark = DarkDeltaMask(LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.69, limitAbsBrightness=0.12, spotSize=5).ExpandMask(2, blur=2)) \
)

# example call with luma, suppressed chroma, chroma override, edge mask and an advanced dark mask
restored = DeltaRestore(filtered, src, \
luma = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4), \
chroma = ChromaDeltaMask(src, filtered, delta=3, spotSize=3).SuppressWith(LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)), \
chromaOverride = ChromaDeltaMask(src, filtered, delta=5, spotSize=6).ExpandMask(3, blur=2), \
edges = src.scharr().ZPadding(60, 14, 14, 10).mt_binarize(52), \
dark = DarkDeltaMask( \
LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.66, spotSize=5) \
.MustBeNear(LumaDeltaMask(src, filtered, ">", brightness=1.04, spotSize=5).ZPadding(60, 14, 14, 10), 28).ExpandMask(3, blur=4), \
secondary = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.69, limitAbsBrightness=0.12, spotSize=5).ExpandMask(2, blur=2), \
edgeMask = src.scharr().ZPadding(60, 14, 14, 10).mt_binarize(54) \
) \
)

I hope this works for John. :)

zorr

17th June 2021, 00:40

Hmm, too bad... How bad are the glitches? Often?
Because Gblur makes the script 3x slower

It's quite rare but I don't want to have any chance of ruined masks. The glitch appears as a line or lines that extend from a mask all the way to the bottom of the screen.

Zorr, if you allow, I can post the script version I am using where I turned Delta Restore into a function (updated to last version).

Please do, I welcome all mods and improvements.

EDIT: But... Maybe bad question but, Is it really important to build the masks in HBD? Couldn't we build them in LBD and apply them in HBD?

Yes the masks could be 8-bit, the end result would be practically same.

kedautinh12

17th June 2021, 01:29

Thank zorr

StainlessS

25th June 2021, 13:58

Just a heads up.
Some time ago I used Spotless on 1946 PAL DVD "A Matter of Life and Death" AKA "Stairway to Heaven" in the USA [David Niven, Kim Hunter].
Just noticed today that in the 2nd section of Table Tennis, not a single ping pong ball was left after Spotless had swallowed the lot. [RadT=2]
Anyways, might be a good test piece if you've got the DVD. [Ping pong balls being white will probably defeat attempts to keep the balls].

https://en.wikipedia.org/wiki/A_Matter_of_Life_and_Death_(film)

chmars

25th June 2021, 22:36

These situations will probably be much enhanced with Delta Restore.

In the next post, the whole script I used on the last 10 reels of 8mm to:
-Clean the spots
-restore the swallowed parts by Spotless
-Sharpen

I hope this script being user friendly.
I added some explanations for noobs like me.

Added some variables to easily chose:
-to grade a clip just for the masks construction
-chose fastblur or Gblur
-restore grain or not
-sharpen or not

On the 8mm reels, I applied this script before grading (Resolve).
So I had time to verify the result while grading.
-> Very clean and very few image parts disappeared.
Once more, never enough, thanks to StainlesS, Zorr and Real.finder.

I don't say the result is 100% perfect. One could find to say, by looking frame by frame. But huge majority for sure! I
mperfections will not be noticeable by looking at the result at normal speed, by far.
Some flying rice (wedding) and some splashing drops disappeared sometimes. But balls, arms, legs... All was there!

What could be improved by tweaking variables:
-blur the returned parts edges a bit more. But I used FastBlur instead of Gbur, Gblur is recommanded by Zorr.

Descratch could also be added but I didn't, as I wished to keep 10bit color depth.

chmars

25th June 2021, 22:37

Part 1
/*
SpotDel (Spotless+Delta Restore) 1.0

Script intended to clean and sharpen digitalized film.
Can optionally restore grain after cleaning.
Sharpening and denoising are basic. Please see doom9 forum, VideoFred's and JohnMeyer's scripts and posts fore more knowlege. (VideoFred's sharp-blur-sharp-blur-sharp trick). Please share improvements!

Functions: Spotless:StainlessS, Delta Restore:Zorr, Restore Grain:real.finder. Huge Thanks !
Script: chmars - 17/06/2021 - some parts copied here and there, thank to solutions given by doom9 forum users.
Thread: https://forum.doom9.org/showthread.php?t=182831
Spotless thread, last versions: https://forum.doom9.org/showthread.php?t=181777
Lots of informaions on film capture/restoration: https://forum.doom9.org/showthread.php?t=144271

Script tested on HD 4:2:2 10bits sources with Avisynth+370_x64
! ! ! ! !
-You should maybe adapt the sizes parameters (blksz, radiuses, spotsizes) if your source is not HD

How it works:
Spotless removes dirt spots that are not present in neibourg frames.
But sometimes, fast moving objects are removed too (balloons, birds, hands, feet)
Hence is Delta Restore needed. It brings back details that are part of the image.
As Spotless tends to soften/degrain the image, parts brought back from the original image by Delta Restore are more grainy.
The solution for a discreete replacement (and nicer result) of these parts is to sharpen the Spotless input.

Washed out, Slog rushes:
Spotless seems to work as well with washed out images (old footage, Slog rushes), but Delta Restore needs colors and contrast to funtion properly.
In this case, a more contrasted clip can be made to feed Delta Restore and only generate masks. However, the final result will not be graded.

*/

#~ SetMemoryMax(8000) # Dunno if optimal...
#~ threads = 16 # Dunno if optimal...
#~ ### Check speed, memory consumption and CPU usage with AVSMeter64 ###
#~ SetFilterMTMode("DEFAULT_MT_MODE", 2) # Dunno if optimal...

# VARIABLES #################################################################################################
########################################################################################################################
########################################################################################################################
# Here, define what you want

# Path to file
video_file = "D:\some_path\some_file"

play_speed = 25 # This will not change your frames/fields, neither their count
txt_sz = 36 #Test size if infos displayed
Sharpen_it = True # Default True. Sharpen clip or not (recomanded, see intro)
Slog = False # Default False. If True, Generate a leveled clip f or wahed-out or Slog rushes. Final result will not be changed levelwise

#Grainitude, cancels sharpen effect
Rgr = False # Default false. True Restores grain after passed through Spotless. Might restore some dust too. 5,10 = good compromise grain/dirt.
val1 = 10
val2 = 20 # 5,10 = good compromise grain/dirt. Dunno how that works exactly...

# Sharpen variables ----------------------------------------------------------------------------------------------------------------------------------------------------------
Usharp_strength= 80 # Bigger = sharper & more grain
Usharp_radius= 5
Usharp_th = 1

# Spotless variables, to remove dirt and spots. -------------------------------------------------------------------------------------------------------------------------
# See Function script for more advanced settings
BlkSz =24 #Spot Size, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]
Pel = 1 # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)
Tm=false #TrueMotion Default True. Some folk swear truemotion=false is better.
Bblur = 0.6 #Default 0.6. If used, Suggest about 0.6, where MAnalyse create vectors is performed on denoised (blurred) super clip for better motion analysis.
ThSAD = 10000 # [SAD, Sum of Absolute (pixelwise) Differences] Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1)
RadT = 1 #Removes Spots on up to RadT [Temporal Radius] consecutive frames. RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
#Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

#Delta Restore variables
dark2Brt = 0.85 #Default 0.97. Decrease if too many spots coming back
dark2BrtLimit = 0.86 # Default 0.86. Same mask. Try to change it too.
Global RiskyBlur = true # True = use fastblur and risk some glitches. False = Use Gblur and go stable, at the price of a slower script

#Denoise----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
# If needed, removes grain
#Not deeply tested, please test and report
#~ denoising_strength= 200 #denoising level of first denoiser: MDegrain() # high reduces colors
#~ block_size= 4 #block size of MVDegrain
#~ block_size_v= 4
#~ block_over= 2 #block overlapping of MVDegrainMulti()
#~ dirt_strength=10 #sets amount of dirt removal (big spots)

# / END OF VARIABLES ######################################################################################################
########################################################################################################################
########################################################################################################################

#~ source1= FFVideoSource("D:\davinci_eric_folder\rushes_cathy_new_system\463_cathy_s2_005-XAVCI-4K-BLD_5_0-trim_crop.mov").KillAudio()
#~ source1= AviSource("D:\davinci_eric_folder\rushes_cathy_new_system\462_cathy_s2_004-XAVCI-4K-BLD_5_0-trim_crop.mov").KillAudio()
source1 = FFVideoSource(video_file).KillAudio()
# Set play speed (if needed)
source1 = assumefps(source1,play_speed)

source1 = source1.Trim(294, 327)
source1 = source1.BilinearResize(source1.width*3 + 0, source1.height*3)
#~ source1 = crop(source1, 500, 200, -400, -200)
#~ source1 = source1.BilinearResize(source1.width/2 + 0, source1.height/2)

# Optional
## ------------------------------------------------------------------------------------
# Keep only from frame n to frame m
# source1 = trim(source1, 2,0)
# Resize (speeds-up for tests?)
# source1 = source1.BilinearResize(source.width/2 , source.height/2)

# Leveled clip
## -------------------------------------------------------------------------------------------------------------------------------------------------------------
#In case of washed-out footage or Slog captures
# Needed to generate dark and saturated enough clips, to feed the filters whith what they need.
# Will NOT apply to final result. Only for mask building
# If Slog = True, make a decently contrasted clip to feed the filters.
# Else, do nothing, source1_lvl will be identical to source1

source1_lvl = Slog ? source1.convertbits(8).Autolevels(filterradius = 3, sceneChgThresh = 20, autogamma = true, midpoint = 0.5, autolevel = true, border=500).convertbits(16) : source1
#~ lvl_src = Slog ? source1.convertbits(8).Autolevels(filterradius = 3, sceneChgThresh = 20, autogamma = true, midpoint = 0.5, autolevel = true, border=500, input_high = 230 ).convertbits(16) : source1
#~ lvl_src = Slog ? ColorYUV(source1, autogain = true) : source1

#Check it
##-----------------------------------------------------------------------------------
# If needed, uncomment "return..." lines to verify levels of mask construction graded clip
h = Histogram(source1,"classic")
h_lvl = Slog ? Histogram(source1_lvl,"classic") : NOP
#~ return(StackHorizontal(h, h_lvl)) # Un comment to see clips with histograms
#~ return(source1)

#Sharpening
# -----------------------------------------------------------------------------------
# Best results found by sharpening before (not after) cleaning.
source1_shp = Sharpen_it ? source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th) : source1
# If Using Slog rushes, also sharpen the leveled clip
source1_lvl_shp = Sharpen_it ? source1_lvl.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th) : source1_lvl

#Remove spots, Spotless, version 1.07
# ------------------------------------------------------------------------------------
source1_shp_spt = SpotLess(source1_shp,blksz=BlkSz,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
source1_lvl_shp_spt = Slog ? SpotLess(source1_lvl_shp,blksz=BlkSz,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT) : source1_shp_spt
#~ return(filtered_orig)
# Restore removed objects seen as dirt
# ------------------------------------------------------------------------------------

#~ DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
#~ RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
# Delta Restore -------------------------------------------------------------

# extra clipping (if needed)
#~ crop_left = 20
#~ crop_right = 1120
#src = src.Crop(crop_left, 0, -crop_right, 0)
#filtered = filtered.Crop(crop_left, 0, -crop_right, 0)

# diff with exaggerated chroma difference
diff1 = mt_makediff(source1_lvl_shp, source1_lvl_shp_spt, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

#Calculate masks with leveled clips
####----------------------------------------------------------------------------------------------------------------
# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
# suppression mask removes too dark areas from chroma mask
suppression_mask = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, "<", brightness=0.9, spotSize=2)
suppressed_chroma_mask = chroma_mask.SuppressWith(suppression_mask)

# edge mask - detects sharp edges
# don't detect edges near the frame edges
edges = scharr(source1_lvl_shp).ZPadding(60, 14, 14, 10)
edgeMask = edges.mt_binarize(52)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, delta=5, spotSize=6).ExpandMask(3, blur=2)

# dark mask: select both darker and lighter mask and keep dark mask if it is near lighter mask or overlaps chroma override mask
dark = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
light = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, ">", brightness=1.04, spotSize=5).ZPadding(60, 14, 14, 10)
dark_near_light = dark.MustBeNear(light, 28).ExpandMask(3, blur=4)
expanded = dark.MustBeNear(light, 28, expanded=true) # for debugging

# secondary dark mask - doesn't need to be near lighter area but has stricter limits
dark2 = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, "<", brightness=dark2Brt, limitBrightness=dark2BrtLimit, limitAbsBrightness=0.12, spotSize=5).ExpandMask(2, blur=2)

# edge test for dark areas
darkEdgeMask = edges.mt_binarize(54) # at least 54 needed for frame 222
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_near_light.AddTo(dark2), chroma="-128") # for debug

dark_final = DarkDeltaMask(dark_near_light, secondary=dark2, edgeMask=darkEdgeMask)

combined_mask = luma_mask.AddTo(suppressed_chroma_mask).AddTo(chroma_override)

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-128") # for debug

#~ Apply masks with unleveled clips
####----------------------------------------------------------------------------------------------------------------
delta_restore = DeltaRestore(source1_shp_spt, source1_shp, luma=luma_mask, chroma=suppressed_chroma_mask, chromaOverride=chroma_override, edges=edgeMask, dark=dark_final)
####----------------------------------------------------------------------------------------------------------------

#~ Display the work---------------------------------------------------------------------------------------------------------------------

edgeAreaMask = mt_logic(edgeAreaMask, darkEdgeAreaMask, "or")
#edgeAreaMask = darkEdgeAreaMask

EDIT: Corrected error in variables names: Delta Restore has to compare sharpened original with sharpened spotlessed. Was: comparing unsharpened original with sharpened spotlessed. (source1_lvl_shp instead of source1_lvl)

chmars

25th June 2021, 22:38

Part 2

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(source1)
yellow = BlankClip(source1, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(source1, color=$40f75b)
gray = BlankClip(source1, color=$d0d0d0)
darkgray = BlankClip(source1, color=$909090)
orange = BlankClip(source1, color=$a36b02)
purple = BlankClip(source1, color=$4e0091)
purple_light = BlankClip(source1, color=$bc6eff)
purple_dark = BlankClip(source1, color=$340061)
red = BlankClip(source1, color=$9e2424)
blue = BlankClip(source1, color=$0d3b7a) #24579e 243c9e

# maskviz for dark mask
maskviz = Overlay(black, purple, mask=expanded)
maskviz = Overlay(maskviz, purple_dark, mask=dark)
maskviz = Overlay(maskviz, purple_light, mask=light)
maskviz = Overlay(maskviz, black, mask=dark_near_light)

# only show dark mask?
showDarkMaskOnly = false
if (!showDarkMaskOnly) {
maskviz = Overlay(maskviz, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = Overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = Overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = Overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = Overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue
}

# mask overlay
comp = Overlay(delta_restore, maskviz, opacity=0.45)
comp = Overlay(comp, purple_light, mask=light, opacity=0.2)
comp = Overlay(comp, black, mask=dark, opacity=0.2)
comp = Overlay(comp, black, mask=dark_final, opacity=0.6)

maskviz = Overlay(maskviz, black, mask=dark_final, opacity=0.6)

stack1 = StackHorizontal(\
sub(source1, "source", txt_sz), \
sub(source1_shp_spt, "sharp+Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz)\
)

stack2 = StackHorizontal(\
sub(maskviz, "masks used", txt_sz),\
sub(edgeMask, "edge mask", txt_sz),\
sub(diff, "diff", txt_sz),\
sub(blankclip(source1), "", txt_sz)\
)

# scaling (if needed)
#~ stack = stack.BilinearResize(stack.width/2 + 1, stack.height/2)
stack1 = stack1.BilinearResize(stack1.width/2 , stack1.height/2)
stack2 = stack2.BilinearResize(stack2.width/2 , stack2.height/2)

stacks = StackVertical(stack1,stack2)

#~ return stacks

inter = Interleave(\
sub(source1, "source", txt_sz), \
sub(source1_shp_spt, "sharp+Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

#~ return inter

function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

#~ Delta Restore End
#~ /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#~/ DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
#~ /RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR

# Restore grain - cancels sharpen
#------------------------------------------------------------------------------------
restored = Rgr ? mRD_RestoreGrain(delta_restore, source1,val1, val2) : delta_restore #5,10 = good compromise grain/dirt

#~ return(Interleave(source1, restored_2, restored))
return(restored)
#~ return(interleave(restored, source1))

#~ FUNCTIONS
##########################################################################################################################
#~ Delta Restore Functions
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
function DeltaRestore(clip filtered, clip source, clip "luma", clip "chroma", clip "chromaOverride", clip "edges", clip "dark") {
Assert(luma.Defined || chroma.Defined || chromaOverride.Defined || dark.Defined, "DeltaRestore:must define at least one of luma, chroma, chromaOverride, dark")

mask = Default(luma, BlankClip(filtered))
if (chroma.Defined) {
mask = mask.AddTo(chroma)
}
if (chromaOverride.Defined) {
mask = mask.AddTo(chromaOverride)
}
if (edges.Defined) {
mask = mask.MustNotOverlap(edges)
}
if (dark.Defined) {
mask = mask.AddTo(dark)
}
return Overlay(filtered, source, mask=mask)
}

function DarkDeltaMask(clip mask, clip "secondary", clip "edgeMask") {
if (secondary.Defined) {
mask = mask.AddTo(secondary)
}
if (edgeMask.Defined) {
mask = mask.MustNotOverlap(edgeMask)
}
return mask
}

function AddTo(clip mask, clip other) {
return mt_logic(mask, other, "or")
}

function SuppressWith(clip mask, clip suppressionMask) {
return mt_lutxy(mask, suppressionMask, expr="x y -")
}

function MustNotOverlap(clip mask, clip other) {
other = mt_hysteresis(other, mask, chroma="-128")
return mt_lutxy(mask, other, expr="x y -")
}

function MustOverlap(clip mask, clip otherMask) {
return mt_logic(mask, mt_hysteresis(otherMask, mask), "and")
}

function MustBeNear(clip mask, clip otherMask, int maxDistance, bool "expanded") {
expanded = Default(expanded, false)

expand_amount = maxDistance / 2
mask_expanded = ExpandMask(mask, expand_amount)
other_expanded = ExpandMask(otherMask, maxDistance - expand_amount)
common = mt_logic(mask_expanded, other_expanded, "or")
if (expanded) {
return common
}
other_near = mt_hysteresis(otherMask, common)
return mt_logic(mask, other_near, "and")
}

function ZPadding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

chmars

25th June 2021, 22:38

Part 3 of 3

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", float "limitAbsBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)
limitAbsBrightness = Default(limitAbsBrightness, 0.0)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# limitAbsBrightness suppression
if (limitAbsBrightness > 0.0) {
mask = mt_lutxy(mask, source, expr="y range_max "+String(limitAbsBrightness)+" * < 0 x ?")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness or limitAbsBrightness suppression (part 2)
if (limitBrightness != brightness || limitAbsBrightness > 0.0) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
if ( RiskyBlur == True) {
mask = mask.FastBlur(blur)
}
if ( RiskyBlur == False) {
mask = mask.GBlur(rad=6, sd=blur, u=true)
}
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
#~ mask = mask.GBlur(rad=6, sd=blur, u=true)
}
return mask
}

# RestoreGrain Function
###############################################################################################################################
###############################################################################################################################

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") { #https://forum.doom9.org/showthread.php?p=1943371#post1943371

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 20.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

# Spotless Function
###############################################################################################################################
###############################################################################################################################
# Version 1.07
Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess107: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

#~ Prefetch(threads)

StainlessS

26th June 2021, 02:15

Test sequence [26secs, ~14MB]:- https://www.mediafire.com/file/v3tt00pidecdugz/AMOLAD.demuxed.m2v.7z/file
I have not re-tried, I might have used Spotless(RadT=1), rather than RadT=2.

To be fair, its quite hard to see the ball even without spotless.

chmars

26th June 2021, 12:39

Oh, sh1t! While cleaning the script to post it, I made an error in one variable name: Delta Restore has to compare sharpened original with sharpened spotlessed. Was: comparing unsharpened original with sharpened spotlessed. (source1_lvl_shp instead of source1_lvl). Must be: source1_lvl_shp
Corrected.

Noticed while trying to recall this torture ping-pong ball.
...Still not succeded btw....

chmars

26th June 2021, 13:55

:D

https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/eS8YU0gQIyMNJxZ/download (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/IOW1Sl5ypvcNnPz/download)
luma_mask = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, ">", brightness=1.01, spotSize=3).ExpandMask(3, blur=4)(brightness 1.01 instead of 1.1)

But:
I skipped edgeMask and dark_final. Would need more time to tweak:
delta_restore = DeltaRestore(source1_shp_spt, source1_shp, luma=luma_mask, chroma=suppressed_chroma_mask, chromaOverride=chroma_override)
instead of:
delta_restore = DeltaRestore(source1_shp_spt, source1_shp, luma=luma_mask, chroma=suppressed_chroma_mask, chromaOverride=chroma_override, edges=edgeMask, dark=dark_final)

Also, I didn't see spots/dirt as I know them from 8mm. So, I'didn't really understand what Spotless is intended to remove here. Some kind of noise maybe?
EDIT: New clip. Error before exporting. The "source" and "restored" parts of the stack were the same. see post #134
.

StainlessS

26th June 2021, 14:17

I had used Spotless on entire movie, spots/marks on film not necessarily on every frame or sequence.
It was only on eventual watching of the movie that I thought, "where the hell is the ball?".
[gettin' a bit sick of doing frame by frame hand adjustments].

EDIT: I spent about 2 years doing repeated little adjustments and fixes on one old movie, and
eventually took up coding again [after 14 year lapse] to do ExBlend(). After ExBlend, ended up before final encode doing 12 passes,
[to lossless each time] with each pass tweaking separate frames/sequences.
Spotless could have fixed many sections of that old movie, all on its own [after Exblend],
I dont want to spend anything like that amount of effort on a 90 mins movie, ever again,
life's just too short.

chmars

26th June 2021, 14:24

I see.
As much more expert than me, did you watch the Delta Restore result? Not bad, or?

StainlessS

26th June 2021, 14:46

As much more expert than me
Dont know bout that.

Not bad, or?
Yes, I've just now examined sample encode frame by frame, and find it hard to believe that Spotless was used on it.
Not bad indeed.

chmars

27th June 2021, 15:50

...
Yes, I've just now examined sample encode frame by frame, and find it hard to believe that Spotless was used on it.
...

Here we can see spotless has removed the ball and Delta Restore brought it back (2 and 3rd pic, near the woman's racket).

https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/ddubNMZSCa2cFst/download

Here (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/vW2LtUOoLKBhcFV/download) is the exact script used.

Reading you, I has a doubt. Tested exact same script with a dirty clip....no differences :o :confused: : error at the end by resizing. Displayed twice the same clip.
But it was a relief to see that once corrected, I didn't dream: it wotks!
I replaced the clip I posted.

kedautinh12

2nd July 2021, 05:35

function RemoveDust(clip input, int "mode") {
mode = Default(mode, 17)

clensed = Clense(input)
rep = Repair(clensed, input, mode=mode)

RemoveGrain(rep, mode=10)
}
[/CODE]

function RemoveDust had duplicates of SpotLess_Mod and SpotRemover from your

Arx1meD

2nd July 2021, 09:06

kedautinh12, I added RemoveDust in post #114 (https://forum.doom9.org/showthread.php?p=1945133#post1945133) so as not to look for this function in other posts. If an error occurs, then remove the duplicate function.

kedautinh12

16th July 2021, 10:03

@zorr, I tested with megui but met error: "the script's return was not a video clip". Sr for noob ask Can apply script with Megui??

LoadPlugin("E:\MeGUI-2924-64\tools\ffms\ffms2.dll")
source=FFVideoSource("D:\Video encode\Da-iCE - DREAMIN' ON [1440x1080i MPEG2 SSTV HD].ts", fpsnum=30000, fpsden=1001, threads=1, colorspace="YUV420P8")

BlkSz = 24
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Sharpen----------------------------------------------------------------------
source_shp=source.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#SpotRemoved_shp=SpotRemoved.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#----------------------------------------------------------------------------------

#Remove spots-----------------------------------------------------------------
spotless=SpotLess(source_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
#SpotRemoved = SpotRemover(sourceYV12,Spot=128, RadT=1, Pel=Pel, BlkSz=32, DeGrain=false, Chroma=true, PreFilter=true, mask=false, Tm = true,thsad=thsad, Glob = true)
#----------------------------------------------------------------------------------

# Delta Restore -------------------------------------------------------------

filtered = spotless

# extra clipping (if needed)
crop_left = 20
crop_right = 1120
#source = source.Crop(crop_left, 0, -crop_right, 0)
#filtered = filtered.Crop(crop_left, 0, -crop_right, 0)

# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(source, filtered, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(source, filtered, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
# suppression mask removes too dark areas from chroma mask
suppression_mask = LumaDeltaMask(source, filtered, "<", brightness=0.9, spotSize=2)
suppressed_chroma_mask = chroma_mask.SuppressWith(suppression_mask)

# edge mask - detects sharp edges
# don't detect edges near the frame edges
edges = scharr(source).ZPadding(60, 14, 14, 10)
edgeMask = edges.mt_binarize(52)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(source, filtered, delta=5, spotSize=6).ExpandMask(3, blur=2)

# dark mask: select both darker and lighter mask and keep dark mask if it is near lighter mask or overlaps chroma override mask
dark = LumaDeltaMask(source, filtered, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
light = LumaDeltaMask(source, filtered, ">", brightness=1.04, spotSize=5).ZPadding(60, 14, 14, 10)
dark_near_light = dark.MustBeNear(light, 28).ExpandMask(3, blur=4)
expanded = dark.MustBeNear(light, 28, expanded=true) # for debugging

# secondary dark mask - doesn't need to be near lighter area but has stricter limits
dark2 = LumaDeltaMask(source, filtered, "<", brightness=0.97, limitBrightness=0.69, limitAbsBrightness=0.12, spotSize=5).ExpandMask(2, blur=2)

# edge test for dark areas
darkEdgeMask = edges.mt_binarize(54) # at least 54 needed for frame 222
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_near_light.AddTo(dark2), chroma="-128") # for debug

dark_final = DarkDeltaMask(dark_near_light, secondary=dark2, edgeMask=darkEdgeMask)

delta_restore = DeltaRestore(filtered, source, luma=luma_mask, chroma=suppressed_chroma_mask, chromaOverride=chroma_override, edges=edgeMask, dark=dark_final)

StainlessS

16th July 2021, 13:22

kedautinh12,
What post is that supposed to be from ? [There is obviously some script missing at the end].

EDIT: Maybe you forgot to add/append posts following the first post of a multi-post script.

kedautinh12

16th July 2021, 13:45

I deleted scripts related mask only for check or debug view

Edit: i copy from post
https://forum.doom9.org/showthread.php?p=1945212#post1945212

StainlessS

16th July 2021, 16:24

No idea if this will work [I aint been following development of that script],
but at end maybe try

dark_final = DarkDeltaMask(dark_near_light, secondary=dark2, edgeMask=darkEdgeMask)

delta_restore = DeltaRestore(filtered, source, luma=luma_mask, chroma=suppressed_chroma_mask, chromaOverride=chroma_override, edges=edgeMask, dark=dark_final)

Return delta_restore # Added

Dont know if that is the result clip or not. [If not then have to wait for young master Zorr to appear].

EDIT: I presume that other script function code is imported somewhere else (or in Plugins as Avsi).

EDIT: If additional script function code not imported elsewhere, then can just use Zorr original script,
with that Return delta_restore line added at that same position, the extra mask and debug code will not be used.
And to view masks or debug stuff, just comment out the Return delta_restore line. [maybe Zorr accidently removed that Return line]

kedautinh12

16th July 2021, 16:47

Yes, i copy this functions to one script .avsi and put in autoload folder

function DeltaRestore(clip filtered, clip source, clip "luma", clip "chroma", clip "chromaOverride", clip "edges", clip "dark") {
Assert(luma.Defined || chroma.Defined || chromaOverride.Defined || dark.Defined, "DeltaRestore:must define at least one of luma, chroma, chromaOverride, dark")

mask = Default(luma, BlankClip(filtered))
if (chroma.Defined) {
mask = mask.AddTo(chroma)
}
if (chromaOverride.Defined) {
mask = mask.AddTo(chromaOverride)
}
if (edges.Defined) {
mask = mask.MustNotOverlap(edges)
}
if (dark.Defined) {
mask = mask.AddTo(dark)
}
return Overlay(filtered, source, mask=mask)
}

function DarkDeltaMask(clip mask, clip "secondary", clip "edgeMask") {
if (secondary.Defined) {
mask = mask.AddTo(secondary)
}
if (edgeMask.Defined) {
mask = mask.MustNotOverlap(edgeMask)
}
return mask
}

function AddTo(clip mask, clip other) {
return mt_logic(mask, other, "or")
}

function SuppressWith(clip mask, clip suppressionMask) {
return mt_lutxy(mask, suppressionMask, expr="x y -")
}

function MustNotOverlap(clip mask, clip other) {
other = mt_hysteresis(other, mask, chroma="-128")
return mt_lutxy(mask, other, expr="x y -")
}

function MustOverlap(clip mask, clip otherMask) {
return mt_logic(mask, mt_hysteresis(otherMask, mask), "and")
}

function MustBeNear(clip mask, clip otherMask, int maxDistance, bool "expanded") {
expanded = Default(expanded, false)

expand_amount = maxDistance / 2
mask_expanded = ExpandMask(mask, expand_amount)
other_expanded = ExpandMask(otherMask, maxDistance - expand_amount)
common = mt_logic(mask_expanded, other_expanded, "or")
if (expanded) {
return common
}
other_near = mt_hysteresis(otherMask, common)
return mt_logic(mask, other_near, "and")
}

function ZPadding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", float "limitAbsBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)
limitAbsBrightness = Default(limitAbsBrightness, 0.0)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# limitAbsBrightness suppression
if (limitAbsBrightness > 0.0) {
mask = mt_lutxy(mask, source, expr="y range_max "+String(limitAbsBrightness)+" * < 0 x ?")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness or limitAbsBrightness suppression (part 2)
if (limitBrightness != brightness || limitAbsBrightness > 0.0) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
mask = mask.GBlur(rad=6, sd=blur, u=true)
}
return mask
}

StainlessS

16th July 2021, 18:15

works OK now ? [with that return line]

zorr

16th July 2021, 20:25

Do not worry mortals, young master Zorr is here! :D

I deleted scripts related mask only for check or debug view

Edit: i copy from post
https://forum.doom9.org/showthread.php?p=1945212#post1945212

The post you quoted contains a "return stack" line. It returns the source, Spotless-filtered version, delta restored version, restore + mask, mask visualization, edge mask and diff stacked horizontally. If you just want the final result of Delta Restore then what his holiness StainlessS told works fine, namely "return delta_restore".

coolgit

8th September 2021, 00:17

Thanks for the delta restore. In addition to spotless it has been a blessing. Recently i had to extract good frames to replace bad spotless frames when there is motion close to the camera.
In some cases when hands/fingers disappear at medium distance from camera but everything else doesn't.

However there is one drawback. A spot in the hand wasn't removed. How can i use delta restore minus the spot. Obviously white spot = flicker.

My codes.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\MedianBlur2.dll")

source=FFVideoSource("E:\currser\BABYLON5\new\12 By Any Means Necessary\By Any Means Necessary.avi").convertTOYV12(interlaced=false).killaudio()

BlkSz = 16
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 2
Tm=true
Bblur = 0.0
ThSAD = 4000
RadT = 1

#Remove spots
SpotLess1=SpotLess(source,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)

# Delta Restore
src = source
filtered = SpotLess1

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# restore
delta_restore = Overlay(filtered, src)

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma",int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur", clip "dc" ) {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,4000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,true) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,16) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,true) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.0) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

output=SpotLess(source)
StackHorizontal(source,output,delta_restore) # stack side by side, source left, output right
#return output
#return delta_restore

I am simply using delta restore to restore missing bits and not grains etc.

My processes are:
1 deinterlace vob file using staxrip qtgmc - f1
2 f1 - staxrip crop, BlackmanResize(720, 576), MDegrain3, TemporalDegrain2 - f2 (no noise/grain and clean source)
3 f2 - spotless (works better after doing 2 above and quicker) - fspotless
4 crosscheck fspotless and f2 using any frames to replace bad fspotless frames.

zorr

8th September 2021, 22:21

Thanks for the delta restore. In addition to spotless it has been a blessing.

You're welcome.

However there is one drawback. A spot in the hand wasn't removed. How can i use delta restore minus the spot. Obviously white spot = flicker.

That's a tough situation, Delta Restore has restored a large piece of the hand and the spot is right on top of that area. That means there is no good frame to replace that spot with.

By the way are you sure that spot is something you need to remove? Even the original frame looks very clean to me except for that one dot so it might be some kind of reflection. Is that spot only in this one frame?

In order to better analyze what (if anything) can be done you should post the mask visualization output, like in this post (https://forum.doom9.org/showthread.php?p=1944708#post1944708).

coolgit

9th September 2021, 08:55

Made some changes in the script and fixed the spot in the hand. New image.

However delta restore failed in this image. Did not restore the finger and i am assuming it is because the source finger was too bright.

Can someone check if the script is correct. Not sure about the LoadPlugin. If i want to output a new file then i should use "return delta_restore", line 364 ish? Thanks.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\MedianBlur2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\masktools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\manyPlus.dll")

source=FFVideoSource("E:\currser\BABYLON5\new\12 By Any Means Necessary\By Any Means Necessary.avi").convertTOYV12(interlaced=false).killaudio()
#source= FFVideoSource("E:\currser\BABYLON5\new\11 Survivors\Survivors.avi").convertTOYV12(interlaced=false).killaudio()

txt_sz = 24

BlkSz = 12
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 2
Tm=false
Bblur = 0.0
ThSAD = 6000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Sharpen----------------------------------------------------------------------
#source_shp=source.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)

#Remove spots-----------------------------------------------------------------
#SpotLess=SpotLess(source_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
SpotLess=SpotLess(source,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)

# Delta Restore -------------------------------------------------------------

src = source
filtered = SpotLess

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# prepare masks --------------------------------------------------------------------
# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3)
luma_mask = ExpandMask(luma_mask, 3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(src, filtered, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
suppression_mask = LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)

# suppression mask removes too dark areas from chroma mask
suppressed_chroma_mask = mt_lutxy(chroma_mask, suppression_mask, expr="x y -")

# edge mask - detects sharp edges
edges = scharr(src)

# don't detect edges near the frame edges
edges = edges.ZPadding(60, 14, 14, 10)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(src, filtered, delta=5, spotSize=6) # orig. spotSize 6
chroma_override = ExpandMask(chroma_override, 3, blur=2)

# new approach for dark mask:
# select both darker and lighter mask and keep dark mask if it is near lighter mask or overlaps chroma override mask
dark = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
light = LumaDeltaMask(src, filtered, ">", brightness=1.04, spotSize=5)
light = light.ZPadding(60, 14, 14, 10)

dark_near_light = isNearMask(dark, light, 28)
expanded = IsNearMask(dark, light, 28, expanded=true) # for debugging
dark_final = ExpandMask(dark_near_light, 3, blur=4)

# secondary dark mask - doesn't need to be near lighter area but has stricter limits
dark2 = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.69, limitAbsBrightness=0.12, spotSize=5)
dark2 = ExpandMask(dark2, 2, blur=2)
dark_final = mt_logic(dark_final, dark2, "or")

# edge test for dark areas
darkEdgeMask = edges.mt_binarize(54) # at least 54 needed for frame 222
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_final, chroma="-127")
dark_final = mt_lutxy(dark_final, darkEdgeAreaMask, expr="x y -")

# combine luma, chroma and chroma override masks
combined_mask = mt_logic(luma_mask, suppressed_chroma_mask, "or")
combined_mask = mt_logic(combined_mask, chroma_override, "or")

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeMask = edges.mt_binarize(52)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-127")
combined_mask = mt_lutxy(combined_mask, edgeAreaMask, expr="x y -")

# add dark areas (no edge test for those)
combined_mask = mt_logic(combined_mask, dark_final, "or")

# restore ---------------------------------------------------------------------------
delta_restore = overlay(filtered, src, mask=combined_mask)

edgeAreaMask = mt_logic(edgeAreaMask, darkEdgeAreaMask, "or")
#edgeAreaMask = darkEdgeAreaMask

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(src)
yellow = BlankClip(src, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(src, color=$40f75b)
gray = BlankClip(src, color=$d0d0d0)
darkgray = BlankClip(src, color=$909090)
orange = BlankClip(src, color=$a36b02)
purple = BlankClip(src, color=$4e0091)
purple_light = BlankClip(src, color=$bc6eff)
purple_dark = BlankClip(src, color=$340061)
red = BlankClip(src, color=$9e2424)
blue = BlankClip(src, color=$0d3b7a) #24579e 243c9e

# maskviz for dark mask
maskviz = overlay(black, purple, mask=expanded)
maskviz = overlay(maskviz, purple_dark, mask=dark)
maskviz = overlay(maskviz, purple_light, mask=light)
maskviz = overlay(maskviz, black, mask=dark_near_light)

# only show dark mask?
showDarkMaskOnly = false
if (!showDarkMaskOnly) {
maskviz = overlay(maskviz, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue
}

# mask overlay
comp = overlay(delta_restore, maskviz, opacity=0.45)
comp = overlay(comp, purple_light, mask=light, opacity=0.2)
comp = overlay(comp, black, mask=dark, opacity=0.2)
comp = overlay(comp, black, mask=dark_final, opacity=0.6)

maskviz = overlay(maskviz, black, mask=dark_final, opacity=0.6)

#return Interleave(StackHorizontal(src, comp), StackHorizontal(delta_restore, maskviz))
#return StackHorizontal(src, filtered, delta_restore, edgeMask, maskviz)

#------------------------------------------------------------------------------------------------------------------
function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

function OverlapsMask(clip mask, clip otherMask) {
return mt_logic(mask, mt_hysteresis(otherMask, mask), "and")
}

function IsNearMask(clip mask, clip otherMask, int maxDistance, bool "expanded") {
expanded = Default(expanded, false)

expand_amount = maxDistance / 2
mask_expanded = ExpandMask(mask, expand_amount)
other_expanded = ExpandMask(otherMask, maxDistance - expand_amount)
common = mt_logic(mask_expanded, other_expanded, "or")
if (expanded) {
return common
}
other_near = mt_hysteresis(otherMask, common)
return mt_logic(mask, other_near, "and")
}

function ZPadding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", float "limitAbsBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)
limitAbsBrightness = Default(limitAbsBrightness, 0.0)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# limitAbsBrightness suppression
if (limitAbsBrightness > 0.0) {
mask = mt_lutxy(mask, source, expr="y range_max "+String(limitAbsBrightness)+" * < 0 x ?")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness or limitAbsBrightness suppression (part 2)
if (limitBrightness != brightness || limitAbsBrightness > 0.0) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff ----------------------------------------------------------------------
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
# mask = mask.ConvertBits(16)
mask = mask.GBlur(rad=6, sd=blur, u=true)
# mask = mask.ConvertBits(10)
}

return mask
}

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,6000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,12) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,false) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.0) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

#output=SpotLess(src)
#StackHorizontal(src,output,delta_restore) # stack side by side, source left, output right
#return output
#return delta_restore

#Debug------------------------------

#StackHorizontal(\
#sub(src,"Source",size = txt_sz),\
#sub(filtered,"Spotless",size = txt_sz))

#StackHorizontal(\
#sub(src,"Source",size = txt_sz),\
#sub(filtered,"Spotless",size = txt_sz),\
#sub(delta_restore,"Delta Restore",size = txt_sz))

StackHorizontal(\
sub(src, "Source", txt_sz),\
sub(filtered, "Spotless", txt_sz),\
sub(delta_restore, "Delta Restore", txt_sz),\
sub(comp, "Restore + Mask", txt_sz),\
sub(maskviz, "Masks Used", txt_sz),\
sub(edgeMask, "Edge Mask", txt_sz),\
sub(diff, "Diff", txt_sz))

#inter = Interleave(\
#sub(src, "source", txt_sz), \
#sub(filtered, "Spotless", txt_sz), \
#sub(delta_restore, "Delta Restore", txt_sz), \
#sub(comp, "restore + mask", txt_sz), \
#sub(maskviz, "masks used", txt_sz),\
#sub(diff, "diff", txt_sz)\
#)

#return inter

Prefetch()

StainlessS

9th September 2021, 14:21

Coolgit,
Zorr,
I note that in a number of posts in spotless related thread
eg here:- https://forum.doom9.org/showthread.php?p=1944487#post1944487

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

think maybe above chroma="-127" should be chroma="-128",
Appears quite a few times in the linked thread.

From Masktools 2 original docs.

EDIT: I recently found a script posted some time ago that now throws an error in recent Pinterf MaskTools, I had wrongly used
chroma="128" [instead of chroma="-128"]
and it now throws a wobbly, says should be setting chroma to -ve ie -128 [although original script did seem to work OK in older masktools].

Yes you're right. I'll fix it when I release the "official" Delta Restore version. It doesn't have any effect on the final result so it's nothing to worry about. :)

so maybe change -127 to -128.

coolgit

9th September 2021, 19:03

Thanks for that. There are other lines with chroma="-127" so do i leave it as is?

I am going to do the nest episode and hopefully this delta restore will reduce the amount of time spent of recovering bad spotless frames.

StainlessS

9th September 2021, 19:35

Nope dont leave as is, all -127 are wrong.

coolgit

10th September 2021, 12:28

A minor flaw. Delta restore output is brighter than source. The face skin tone became lighter.
Using this took 1 hr 35 minutes for 40 minutes pal sd video at 11 fps. Seems too slow or I am doing something wrong.

Any solution?

zorr

10th September 2021, 21:00

A minor flaw. Delta restore output is brighter than source. The face skin tone became lighter.
Any solution?

Can you provide a short clip and your script (unless it's the same you gave earlier) to demonstrate the effect?

Using this took 1 hr 35 minutes for 40 minutes pal sd video at 11 fps. Seems too slow or I am doing something wrong.

Sounds about right. Delta Restore is not at the moment optimized for performance.

coolgit

11th September 2021, 14:01

Here the 20 secs clip.

https://mega.nz/file/GrJQjZRD#xzf2L6_ZtOquWsTSW13LLFWpg-qd7mqxX5O0hsbi7wE

# VARIABLES--------------------------------------------------------------------

txt_sz = 24 # Test size if infos displayed

# Spotless variables, to remove dirt and spots.-------------------------------
RadT = 1 # Temporal radius. (MCompensate arg)
ThSAD = 8000 # SAD threshold at radius 1 (Default=10000 Nearly OFF).
Pel = 2 # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
BlkSz = 16 # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Tm=true # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Bblur = 0.3 # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.

# Sharpen variables-----------------------------------------------------------
Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

# Sharpen----------------------------------------------------------------------
#source_shp=source.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th) # If using sharpen then use SpotLess_shp in Remove spots and Delta Restore

# Remove spots-----------------------------------------------------------------
#SpotLess_shp=SpotLess(source_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
SpotLess=SpotLess(source,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)

# Delta Restore---------------------------------------------------------------

src = source
filtered = SpotLess
#filtered = SpotLess_shp

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# prepare masks---------------------------------------------------------------
# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3)
luma_mask = ExpandMask(luma_mask, 3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(src, filtered, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
suppression_mask = LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)

# suppression mask removes too dark areas from chroma mask
suppressed_chroma_mask = mt_lutxy(chroma_mask, suppression_mask, expr="x y -")

# edge mask - detects sharp edges
edges = scharr(src)

# don't detect edges near the frame edges
edges = edges.ZPadding(60, 14, 14, 10)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(src, filtered, delta=5, spotSize=6) # orig. spotSize 6
chroma_override = ExpandMask(chroma_override, 3, blur=2)

# new approach for dark mask:
# select both darker and lighter mask and keep dark mask if it is near lighter mask or overlaps chroma override mask
dark = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
light = LumaDeltaMask(src, filtered, ">", brightness=1.04, spotSize=5).ZPadding(60, 14, 14, 10)
dark_near_light = isNearMask(dark, light, 28)
expanded = IsNearMask(dark, light, 28, expanded=true) # for debugging
dark_final = ExpandMask(dark_near_light, 3, blur=4)

# secondary dark mask - doesn't need to be near lighter area but has stricter limits
dark2 = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.69, limitAbsBrightness=0.12, spotSize=5)
dark2 = ExpandMask(dark2, 2, blur=2)
dark_final = mt_logic(dark_final, dark2, "or")

# edge test for dark areas
darkEdgeMask = edges.mt_binarize(54) # at least 54 needed for frame 222
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_final, chroma="-128")
dark_final = mt_lutxy(dark_final, darkEdgeAreaMask, expr="x y -")

# combine luma, chroma and chroma override masks
combined_mask = mt_logic(luma_mask, suppressed_chroma_mask, "or")
combined_mask = mt_logic(combined_mask, chroma_override, "or")

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeMask = edges.mt_binarize(52)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-128")
combined_mask = mt_lutxy(combined_mask, edgeAreaMask, expr="x y -")

# add dark areas (no edge test for those)
combined_mask = mt_logic(combined_mask, dark_final, "or")

# restore ---------------------------------------------------------------------------
delta_restore = overlay(filtered, src, mask=combined_mask)

edgeAreaMask = mt_logic(edgeAreaMask, darkEdgeAreaMask, "or")
#edgeAreaMask = darkEdgeAreaMask

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(src)
yellow = BlankClip(src, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(src, color=$40f75b)
gray = BlankClip(src, color=$d0d0d0)
darkgray = BlankClip(src, color=$909090)
orange = BlankClip(src, color=$a36b02)
purple = BlankClip(src, color=$4e0091)
purple_light = BlankClip(src, color=$bc6eff)
purple_dark = BlankClip(src, color=$340061)
red = BlankClip(src, color=$9e2424)
blue = BlankClip(src, color=$0d3b7a) #24579e 243c9e

# maskviz for dark mask
maskviz = overlay(black, purple, mask=expanded)
maskviz = overlay(maskviz, purple_dark, mask=dark)
maskviz = overlay(maskviz, purple_light, mask=light)
maskviz = overlay(maskviz, black, mask=dark_near_light)

# only show dark mask?
showDarkMaskOnly = false
if (!showDarkMaskOnly) {
maskviz = overlay(maskviz, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue
}

# mask overlay
comp = overlay(delta_restore, maskviz, opacity=0.45)
comp = overlay(comp, purple_light, mask=light, opacity=0.2)
comp = overlay(comp, black, mask=dark, opacity=0.2)
comp = overlay(comp, black, mask=dark_final, opacity=0.6)

maskviz = overlay(maskviz, black, mask=dark_final, opacity=0.6)

# Restore grain - cancels sharpen----------------------------------------------------------------------------------
#restored = mRD_RestoreGrain(src, delta_restore, 5, 10) #5,10 = good compromise grain/dirt

#return(restored)
#return(interleave(src, delta_restore, restored))

# FUNCTIONS
#------------------------------------------------------------------------------------------------------------------
function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

function OverlapsMask(clip mask, clip otherMask) {
return mt_logic(mask, mt_hysteresis(otherMask, mask), "and")
}

function IsNearMask(clip mask, clip otherMask, int maxDistance, bool "expanded") {
expanded = Default(expanded, false)

expand_amount = maxDistance / 2
mask_expanded = ExpandMask(mask, expand_amount)
other_expanded = ExpandMask(otherMask, maxDistance - expand_amount)
common = mt_logic(mask_expanded, other_expanded, "or")
if (expanded) {
return common
}
other_near = mt_hysteresis(otherMask, common)
return mt_logic(mask, other_near, "and")
}

function ZPadding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-128")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-128")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", float "limitAbsBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)
limitAbsBrightness = Default(limitAbsBrightness, 0.0)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# limitAbsBrightness suppression
if (limitAbsBrightness > 0.0) {
mask = mt_lutxy(mask, source, expr="y range_max "+String(limitAbsBrightness)+" * < 0 x ?")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness or limitAbsBrightness suppression (part 2)
if (limitBrightness != brightness || limitAbsBrightness > 0.0) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-128")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff ----------------------------------------------------------------------
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-128")
}
if (blur > 0) {
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
#mask = mask.ConvertBits(16)
mask = mask.GBlur(rad=3, sd=blur, u=true)
#mask = mask.ConvertBits(10)
}

return mask
}

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,8000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,16) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,true) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.3) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

output=SpotLess(src)
StackHorizontal(src,output,delta_restore) # stack side by side, source left, output middle, delta restore right
#return output
#return delta_restore

#Debug------------------------------

S1 = StackHorizontal(\
sub(src, "Source", txt_sz),\
sub(filtered, "Spotless", txt_sz)\
)
S2 = StackHorizontal(\
sub(delta_restore, "Delta Restore", txt_sz),\
sub(diff, "Diff", txt_sz)\
)
output = StackVertical(S1,S2)

return output

S1 = StackHorizontal(\
sub(src, "Source", txt_sz),\
sub(filtered, "Spotless", txt_sz),\
sub(delta_restore, "Delta Restore", txt_sz),\
sub(comp, "Restore + Mask", txt_sz)\
)
S2 = StackHorizontal(\
sub(maskviz, "Masks Used", txt_sz),\
sub(edgeMask, "Edge Mask", txt_sz),\
sub(diff, "Diff", txt_sz),\
sub(blankclip(src), "", txt_sz)\
)
output = StackVertical(S1,S2)

#return output

Prefetch(8)

poisondeathray

11th September 2021, 15:07

A minor flaw. Delta restore output is brighter than source. The face skin tone became lighter.

No change.

How are you taking screenshots or previewing script?

Some players will use Rec709 if the dimensions are > than some height, Rec601 if less than some height ; so on the stackvertical/stackhorizontal comparison which is "HD" now, the color would change

You can explicitly control the matrix used for the RGB preview, but adding ConvertToRGB24(matrix="rec601") (or don't forget to remove when encoding). SD typically uses rec601, HD rec709

StainlessS

11th September 2021, 15:50

so on the stackvertical/stackhorizontal comparison which is "HD" now, the color would change
Astute observation, thanks.

coolgit

11th September 2021, 20:48

No change.

How are you taking screenshots or previewing script?

Some players will use Rec709 if the dimensions are > than some height, Rec601 if less than some height ; so on the stackvertical/stackhorizontal comparison which is "HD" now, the color would change

You can explicitly control the matrix used for the RGB preview, but adding ConvertToRGB24(matrix="rec601") (or don't forget to remove when encoding). SD typically uses rec601, HD rec709

Preview using vdub2. dimension 720x576 and at no point did i change it so should still be rec601?

poisondeathray

11th September 2021, 21:07

Preview using vdub2. dimension 720x576 and at no point did i change it so should still be rec601?

The script in post #152 has the stack enabled. Clarify what version of the script you used in vdub2

Then something else is going on, maybe some plugin or version difference

I see no difference in original vs. output using the script you posted.

Histogram("levels") show no significant difference either. (you can see spots removed in "b")

a
https://postimg.cc/JDMDDh6Z

b
https://postimg.cc/dDQZJLTK

zorr

11th September 2021, 22:03

coolgit

12th September 2021, 11:32

The script in post #152 has the stack enabled. Clarify what version of the script you used in vdub2

Then something else is going on, maybe some plugin or version difference

I see no difference in original vs. output using the script you posted.

Histogram("levels") show no significant difference either. (you can see spots removed in "b")

a
https://postimg.cc/JDMDDh6Z

b
https://postimg.cc/dDQZJLTK

avisynth+ 3.7.0 r3382 3.7 i386

Recently installed so all files etc version are the latest.

What did you mean by "stack enabled"?

The issue I had no longer exist. I can't seem to replicate it either. Must be something to do with the variables I kept messing around with to find the optimum performance.

coolgit

12th September 2021, 11:38

I agree with poisondeathray, cannot see any difference. You can try this to verify it yourself:

return Interleave(src.Histogram("levels"), filtered.Histogram("levels"))

instead of the current return statement. By the way the script didn't contain source filter, did you use

source=FFVideoSource("clip.avi").convertTOYV12(interlaced=false).killaudio()

or something else?

Yes. I removed some of the top part because of the 16,000 chars limit.

I will be posting my analysis of spotless/delta restore later. Some flaws to work out.

coolgit

12th September 2021, 15:40

A few flaws here and there. Keep both eyes out for fast motion toward the camera, equate missing hand. Away from the camera is ok.

Spotless removes the following defects but delta restore restored them. Commonly happens when whitish to light yellowish and black defects surrounded by light colour background. Blackish defect surrounded by black background. These defects have none to little motion between frames and are background defects. Why would delta restore restore them?

example pics.

https://mega.nz/file/2yJ2ULxD#6bbH8jxb3TjhMLYMqD3EppJm7KPHrkG--fJpXrha7IE
https://mega.nz/file/GzRSUDya#mwtVg_u0UIGc7dZOgQSpkhME_qXbet5KsikmICbxpCs
https://mega.nz/file/zzAAjLib#HkyjjculNBgVL73Bo22W3cPbLBexU7R8zfKwwMzY6xY

poisondeathray

12th September 2021, 15:56

What did you mean by "stack enabled"?

The issue I had no longer exist. I can't seem to replicate it either. Must be something to do with the variables I kept messing around with to find the optimum performance.

"stack enabled" - I mean the stackhorizontal/stackvertical view is the current output in the script you provided . i.e The view comparing 4 versions at the same time in a 2x2 grid

zorr

12th September 2021, 20:49

Spotless removes the following defects but delta restore restored them. Commonly happens when whitish to light yellowish and black defects surrounded by light colour background. Blackish defect surrounded by black background. These defects have none to little motion between frames and are background defects. Why would delta restore restore them?

I need the original video or at least shots of the mask visualization to analyze those. Most likely the reason these defects are restored is that the default Delta Restore parameter values which you're using were tuned for other clip and need some changes. And even then the process is not perfect.

johnmeyer

28th September 2021, 23:21

I finally got around to trying Zorr's "Delta Restore (Dark Mask).avs" as shown in Post #101 (https://forum.doom9.org/showpost.php?p=1944703&postcount=101) in this thread.

I initially tried it on some American football games from 1941 that I'm restoring, but then tried it on the "torture" test clip I posted earlier in this thread.

Bottom line: it did not work well at all. The thrown balls disappeared just like they do with "RemoveDirtMC". I saw no evidence of any masking in the two far-right windows in the debug display.

The ONLY change I made to the script from what was posted is that I removed the UnsharpMask call because it kept complaining that there is no such thing as "sspasses". I looked all over for that, and then decided that I really didn't need sharpening.

I've copied the script that I used into the next post.

This is too bad because I was really looking forward to not having to turn dirt removal on/off around each thrown ball in a football game.

johnmeyer

28th September 2021, 23:24

Here is the copy/pasted version of the script I'm using. It should be identical to Post #101, except for the removal of the UnsharpMask_avsi call.

loadplugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\MVTools2 (latest)\mvtools2.dll")

source1= AVISource("e:\fs.avi")
source1= KillAudio(source1).ConvertToYV12

source1_8bit = ConvertBits(source1, bits=8) #I dont'know much about formats, maybe this is not ideal...
source1YV12 = ConvertToYV12 (source1_8bit) #I dont'know much about formats, maybe this is not ideal...

txt_sz = 36

BlkSz = 24
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

Usharp_strength= 80
Usharp_radius= 5
Usharp_th = 1 #too high = too many dark pixel packs (ex trees)

#Sharpen----------------------------------------------------------------------
#source1_shp=source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
source1_shp=source1
#SpotRemoved_shp=SpotRemoved.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th)
#----------------------------------------------------------------------------------

#Remove spots-----------------------------------------------------------------
spotless_107=SpotLess107(source1_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)
#SpotRemoved = SpotRemover(source1YV12,Spot=128, RadT=1, Pel=Pel, BlkSz=32, DeGrain=false, Chroma=true, PreFilter=true, mask=false, Tm = true,thsad=thsad, Glob = true)
#----------------------------------------------------------------------------------

#SpotRemoved = ConvertBits(SpotRemoved, bits=10) #I dont'know much about formats, maybe this is not ideal...
#SpotRemoved = ConvertToYUV422(SpotRemoved) #I dont'know much about formats, maybe this is not ideal...

# Restore grain -------------------------------------------------------------
#spotless_107_shp_Rg = mRD_RestoreGrain(spotless_107_shp, source1)
#SpotRemoved_shp_Rg = mRD_RestoreGrain(SpotRemoved_shp, source1)
#----------------------------------------------------------------------------------

# Delta Restore -------------------------------------------------------------

src = source1
filtered = spotless_107

# extra clipping (if needed)
#crop_left = 800
#crop_right = 280
#src = src.Crop(crop_left, 0, -crop_right, 0)
#filtered = filtered.Crop(crop_left, 0, -crop_right, 0)

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(src, filtered, ">", brightness=1.1, spotSize=3)
luma_mask = ExpandMask(luma_mask, 3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(src, filtered, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
suppression_mask = LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)

# suppression mask removes too dark areas from chroma mask
suppressed_chroma_mask = mt_lutxy(chroma_mask, suppression_mask, expr="x y -")

# edge mask - detects sharp edges
edges = scharr(src)

# don't detect edges near the frame edges
edges = edges.ZPadding(60, 14, 14, 10)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(src, filtered, delta=5, spotSize=6) # orig. spotSize 6
chroma_override = ExpandMask(chroma_override, 3, blur=2)

# new approach for dark mask:
# select both darker and lighter mask and keep dark mask if it is near lighter mask or overlaps chroma override mask
dark = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
light = LumaDeltaMask(src, filtered, ">", brightness=1.04, spotSize=5)
light = light.ZPadding(60, 14, 14, 10)

dark_near_light = isNearMask(dark, light, 28)
expanded = IsNearMask(dark, light, 28, expanded=true) # for debugging
dark_final = ExpandMask(dark_near_light, 3, blur=4)

# secondary dark mask - doesn't need to be near lighter area but has stricter limits
dark2 = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.69, limitAbsBrightness=0.12, spotSize=5)
dark2 = ExpandMask(dark2, 2, blur=2)
dark_final = mt_logic(dark_final, dark2, "or")

# edge test for dark areas
darkEdgeMask = edges.mt_binarize(54) # at least 54 needed for frame 222
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_final, chroma="-127")
dark_final = mt_lutxy(dark_final, darkEdgeAreaMask, expr="x y -")

# combine luma, chroma and chroma override masks
combined_mask = mt_logic(luma_mask, suppressed_chroma_mask, "or")
combined_mask = mt_logic(combined_mask, chroma_override, "or")

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeMask = edges.mt_binarize(52)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-127")
combined_mask = mt_lutxy(combined_mask, edgeAreaMask, expr="x y -")

# add dark areas (no edge test for those)
combined_mask = mt_logic(combined_mask, dark_final, "or")

# restore ---------------------------------------------------------------------------
delta_restore = Overlay(filtered, src, mask=combined_mask)

edgeAreaMask = mt_logic(edgeAreaMask, darkEdgeAreaMask, "or")
#edgeAreaMask = darkEdgeAreaMask

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(src)
yellow = BlankClip(src, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(src, color=$40f75b)
gray = BlankClip(src, color=$d0d0d0)
darkgray = BlankClip(src, color=$909090)
orange = BlankClip(src, color=$a36b02)
purple = BlankClip(src, color=$4e0091)
purple_light = BlankClip(src, color=$bc6eff)
purple_dark = BlankClip(src, color=$340061)
red = BlankClip(src, color=$9e2424)
blue = BlankClip(src, color=$0d3b7a) #24579e 243c9e

# maskviz for dark mask
maskviz = Overlay(black, purple, mask=expanded)
maskviz = Overlay(maskviz, purple_dark, mask=dark)
maskviz = Overlay(maskviz, purple_light, mask=light)
maskviz = Overlay(maskviz, black, mask=dark_near_light)

# only show dark mask?
showDarkMaskOnly = false
if (!showDarkMaskOnly) {
maskviz = Overlay(maskviz, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = Overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = Overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = Overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = Overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue
}

# mask overlay
comp = Overlay(delta_restore, maskviz, opacity=0.45)
comp = Overlay(comp, purple_light, mask=light, opacity=0.2)
comp = Overlay(comp, black, mask=dark, opacity=0.2)
comp = Overlay(comp, black, mask=dark_final, opacity=0.6)

maskviz = Overlay(maskviz, black, mask=dark_final, opacity=0.6)

#return Interleave(StackHorizontal(src, comp), StackHorizontal(delta_restore, maskviz))
#return StackHorizontal(src, filtered, delta_restore, edgeMask, maskviz)

stack = StackHorizontal(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(edgeMask, "edge mask", txt_sz),\
sub(diff, "diff", txt_sz)\
)

# scaling (if needed)
stack = stack.BilinearResize(stack.width/2 + 2, stack.height/2)

return stack

inter = Interleave(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

return inter

function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

video = Interleave(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(overlay,"delta restore",size = txt_sz,align=2),\
subtitle(restore_nosuppression,"no suppression",size = txt_sz,align=2),\
subtitle(spotless_107_shp,"spotless_107_shp",size = txt_sz,align=2)\
)

masks = Interleave(\
subtitle(BlankClip(source1),"",size = txt_sz,align=2),\
subtitle(combined_mask,"combined_mask",size = txt_sz,align=2),\
subtitle(BlankClip(spotless_107_shp),"",size = txt_sz,align=2)\
)

return StackHorizontal(video, masks)

group = StackHorizontal(\
subtitle(source1,"original",size = txt_sz,align=2),\
subtitle(spotless_107_shp,"spotless_107_shp",size = txt_sz,align=2),\
subtitle(overlay,"delta restore",size = txt_sz,align=2),\
subtitle(combined_mask,"combined mask",size = txt_sz,align=2),\
subtitle(luma_mask,"luma delta mask",size = txt_sz,align=2),\
subtitle(suppression_mask,"suppression mask",size = txt_sz,align=2),\
subtitle(suppressed_chroma_mask,"suppressed chroma mask",size = txt_sz,align=2),\
subtitle(chroma_mask,"chroma delta mask",size = txt_sz,align=2),\
subtitle(diff,"diff",size = txt_sz,align=2))
#subtitle(chroma_mask,"chroma mask",size = txt_sz,align=2),\
#subtitle(whitemask,"whitemask",size = txt_sz,align=2),\
#subtitle(whitemask,"filled whitemask",size = txt_sz,align=2),\
#subtitle(uv,"UV",size = txt_sz,align=2))
#subtitle(whitemask_orig,"orig. whitemask",size = txt_sz,align=2),\
#subtitle(SpotRemoved,"SpotRemover",size = txt_sz,align=2))
#subtitle(spotless_107_shp_Rg,"spotless_107_shp_Rg",size = txt_sz,align=2),\

return(group)#.AddBorders(1000, 0, 500, 0, color=$000000)

#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------
#~ ------------------------------------------------------------------------------------------------------------------

function OverlapsMask(clip mask, clip otherMask) {
return mt_logic(mask, mt_hysteresis(otherMask, mask), "and")
}

function IsNearMask(clip mask, clip otherMask, int maxDistance, bool "expanded") {
expanded = Default(expanded, false)

expand_amount = maxDistance / 2
mask_expanded = ExpandMask(mask, expand_amount)
other_expanded = ExpandMask(otherMask, maxDistance - expand_amount)
common = mt_logic(mask_expanded, other_expanded, "or")
if (expanded) {
return common
}
other_near = mt_hysteresis(otherMask, common)
return mt_logic(mask, other_near, "and")
}

function ZPadding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", float "limitAbsBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)
limitAbsBrightness = Default(limitAbsBrightness, 0.0)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# limitAbsBrightness suppression
if (limitAbsBrightness > 0.0) {
mask = mt_lutxy(mask, source, expr="y range_max "+String(limitAbsBrightness)+" * < 0 x ?")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness or limitAbsBrightness suppression (part 2)
if (limitBrightness != brightness || limitAbsBrightness > 0.0) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff ----------------------------------------------------------------------
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
# mask = mask.ConvertBits(16)
mask = mask.GBlur(rad=6, sd=blur, u=true)
# mask = mask.ConvertBits(10)
}

return mask
}

Function mRD_RestoreGrain (clip rd, clip o, float "str1", float "str2") {

sisavs26 = !(VersionNumber() < 2.60)

str1 = string(default(str1, 10.0))
str2 = string(default(str2, 5.0))

expr = sisavs26 ? " x range_half - abs "+str1+" scalef min "+str2+" scalef x range_half - abs - min 0 max x range_half - x range_half - abs 1 max / * range_half + " : " x 128 - abs "+str1+" min "+str2+" x 128 - abs - min 0 max x 128 - x 128 - abs 1 max / * 128 + "

Return sisavs26 ? mt_makediff(o, rd).mt_lut(expr, use_expr=2).mt_adddiff(rd, chroma="copy second") : mt_makediff(o, rd).mt_lut(expr).mt_adddiff(rd, chroma="copy second")
}

zorr

2nd October 2021, 00:19

I finally got around to trying Zorr's "Delta Restore (Dark Mask).avs"
...
tried it on the "torture" test clip I posted earlier in this thread.
Bottom line: it did not work well at all. The thrown balls disappeared just like they do with "RemoveDirtMC". I saw no evidence of any masking in the two far-right windows in the debug display.

There might be something weird going on as you should see some masking even with those "default" settings which were tuned for other video.

But you're right that the script without any changes doesn't do a very good job with the torture clip's ball. I never meant the script to work as-is for every video, it was just an example use case. You really need to tune the values for each case.

When you start you should only use the basic features starting with luma mask, adjust that and if it's not enough you can try the chroma mask and/or the dark mask. The dark mask is quite tricky to get right so I advise to use the luma and chroma masks if possible.

I did just that for the torture clip. Some minor adjustments were needed for the luma mask and already most of the missing balls were restored. The remaining problem was when ball moves across the sky and it's darker than the background. That's where you usually need the dark mask but in this case there's also big difference in chroma (easy to see in the diff debug view) so I enabled the chroma mask and it did a pretty good job. There's still some frames where ball is missing (I didn't aim for perfection) which might be salvageable with more tuning and/or using the more advanced mask features.

The tuning of parameters may not be very intuitive and I had an idea how to "learn" the correct parameters using Zopti and user provided examples, but I'm not working on that currently so the manual tinkering is unfortunately needed for now.

I'll post the script in the next message.

zorr

2nd October 2021, 00:22

Here's the modified script:

source1= FFVideoSource("Dirt Removal Test Clip - Thrown Ball.mp4")
source1= KillAudio(source1)

source1_8bit = ConvertBits(source1, bits=8) #I dont'know much about formats, maybe this is not ideal...
source1YV12 = ConvertToYV12 (source1_8bit) #I dont'know much about formats, maybe this is not ideal...

txt_sz = 36

BlkSz = 24
OLap = (BlkSz/2) #important not too low (eg 4 for blksz 12, because creates stairs)
Pel = 1
Tm=false
Bblur = 0.6
ThSAD = 10000
RadT = 1

#Remove spots-----------------------------------------------------------------
spotless_107=SpotLess107(source1,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT)

# Delta Restore -------------------------------------------------------------

src = source1
filtered = spotless_107

# extra clipping (if needed)
crop_left = 20
crop_right = 1120
#src = src.Crop(crop_left, 0, -crop_right, 0)
#filtered = filtered.Crop(crop_left, 0, -crop_right, 0)

# diff with exaggerated chroma difference
diff1 = mt_makediff(src, filtered, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(src, filtered, ">", brightness=1.13, spotSize=2).ExpandMask(3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(src, filtered, delta=4, spotSize=3).ExpandMask(0, blur=2)

# suppression mask - selects where source brightness < filtered brightness
# suppression mask removes too dark areas from chroma mask
suppression_mask = LumaDeltaMask(src, filtered, "<", brightness=0.9, spotSize=2)
suppressed_chroma_mask = chroma_mask.SuppressWith(suppression_mask)

# edge mask - detects sharp edges
# don't detect edges near the frame edges
edges = scharr(src).ZPadding(60, 14, 14, 10)
edgeMask = edges.mt_binarize(52)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(src, filtered, delta=5, spotSize=6).ExpandMask(3, blur=2)

# dark mask: select both darker and lighter mask and keep dark mask if it is near lighter mask or overlaps chroma override mask
dark = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
light = LumaDeltaMask(src, filtered, ">", brightness=1.04, spotSize=5).ZPadding(60, 14, 14, 10)
dark_near_light = dark.MustBeNear(light, 28)#.ExpandMask(3, blur=4)
expanded = dark.MustBeNear(light, 28, expanded=true) # for debugging

# secondary dark mask - doesn't need to be near lighter area but has stricter limits
dark2 = LumaDeltaMask(src, filtered, "<", brightness=0.97, limitBrightness=0.69, limitAbsBrightness=0.12, spotSize=5).ExpandMask(2, blur=2)

# edge test for dark areas
darkEdgeMask = edges.mt_binarize(54) # at least 54 needed for frame 222
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_near_light.AddTo(dark2), chroma="-128") # for debug

dark_final = DarkDeltaMask(dark_near_light, secondary=dark2, edgeMask=darkEdgeMask)

combined_mask = luma_mask.AddTo(suppressed_chroma_mask).AddTo(chroma_override)

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-128") # for debug

#delta_restore = DeltaRestore(filtered, src, luma=luma_mask, chroma=suppressed_chroma_mask, chromaOverride=chroma_override, edges=edgeMask, dark=dark_final)
#delta_restore = DeltaRestore(filtered, src, luma=luma_mask)
delta_restore = DeltaRestore(filtered, src, luma=luma_mask, chroma=chroma_mask)

edgeAreaMask = mt_logic(edgeAreaMask, darkEdgeAreaMask, "or")
#edgeAreaMask = darkEdgeAreaMask

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(src)
yellow = BlankClip(src, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(src, color=$40f75b)
gray = BlankClip(src, color=$d0d0d0)
darkgray = BlankClip(src, color=$909090)
orange = BlankClip(src, color=$a36b02)
purple = BlankClip(src, color=$4e0091)
purple_light = BlankClip(src, color=$bc6eff)
purple_dark = BlankClip(src, color=$340061)
red = BlankClip(src, color=$9e2424)
blue = BlankClip(src, color=$0d3b7a) #24579e 243c9e

# maskviz for dark mask
maskviz = Overlay(black, purple, mask=expanded)
maskviz = Overlay(maskviz, purple_dark, mask=dark)
maskviz = Overlay(maskviz, purple_light, mask=light)
maskviz = Overlay(maskviz, black, mask=dark_near_light)

# reset masks
maskviz = black
#chroma_mask = black
suppression_mask = black
chroma_override = black
edgeAreaMask = black

# only show dark mask?
showDarkMaskOnly = false
if (!showDarkMaskOnly) {
maskviz = Overlay(maskviz, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = Overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = Overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = Overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = Overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue
}

# mask overlay
comp = Overlay(delta_restore, maskviz, opacity=0.45)
comp = Overlay(comp, purple_light, mask=light, opacity=0.2)
comp = Overlay(comp, black, mask=dark, opacity=0.2)
comp = Overlay(comp, black, mask=dark_final, opacity=0.6)

maskviz = Overlay(maskviz, black, mask=dark_final, opacity=0.6)

stack = StackHorizontal(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)
#sub(edgeMask, "edge mask", txt_sz),\

# scaling (if needed)
#stack = stack.BilinearResize(stack.width/2 + 1, stack.height/2)

return stack

inter = Interleave(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

return inter

function sub(c, label, txt_sz) {
return subtitle(c, label, size = txt_sz, align=8)
}

#~ ------------------------------------------------------------------------------------------------------------------

function DeltaRestore(clip filtered, clip source, clip "luma", clip "chroma", clip "chromaOverride", clip "edges", clip "dark") {
Assert(luma.Defined || chroma.Defined || chromaOverride.Defined || dark.Defined, "DeltaRestore:must define at least one of luma, chroma, chromaOverride, dark")

mask = Default(luma, BlankClip(filtered))
if (chroma.Defined) {
mask = mask.AddTo(chroma)
}
if (chromaOverride.Defined) {
mask = mask.AddTo(chromaOverride)
}
if (edges.Defined) {
mask = mask.MustNotOverlap(edges)
}
if (dark.Defined) {
mask = mask.AddTo(dark)
}
return Overlay(filtered, source, mask=mask)
}

function DarkDeltaMask(clip mask, clip "secondary", clip "edgeMask") {
if (secondary.Defined) {
mask = mask.AddTo(secondary)
}
if (edgeMask.Defined) {
mask = mask.MustNotOverlap(edgeMask)
}
return mask
}

function AddTo(clip mask, clip other) {
return mt_logic(mask, other, "or")
}

function SuppressWith(clip mask, clip suppressionMask) {
return mt_lutxy(mask, suppressionMask, expr="x y -")
}

function MustNotOverlap(clip mask, clip other) {
other = mt_hysteresis(other, mask, chroma="-128")
return mt_lutxy(mask, other, expr="x y -")
}

function DeltaMaskChroma(clip src, clip filtered, int threshold, int "spotSize", int "expand", float "blur") {
spotSize = Default(spotSize, 3)
expand = Default(expand, 0)
blur = Default(blur, 0)

chroma_mask = ChromaDeltaMask(src, filtered, delta=threshold, spotSize=spotSize)
chroma_mask = ExpandMask(chroma_mask, expand, blur=blur)
return chroma_mask
}

function MustOverlap(clip mask, clip otherMask) {
return mt_logic(mask, mt_hysteresis(otherMask, mask), "and")
}

function storeClip(clip c, clip stored, string name) {
return propSetClip(c, name, function[stored](clip c) { return stored } )
}

function restoreClip(clip c, string name) {
return ScriptClip(c, function[name]() { return propGetClip(name) })
}

function MustBeNear(clip mask, clip otherMask, int maxDistance, bool "expanded") {
expanded = Default(expanded, false)

expand_amount = maxDistance / 2
mask_expanded = ExpandMask(mask, expand_amount)
other_expanded = ExpandMask(otherMask, maxDistance - expand_amount)
common = mt_logic(mask_expanded, other_expanded, "or")

if (expanded) {
return common
}
other_near = mt_hysteresis(otherMask, common)
final = mt_logic(mask, other_near, "and")

return final
}

function ZPadding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-127")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", float "limitAbsBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)
limitAbsBrightness = Default(limitAbsBrightness, 0.0)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# limitAbsBrightness suppression
if (limitAbsBrightness > 0.0) {
mask = mt_lutxy(mask, source, expr="y range_max "+String(limitAbsBrightness)+" * < 0 x ?")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness or limitAbsBrightness suppression (part 2)
if (limitBrightness != brightness || limitAbsBrightness > 0.0) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-127")
}
if (blur > 0) {
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
mask = mask.GBlur(rad=6, sd=blur, u=true)
}
return mask
}

Function SpotLess107(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess107: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

johnmeyer

2nd October 2021, 04:01

Zorr, thanks for the updated information. I had to complete the project, so I did the old-fashioned thing of creating two restored clips: one with dirt removal, and the other identical in every way, but with the call to RemoveDirtMC removed. I then laboriously go through the entire game, and any time the ball is hiked, kicked, fumbled, or thrown, I cut over to the version without dirt removal.

I still have the project all set up, so I'll try again tomorrow, starting with the settings you provided, and then attempting to tune the script using the information you provided.

coolgit

2nd October 2021, 12:09

Have you ever tried reducing ThSAD = Default(ThSAD,10000) to ThSAD = Default(ThSAD,6000) and Tm = Default(tm,True) to Tm = Default(tm,False)?
BlkSz,8 should be 12 or 16

I find that reducing ThSAD helps with motion analysis but too low and big spot, say size of 2 pence coin or more doesn't get wholly remove. tm,False is good for motion analysis but poorer at removing big spots.

Having used this for 2 months now I've now understood the limitations and able to detect them when scrolling through a video.

I create two files.

1 return spotless with
RadT = 1
ThSAD = 6000
Pel = 2
BlkSz = 16
OLap = (BlkSz/2)
Tm=true
Bblur = 0.3

This will remove the defects but poor at motion when close to camera. Knowing these limitations I use 2nd file.

2 return delta restore with
RadT = 1
ThSAD = 6000
Pel = 2
BlkSz = 16
OLap = (BlkSz/2)
Tm=false
Bblur = 0.3

I can run both script at the same time since vdub is piss poor at multithreading so 2 instances of vdub doesn't affect each other performance.

open 3 instances of vdub. 1 for each above files and the original file.
1 extract all spotless frames and go through spotless file for scenes that has excessive motion and close to camera motion.
2 extract those delta restore frames to replace bad spotless frames. Check no defects before extracting.
3 extract original frames to replace bad frames from the above 2. Rare. Case examples are scenes where there are white flashes etc from weapon fire or explosion. Check no defects before extracting.

Put extracted frames together and it will be 99.5% fixed. When I watch the file before encoding i detected a massive defect like over 50 pixel wide and 100 or so pixel length on a 720x576 vid.

At the beginning an episode would take 2/3 days. Spotless got that down to 12 hours. Now with the above I can do an episode in under 3 hours. That includes fixing colours, contrasts etc with photoshop.

johnmeyer

2nd October 2021, 16:20

StainlessS

2nd October 2021, 17:40

ThSAD may be extraordinarily high, but,
If src block is bad, then low ThSAD will likely use the bad src block when doing median

ThSAD, Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences].
ThSAD and ThSAD2 suggested absolute minimum of maybe about 400.
ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
in the result MCompensated frame].
Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
or some other mixture.
A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
middle pixel value.
I've just tested real bad double frame, full frame luma and chroma corruption, with below line:
SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
as we still had the temporal median filter to fall back on and pick the middle pixel value.

From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

coolgit

2nd October 2021, 19:46

The ThSAD numbers are extraordinarily high, so that was one of the first things I reduced.

As for your "1, 2, 3" suggestion, that is a much more complicated way of doing what I've been doing in my NLE for many years now (i.e., cutting between two versions of the restoration). It's a LOT easier doing it in Vegas, especially because I can use Vegas' scripts to automate much of the cutting.

I find it easier since I am good with Photoshop but zero skills using film editing software and I can't be arsed to learn it. Photoshop has scripts like 'action' command which automate the processes and image sequences (batch processing), overnight when necessary.

Each to their own I suppose.

johnmeyer

2nd October 2021, 19:54

coolgit

2nd October 2021, 19:56

coolgit

2nd October 2021, 20:11

I tried the modified script given in post #166. The good news is that it restored the ball in flight. The bad new is that it also restored almost all of the dirt.

I'll spend a little more time tweaking parameters, but for me, I don't think this is going to be useful.

Similar issue with table tennis ball (white). If a ball only exist in 1 frame and none previous/next then it would be a spot. Any sport with a ball in motion is going to be in one location then another location too far away for accurate motion analysis. I had the same when an arm moved fast in 5 frames toward the camera. Half of the arm and hand disappeared.

I do think there is a need to be a more specialise script when dealing with ball in sports.

real.finder

2nd October 2021, 20:29

if someone don't mind manual work there are

string outfile = ""

Nname of a generated file containing the detected spot locations. The file is in the .ASS subtitle file format. Spots are located with their bounding boxes drawn as magenta rectangles (dilated by 2 pixels in each direction relative to the actual bounding boxes, for clarity). The default Outline style can be later changed to Mask in an ASS editor (like Aegisub) to produce a mask made of rectangles filled with white. The file is overwritten at each filter run, so make a copy before editing it!

in DeSpot (3.6.2.0 or better), I think it can be used to mask only the dirts

zorr

2nd October 2021, 20:52

I tried the modified script given in post #166. The good news is that it restored the ball in flight. The bad new is that it also restored almost all of the dirt.

This is a different video than the one I adjusted the parameters for. The "torture clip" was in this post (https://forum.doom9.org/showthread.php?p=1945111#post1945111), right?

Thanks for posting the screenshots, it's easy to see what is wrong. The yellow chroma mask fills the whole screen (not sure why) which means it will take everything from the original source and nothing from the Spotless-filtered version. Since this looks to be a black and white clip chroma mask is going to be useless.

The ball and the dust are the same size and have pretty much the same brightness and sharpness, I think automatic masking is not going to work here.

johnmeyer

2nd October 2021, 22:13

Similar issue with table tennis ball (white). If a ball only exist in 1 frame and none previous/next then it would be a spot. Any sport with a ball in motion is going to be in one location then another location too far away for accurate motion analysis. I had the same when an arm moved fast in 5 frames toward the camera. Half of the arm and hand disappeared.

I do think there is a need to be a more specialise script when dealing with ball in sports.I think you are correct, and I don't think Zorr or anyone else should spend more time trying to perfect the masking technique, at least not for my situations.

Another big contributing factor to the masking failure in my example above and in my "torture clip" is the low frame rate of most amateur film. The American football game shown in my screenshot above is 24 fps, but most amateur film is 16 or 18 fps. The temporal gap between frames at these fps speeds is such that a thrown ball will have no spatial overlap with the ball in the previous frame, and thus will behave just like a dirt spot (i.e., it only appears in one location for one single frame, without any way to tie it to the previous frame).

Which brings me to the one thing that WOULD be useful: a luma parameter for dirt spots. I know this has been requested before, and I think someone did some work on it, but I don't know if it ever made its way into the algorithms.

Dirt on film prints is always near-black. Speckles on video are almost always near-white. A ball is almost never as dark as the dirt (although in my example above, the dirt is gray, not black). Thus, if there was a way to ignore any "object" that didn't contain at least "n" pixels less than a luma threshhold, that would help a lot.

zorr

2nd October 2021, 22:48

I think there's a little misunderstanding of how Spotless + Delta Restore works. What coolgit mentioned about the ball moving too fast for motion analysis is true - but it only affects Spotless. Delta Restore does not use any motion analysis, it only cares about the differences between original and the Spotless (or any other noise removal method) processed version.

As long as there are some consistent differences in brightness, chroma, sharpness and/or the size of the spots between real features and dust then Delta restore can restore mostly features and not the dust.

And what John just requested is one of the features of Delta Restore. :)

johnmeyer

2nd October 2021, 23:30

OK, I get it. Delta Restore is actually doing what I asked, and doing it in a rather advanced way, using more than just luma to decide which things should not have been removed. I'll try tweaking one more time.

Blankmedia

31st March 2025, 21:04

I made a function out of it,

- changed the -127 to -128
- added DCT, chroma, searchparam to Spotless107
- added frame number to debugging output, useful when interleaved

Part 1 of 2
#
# https://forum.doom9.org/showthread.php?t=182831&page=7
# Fonction SpotDel
/*
SpotDel (Spotless+Delta Restore) 1.0

Example

SpotDel(dark2Brt=0.9, dark2BrtLimit=0.85, SpotTH=10000, Rgr=True, Rgr1=5, Rgr2=10, RiskyBlur= False, TM=False, BlkSz=16)

Script intended to clean and sharpen digitalized film.
Can optionally restore grain after cleaning.
Sharpening and denoising are basic. Please see doom9 forum, VideoFred's and JohnMeyer's scripts and posts fore more knowlege. (VideoFred's sharp-blur-sharp-blur-sharp trick). Please share improvements!

Functions: Spotless:StainlessS, Delta Restore:Zorr, Restore Grain:real.finder. Huge Thanks !
Script: chmars - 17/06/2021 - some parts copied here and there, thank to solutions given by doom9 forum users.
Thread: https://forum.doom9.org/showthread.php?t=182831
Spotless thread, last versions: https://forum.doom9.org/showthread.php?t=181777
Lots of informaions on film capture/restoration: https://forum.doom9.org/showthread.php?t=144271

Script tested on HD 4:2:2 10bits sources with Avisynth+370_x64
! ! ! ! !
-You should maybe adapt the sizes parameters (blksz, radiuses, spotsizes) if your source is not HD

How it works:
Spotless removes dirt spots that are not present in neibourg frames.
But sometimes, fast moving objects are removed too (balloons, birds, hands, feet)
Hence is Delta Restore needed. It brings back details that are part of the image.
As Spotless tends to soften/degrain the image, parts brought back from the original image by Delta Restore are more grainy.
The solution for a discreete replacement (and nicer result) of these parts is to sharpen the Spotless input.

Washed out, Slog rushes:
Spotless seems to work as well with washed out images (old footage, Slog rushes), but Delta Restore needs colors and contrast to funtion properly.
In this case, a more contrasted clip can be made to feed Delta Restore and only generate masks. However, the final result will not be graded.

*/

function SpotDel(clip input, int "RadT", float "dark2Brt", float "dark2BrtLimit", bool "RiskyBlur", bool "Rgr", int "Rgr1", int "Rgr2", \
int "Usharp_strength", int "Usharp_radius", int "Usharp_th", bool "Sharpen_it", bool "Slog", int "Pel", bool "Chroma", \
int "BlkSz", int "Olap", Int "SchParam", bool "Tm", int "DCT", int "SpotTH", float "Bblur", int "txt_sz", \
bool "ReturnRestored", bool "ReturnStacked" \
)
{
# VARIABLES
# Here, define what you want

ReturnRestored = default(ReturnRestored, True) # True to returned the filterd clip, false to return comparaison
ReturnStacked = default(ReturnStacked, True) # If false return interleaved

txt_sz = default(txt_sz, 36) # Test size if infos displayed
Sharpen_it = default(Sharpen_it, True) # Default True. Sharpen clip or not (recomanded, see intro)
Slog = default(Slog, False) # Default False. If True, Generate a leveled clip f or wahed-out or Slog rushes. Final result will not be changed levelwise

#Grainitude, cancels sharpen effect
Rgr = default(Rgr, False) # Default false. True Restores grain after passed through Spotless. Might restore some dust too. 5,10 = good compromise grain/dirt.
val1 = default(Rgr1, 10)
val2 = default(Rgr2, 20) # 5,10 = good compromise grain/dirt. Dunno how that works exactly...

# Sharpen variables ----------------------------------------------------------------------------------------------------------------------------------------------------------
Usharp_strength = default(Usharp_strength, 80) # Bigger = sharper & more grain
Usharp_radius = default(Usharp_radius, 5)
Usharp_th = default(Usharp_th, 1)

# Spotless variables, to remove dirt and spots. -------------------------------------------------------------------------------------------------------------------------
# See Function script for more advanced settings
Chroma = default(Chroma, True)
BlkSz = default(BlkSz, 32) # Spot Size, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]
Olap = default(Olap, BlkSz/2)
SchParam = default(SchParam, Olap/2)
Pel = default(Pel, 1) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)
Tm = default(Tm, False) # TrueMotion Default True. Some folk swear truemotion=false is better.
DCT = Default(DCT, 0)
Bblur = default(Bblur, 0.6) # Default 0.6. If used, Suggest about 0.6, where MAnalyse create vectors is performed on denoised (blurred) super clip for better motion analysis.
ThSAD = default(SpotTH, 10000) # [SAD, Sum of Absolute (pixelwise) Differences] Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1)
RadT = default(RadT, 1) # Removes Spots on up to RadT [Temporal Radius] consecutive frames. RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
# Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

#Delta Restore variables
dark2Brt = default(dark2Brt, 0.97) # Default 0.97. Decrease if too many spots coming back
dark2BrtLimit = default(dark2BrtLimit, 0.86) # Default 0.86. Same mask. Try to change it too.
Global RiskyBlur = default(RiskyBlur, True) # True = use fastblur and risk some glitches. False = Use Gblur and go stable, at the price of a slower script

bi = BitsPerComponent(input)
source1 = input
# / END OF VARIABLES

# Leveled clip
## -------------------------------------------------------------------------------------------------------------------------------------------------------------
#In case of washed-out footage or Slog captures
# Needed to generate dark and saturated enough clips, to feed the filters whith what they need.
# Will NOT apply to final result. Only for mask building
# If Slog = True, make a decently contrasted clip to feed the filters.
# Else, do nothing, source1_lvl will be identical to source1

source1_lvl = Slog ? source1.convertbits(8).Autolevels(filterradius = 3, sceneChgThresh = 20, autogamma = true, midpoint = 0.5, autolevel = true, border=500).convertbits(bi) : source1
#~ lvl_src = Slog ? source1.convertbits(8).Autolevels(filterradius = 3, sceneChgThresh = 20, autogamma = true, midpoint = 0.5, autolevel = true, border=500, input_high = 230 ).convertbits(bi) : source1
#~ lvl_src = Slog ? ColorYUV(source1, autogain = true) : source1

#Sharpening
# -----------------------------------------------------------------------------------
# Best results found by sharpening before (not after) cleaning.
source1_shp = Sharpen_it ? source1.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th) : source1
# If Using Slog rushes, also sharpen the leveled clip
source1_lvl_shp = Sharpen_it ? source1_lvl.UnsharpMask_avsi(Usharp_strength,Usharp_radius,Usharp_th) : source1_lvl

#Remove spots, Spotless, version 1.07
# ------------------------------------------------------------------------------------
source1_shp_spt = SpotLess107(source1_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT,chroma=Chroma, DCT=DCT, SchParam=SchParam)
source1_lvl_shp_spt = Slog ? SpotLess107(source1_lvl_shp,blksz=BlkSz,OLap=Olap,pel=pel,Tm=Tm,Bblur=Bblur,thsad=thsad,RadT=RadT,chroma=Chroma, DCT=DCT, SchParam=SchParam) : source1_shp_spt

# Delta Restore -------------------------------------------------------------

# diff with exaggerated chroma difference
diff1 = mt_makediff(source1_lvl_shp, source1_lvl_shp_spt, y=3, u=3, v=3)
diff2 = diff1.mt_adddiff(diff1, y=0, u=3, v=3)
diff2 = diff2.mt_adddiff(diff1, y=0, u=3, v=3)
diff = CombinePlanes(diff1, diff2, diff2, planes="YUV", source_planes="YUV", sample_clip=diff1)

#Calculate masks with leveled clips
####----------------------------------------------------------------------------------------------------------------
# prepare masks --------------------------------------------------------------------

# luma mask - selects where source brightness > filtered brightness

# luma mask - selects where source brightness > filtered brightness
luma_mask = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, ">", brightness=1.1, spotSize=3).ExpandMask(3, blur=4)

# chroma mask - selects where source and filtered chromas have large enough difference (formula: sqrt(u*u + v*v))
chroma_mask = ChromaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, delta=3, spotSize=3)

# suppression mask - selects where source brightness < filtered brightness
# suppression mask removes too dark areas from chroma mask
suppression_mask = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, "<", brightness=0.9, spotSize=2)
suppressed_chroma_mask = chroma_mask.SuppressWith(suppression_mask)

# edge mask - detects sharp edges
# don't detect edges near the frame edges
edges = scharr(source1_lvl_shp).ZPadding(60, 14, 14, 10)
edgeMask = edges.mt_binarize(52)

# chroma override mask - selects larger color deltas and spot sizes than plain chroma mask and is not suppressed
chroma_override = ChromaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, delta=5, spotSize=6).ExpandMask(3, blur=2)

# dark mask: select both darker and lighter mask and keep dark mask if it is near lighter mask or overlaps chroma override mask
dark = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, "<", brightness=0.97, limitBrightness=0.66, spotSize=5)
light = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, ">", brightness=1.04, spotSize=5).ZPadding(60, 14, 14, 10)
dark_near_light = dark.MustBeNear(light, 28).ExpandMask(3, blur=4)
expanded = dark.MustBeNear(light, 28, expanded=true) # for debugging

# secondary dark mask - doesn't need to be near lighter area but has stricter limits
dark2 = LumaDeltaMask(source1_lvl_shp, source1_lvl_shp_spt, "<", brightness=dark2Brt, limitBrightness=dark2BrtLimit, limitAbsBrightness=0.12, spotSize=5).ExpandMask(2, blur=2)

# edge test for dark areas
darkEdgeMask = edges.mt_binarize(54) # at least 54 needed for frame 222
darkEdgeAreaMask = mt_hysteresis(darkEdgeMask, dark_near_light.AddTo(dark2), chroma="-128") # for debug

dark_final = DarkDeltaMask(dark_near_light, secondary=dark2, edgeMask=darkEdgeMask)

combined_mask = luma_mask.AddTo(suppressed_chroma_mask).AddTo(chroma_override)

# remove mask areas where sharp edges were found (those are most likely dirt)
edgeAreaMask = mt_hysteresis(edgeMask, combined_mask, chroma="-128") # for debug

#~ Apply masks with unleveled clips
####----------------------------------------------------------------------------------------------------------------
delta_restore = DeltaRestore(source1_shp_spt, source1_shp, luma=luma_mask, chroma=suppressed_chroma_mask, chromaOverride=chroma_override, edges=edgeMask, dark=dark_final)
####----------------------------------------------------------------------------------------------------------------

# Restore grain - cancels sharpen
#------------------------------------------------------------------------------------
delta_restore = Rgr ? mRD_RestoreGrain(delta_restore, source1,val1, val2) : delta_restore #5,10 = good compromise grain/dirt

if (ReturnRestored) {
return delta_restore
}

edgeAreaMask = mt_logic(edgeAreaMask, darkEdgeAreaMask, "or")
#edgeAreaMask = darkEdgeAreaMask

# create mask visualization (for debugging) -----------------------------------------
black = BlankClip(source1)
yellow = BlankClip(source1, color=$d1cc2e) # #d1cc2e f7f140
green = BlankClip(source1, color=$40f75b)
gray = BlankClip(source1, color=$d0d0d0)
darkgray = BlankClip(source1, color=$909090)
orange = BlankClip(source1, color=$a36b02)
purple = BlankClip(source1, color=$4e0091)
purple_light = BlankClip(source1, color=$bc6eff)
purple_dark = BlankClip(source1, color=$340061)
red = BlankClip(source1, color=$9e2424)
blue = BlankClip(source1, color=$0d3b7a) #24579e 243c9e

# maskviz for dark mask
maskviz = Overlay(black, purple, mask=expanded)
maskviz = Overlay(maskviz, purple_dark, mask=dark)
maskviz = Overlay(maskviz, purple_light, mask=light)
maskviz = Overlay(maskviz, black, mask=dark_near_light)

# reset masks
maskviz = black
#chroma_mask = black
suppression_mask = black
chroma_override = black
edgeAreaMask = black

# only show dark mask?
showDarkMaskOnly = false
if (!showDarkMaskOnly) {
maskviz = Overlay(maskviz, yellow, mask=chroma_mask) # chroma mask - yellow
maskviz = Overlay(maskviz, red, mask=suppression_mask, opacity=0.7) # suppression mask - red
maskviz = Overlay(maskviz, green, mask=chroma_override) # chroma override - green
maskviz = Overlay(maskviz, gray, mask=luma_mask) # luma mask - gray
maskviz = Overlay(maskviz, blue, mask=edgeAreaMask, opacity=0.8) # sharpness mask - blue
}

# mask overlay
comp = Overlay(delta_restore, maskviz, opacity=0.45)
comp = Overlay(comp, purple_light, mask=light, opacity=0.2)
comp = Overlay(comp, black, mask=dark, opacity=0.2)
comp = Overlay(comp, black, mask=dark_final, opacity=0.6)

maskviz = Overlay(maskviz, black, mask=dark_final, opacity=0.6)

ReturnStacked ? stacker(source1, source1_shp_spt, delta_restore, comp, maskviz, diff, txt_sz) : interleaver(source1, source1_shp_spt, delta_restore, comp, maskviz, diff, txt_sz)

}

Blankmedia

31st March 2025, 21:05

Part 2 of 2

function stacker(clip src, clip filtered, clip delta_restore, clip comp, clip maskviz, clip diff, int "txt_sz")
{
stack = StackHorizontal(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)
#sub(edgeMask, "edge mask", txt_sz),\

# scaling (if needed)
#stack = stack.BilinearResize(stack.width/2 + 1, stack.height/2)

return stack
}

function interleaver(clip src, clip filtered, clip delta_restore, clip comp, clip maskviz, clip diff, int "txt_sz")
{
inter = Interleave(\
sub(src, "source", txt_sz), \
sub(filtered, "Spotless", txt_sz), \
sub(delta_restore, "Delta Restore", txt_sz), \
sub(comp, "restore + mask", txt_sz), \
sub(maskviz, "masks used", txt_sz),\
sub(diff, "diff", txt_sz)\
)

return inter
}

function sub(c, label, txt_sz) {
y=Round(txt_sz/2)+10
return subtitle(c, label, size = txt_sz, align=8).ShowFrameNumber(scroll=true, x=10, y=y,size = txt_sz)
}

function DeltaRestore(clip filtered, clip source, clip "luma", clip "chroma", clip "chromaOverride", clip "edges", clip "dark") {
Assert(luma.Defined || chroma.Defined || chromaOverride.Defined || dark.Defined, "DeltaRestore:must define at least one of luma, chroma, chromaOverride, dark")

mask = Default(luma, BlankClip(filtered))
if (chroma.Defined) {
mask = mask.AddTo(chroma)
}
if (chromaOverride.Defined) {
mask = mask.AddTo(chromaOverride)
}
if (edges.Defined) {
mask = mask.MustNotOverlap(edges)
}
if (dark.Defined) {
mask = mask.AddTo(dark)
}
return Overlay(filtered, source, mask=mask)
}

function DarkDeltaMask(clip mask, clip "secondary", clip "edgeMask") {
if (secondary.Defined) {
mask = mask.AddTo(secondary)
}
if (edgeMask.Defined) {
mask = mask.MustNotOverlap(edgeMask)
}
return mask
}

function AddTo(clip mask, clip other) {
return mt_logic(mask, other, "or")
}

function SuppressWith(clip mask, clip suppressionMask) {
return mt_lutxy(mask, suppressionMask, expr="x y -")
}

function MustNotOverlap(clip mask, clip other) {
other = mt_hysteresis(other, mask, chroma="-128")
return mt_lutxy(mask, other, expr="x y -")
}

function DeltaMaskChroma(clip src, clip filtered, int threshold, int "spotSize", int "expand", float "blur") {
spotSize = Default(spotSize, 3)
expand = Default(expand, 0)
blur = Default(blur, 0)

chroma_mask = ChromaDeltaMask(src, filtered, delta=threshold, spotSize=spotSize)
chroma_mask = ExpandMask(chroma_mask, expand, blur=blur)
return chroma_mask
}

function MustOverlap(clip mask, clip otherMask) {
return mt_logic(mask, mt_hysteresis(otherMask, mask), "and")
}

function storeClip(clip c, clip stored, string name) {
return propSetClip(c, name, function[stored](clip c) { return stored } )
}

function restoreClip(clip c, string name) {
return ScriptClip(c, function[name]() { return propGetClip(name) })
}

function MustBeNear(clip mask, clip otherMask, int maxDistance, bool "expanded") {
expanded = Default(expanded, false)

expand_amount = maxDistance / 2
mask_expanded = ExpandMask(mask, expand_amount)
other_expanded = ExpandMask(otherMask, maxDistance - expand_amount)
common = mt_logic(mask_expanded, other_expanded, "or")

if (expanded) {
return common
}
other_near = mt_hysteresis(otherMask, common)
final = mt_logic(mask, other_near, "and")

return final
}

function ZPadding(clip c, int left, int top, int right, int bottom) {
return c.Crop(left, top, -right, -bottom).AddBorders(left, top, right, bottom, color_yuv=$008080)
}

function scharr(clip c) {
scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), chroma="-128")
return scharr
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-128")
return mask
}

function LumaDeltaMask(clip source, clip filtered, string direction, float "brightness", float "limitBrightness", float "limitAbsBrightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# remove too bright or dark areas using limitBrightness
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)
limitBrightness = Default(limitBrightness, brightness)
limitAbsBrightness = Default(limitAbsBrightness, 0.0)

# diff
diff = Grayscale(mt_makediff(source, filtered))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")
mask_orig = mask

# limitBrightness suppression
if (limitBrightness != brightness) {
mask_suppression = mt_lut(diff, expr="x range_half "+String(limitBrightness)+" * " + direction + " 255 scalef 0 ?")
mask = mt_lutxy(mask, mask_suppression, expr="x y -")
}

# limitAbsBrightness suppression
if (limitAbsBrightness > 0.0) {
mask = mt_lutxy(mask, source, expr="y range_max "+String(limitAbsBrightness)+" * < 0 x ?")
}

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# limitBrightness or limitAbsBrightness suppression (part 2)
if (limitBrightness != brightness || limitAbsBrightness > 0.0) {
# if suppression didn't kill the whole mask then return it without suppression
mask = mt_hysteresis(mask, mask_orig, chroma="-128")
}

return mask
}

function ChromaDeltaMask(clip source, clip filtered, int "delta", int "spotSize") {

# adjust delta to change how different colors are selected
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

delta = Default(delta, 3)
spotSize = Default(spotSize, 2)

# diff
diff = mt_makediff(source, filtered, y=3, u=3, v=3)

# extract chroma channels
u = ExtractU(diff)
v = ExtractV(diff)

# calculate deltas
u_delta = mt_lut(u, expr="x range_half - abs")
v_delta = mt_lut(v, expr="x range_half - abs")

# combine deltas
# uv_delta = mt_lutxy(u_delta, v_delta, expr="x y +")
uv_delta = mt_lutxy(u_delta, v_delta, expr="x x * y y * + 0.5 ^")

# rescale (if needed)
if (uv_delta.width < source.width || uv_delta.height < source.height) {
uv_delta = uv_delta.BilinearResize(source.width, source.height)
}

# create mask
mask = uv_delta.mt_binarize(delta)

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

# restore original format
blank = BlankClip(source)
mask = CombinePlanes(mask, blank, blank, planes="YUV", source_planes="YUV", sample_clip=source)

return mask
}

function ExpandMask(clip mask, int amount, float "blur") {
blur = Default(blur, 0)

for (i = 1, amount) {
mask = mask.mt_expand(chroma="-128")
}
if (blur > 0) {
#mask = mask.FastBlur(blur)
#mask = mask.binomialBlur(varY=blur)
mask = mask.GBlur(rad=6, sd=blur, u=true)
}
return mask
}

Function SpotLess107(clip c, int "RadT", int "ThSAD", int "ThSAD2", int "pel", bool "chroma", int "BlkSz", Int "Olap", Int "SchParam", bool "tm", int "DCT", Bool "glob", Float "bBlur", clip "dc" ) {
myName = "SpotLess107: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
SchParam = Default(SchParam, OLap/2)
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
DCT = Default(DCT,0)
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.

Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap, searchparam=SchParam, chroma=Chroma,truemotion=Tm,global=Glob, DCT=DCT)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}