SpotLess first frame isn't processing

[mohanohi]

Hi,
I am using spotless to remove spots and hairs from a 2k film scan. I am using Hybrid Vapoursynth. It does excellent job at removing all spots and hairs. But first frame doesn't seems to be affected? Can anyone pls guide me.

[johnmeyer]

I think it needs both the previous and subsequent frame for processing. Therefore I suspect you will get no improvement on either the first or last frame of your clip.

[poisondeathray]

One way you can sometimes "trick" mvtools2 based denoisers/dirt removers at a scene change, or start/end - to somewhat work - is to insert a copy a frame 2 or 3 frames away, apply filter, then delete the inserted frame . e.g if it was the first frame (frame zero) , you would insert maybe frame 2 or 3 before frame zero to "sandwich" the frame you want affect . The effectiveness is usually not as good, and more prone to artifacts, but at least you should get some cleaning if the "noise" is changing

[tormento]

One way you can sometimes "trick" mvtools2 based denoisers/dirt removers at a scene change
Shouldn't be better to insert a time reversed part of the video following the first frame?

That's a strategy that always intrigued me but I haven't the script capabilities to do that.

[Selur]

Shouldn't be better to insert a time reversed part of the video following the first frame?
untested, but I think this should do it:

# assuming source is progressive and without audio
X = 10 # needs to be > 0
clip = Trim(1,X-1).Reverse() # cuts frames 1 to X-1 and reverses them (so frames 0,1,2,3,4,...,x-1 turns into x-1,x-2,....,1)
last ++ clip # put the reversed part before the start of the clip (so now you have x-1, x-2,...,1,0,1,2,3,....)
SpotLess() # apply your filter, i.e. Spotless
Trim(X-1) # remove the first x-1 frames (now you should have 0,1,2,3,...)

Something similar could be done for the end.
Note: untested + no clue how slow Reverse is, and if your source isn't frame accurate this might cause problems. (maybe using PreRoll(X) before this helps with potential problems)

Therefore I suspect you will get no improvement on either the first or last frame of your clip.
I think so too.

Cu Selur

[anton_foy]

Couldn't this "hack" be done directly in the Spotless (or any other MC function)? It would be great as an option and I think of adding it to Clay if there is a simple solution.

[Selur]

Sure, if it works, it could be included in SpotLess, not sure whether some of the MC functions do not already have something like this.

[tormento]

Couldn't this "hack" be done directly in the Spotless (or any other MC function)? It would be great as an option and I think of adding it to Clay if there is a simple solution.
It would be much better to have a generic function to stitch in any script the reversed video to both the beginning and the end.

An incredible function would be with to apply it to every scene change, given that tools to generate scene frame start-end already exist, such as PySceneDetect (https://github.com/Breakthrough/PySceneDetect).

It was a lot that I was thinking about that strategy when denoising a video.

[tormento]

untested, but I think this should do it
Could you please extend it to make it work with both the beginning and the end of a video plus addressing the start of the reverse at a specific frame?

I'd like to test it with scenes in a video too.

[Selur]

Here's an Avisynth version:
Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
Bblur = Default(bblur,0.0) # Default OFF

preClip = Trim(c, 0, radT-1)
postClip = Trim(c, 0, -radT)

c = Reverse(preClip) + c + Reverse(postClip)

Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (bBlur<=0.0 ? c : c.blur(bblur)).MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2)
sup_rend = (bBlur<=0.0) ? sup : c.MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2,levels=1) # Only 1 Level required where not MAnalyse-ing.
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
return Trim(preClip.FrameCount, last.FrameCount -1 - radT)
}
Here's a Vapoursynth version: https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/SpotLess.py version.
But in the Avisynth version the last clip does not seem to get filtered, no clue why.
The Vapoursynth version does seem to work correctly.
Don't see what is wrong in the Avisynth version, maybe someone else sees the problem and can fix it. :)

Cu Selur

Ps.: away from PC for the rest of the day.

[johnmeyer]

Would you please actually test the function and then report on the results before you start asking for further improvements? I strongly suspect that, while the hack provided will do what it says, that first frame will probably not be cleaned as well as others.

[poisondeathray]

For the avs version, It should be preClip = Trim(c, 1, radT)

Right now Reverse(preClip) + c returns ...2,1,0,0,1,2,3... And duplicates impair cleaning/filtering

Similar for the end, it should be postClip = Trim(c, c.framecount-radT-1, c.framecount-2) because you want to append the end of the clip, not the beginning frames

Tested, working

[StainlessS]

Not really looked at PDR code (gonna have a nap), but already done this.

Test

Colorbars().Killaudio.ShowFrameNumber.Trim(0,-5) # 5 frames, numbered 0 to 4
RadT=3 # test with 1,2,3
c=last

preClip = Trim(c, 1, -radT)
postClip = Trim(c, c.Framecount-1-radT, -radT)

Reverse(preClip) + c + Reverse(postClip)

# Trim(radT, -(last.FrameCount-(2*radT)) ) # uncomment to test trimmed result

return last


My guestimate (untested)

Function SpotLessMod(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
Bblur = Default(bblur,0.0) # Default OFF

# Do validation checks first
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")

preClip = Trim(c, 1, -radT) # preClip.FrameCount=radT
postClip = Trim(c, c.Framecount-1-radT, -radT) # postClip.FrameCount=radT

c = Reverse(preClip) + c + Reverse(postClip)


pad = max(BlkSz,8)
sup = (bBlur<=0.0 ? c : c.blur(bblur)).MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2)
sup_rend = (bBlur<=0.0) ? sup : c.MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2,levels=1) # Only 1 Level required where not MAnalyse-ing.
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame

return Trim(radT, -(last.FrameCount-(2*radT)) )
}


EDIT:
PDR says 2,1,0,0,1,2,3

The way I did it was 2,1,0,1,2, ... only single 0, is mine wrong intent. (gonna nap).

EDIT: First time I did it same as PDR, then changed my mind.

[poisondeathray]

PDR says 2,1,0,0,1,2,3


To clarify I'm saying that's wrong ( the way Selur initially had it - because preClip = Trim(c, 0, radT-1) starts at "zero", so you get a duplicate)

[StainlessS]

Yeah I saw that as wrong.

How bout my sample, is that about right or should it have duplicated first and last frames ?

EDIT: very short c clip (eg 1 frame) would produce errors probably (without extra safety detection).

To below: Thanx PDR :)

[poisondeathray]

Yeah I saw that as wrong.

How bout my sample, is that about right or should it have duplicated first and last frames ?

Duplicates=bad

It works, and the syntax is cleaner (of course :) )

Tested on real film scans / video

[Selur]

Making mirroring optional probably might be a good idea:
/*
SpotLess v1.06. temporal spot/noise remover, by StainlessS @ Doom9. https://forum.doom9.org/showthread.php?t=181777
Original idea from Didée post :- https://forum.doom9.org/showthread.php?p=1402690#post1402690
Req:- Pinterf MvTools2(), Pinterf Medianblur2() with MedianBlurTemporal rather than the MedianBlurT() from Didée post.
With appropriate plugins, will be AVS+ colorspace and x64 compatible.
Fine using Avs+ Prefetch() so long as current Pinterf plugins, and Frame Accurate source. RequestLinear() following the source filter might suffice as frame accurate source filter.

NOT FOR cartoon/anime, live video only, sorry.

v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
v1.02, Add some stuff.
v1.03, Frame size auto config of args removed (found instances where caused problems). Added Glob and bBlur args.
v1.04, Script breaking changes, I guess is more flexible if can also set ThSAD, inserted ThSAD 3rd arg. RadT now default 1, was 2, dont over denoise unless requested.
v1.05, Additional checks on args.
v1.06, Glob Default true, Almost always better.
modified to filter first&last frame too

SpotLess(clip c,int "RadT"=1,int "ThSAD"=10000,int "ThSAD2"=ThSAD,int "pel"=2,bool "chroma"=true, int "BlkSz"=8,Int "Olap"=BlkSz/2,
\ bool "tm"=true,Bool "glob"=True,Float "bBlur"=0.0)

RadT, 1 or more, Default 1. Removes Spots on up to RadT [Temporal Radius] consecutive frames.
RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

ThSAD, Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences].
ThSAD and ThSAD2 suggested absolute minimum of maybe about 400.
ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
in the result MCompensated frame].
Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
or some other mixture.
A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
middle pixel value.
I've just tested real bad double frame, full frame luma and chroma corruption, with below line:
SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
as we still had the temporal median filter to fall back on and pick the middle pixel value.

From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

ThSAD2, Default ThSAD, 0 < ThSAD2 < 16320(8*8*255), Lower removes fewer spots, but less chance of blurring.
ThSAD2 sets the SAD [Sum of Absolute Differences] threshold for most distant frame from current_frame at distance RadT, with those frames that are distances in-between 1 and RadT
acquiring a SAD threshold linearly interpolated between the two.
From mvtools2 docs:
ThSAD2:
Defines the SAD soft threshold for the furthest frames at current_frame +- RadT.
The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame)
and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
EDIT: Although I have said that SAD threshold being too high could result in blurred frames, that is really taken from above "risk of bluring" line from mvtools docs,
however, that warning says "temporal denoising filter", which might suggest pixel averaging, whereas we are using pixel median. I'm not sure that blurring would be the result
of having too high a SAD threshold.

Pel, Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)

Chroma, Default True. MAnalyse chroma arg. If true, use chroma in block matching when creating vectors. Maybe use False if source B&W or color noise.

BlkSz, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]

OLap, Default half BlkSz, Block overlap.

Tm, TrueMotion Default True. Some folk swear truemotion=false is better.

Glob, Default True (True v1.06, was same as Tm, true almost always better), Allow set MAnalyse(global) independently of TrueMotion.
From MvTools2 docs for MAnalyse,
global
Estimate global motion (at every level) and use it as an additional predictor.
Only pan shift is estimated (no zoom and rotation).
Use false to disable, use true to enable.

bBlur, Default 0.0 [OFF]. If used, Suggest about 0.6, where MAnalyse create vectors is performed on denoised (blurred) super clip.
mStart, Default False. Mirror start to allow filtering first frame
mEnd, Default False. Mirror end to allow filtering first frame
*/

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur", bool "mStart", bool "mEnd") {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
Bblur = Default(bblur,0.0) # Default OFF
mStart = Default(mStart,False) # Mirror Start
mEnd = Default(mEnd,False) # Mirror End

# Do validation checks first
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")

# Add mirrored frames if specified
c = (mStart) ? Reverse(Trim(c, 1, RadT)) + c : c # add mirror frames to the start
c = (mEnd) ? c + Reverse(Trim(c, c.Framecount - RadT - 1, c.Framecount - 2)) : c # add mirror frames to the end


pad = max(BlkSz,8)
sup = (bBlur<=0.0 ? c : c.blur(bblur)).MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2)
sup_rend = (bBlur<=0.0) ? sup : c.MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2,levels=1) # Only 1 Level required where not MAnalyse-ing.
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
out = SelectEvery(RadT*2+1,RadT) # Return middle frame

# Remove mirrored frames if added
out = (mStart) ? Trim(out, RadT, out.Framecount - 1) : out # remove mirrored frames from the start
out = (mEnd) ? Trim(out, 0, out.Framecount - 1 - RadT) : out # remove mirrored frames from the end

return out
}

[StainlessS]

Making mirroring optional probably might be a good idea:
Optional Mirroring removed. ie Mirror-ing ALWAYS.

New requirement for v1.07,

v1.07, Modified to filter first&last frame too. (Selur) : FrameCount MUST be greater than radT.

Actually, probably sub-optimal when Framecount was allowed <= RadT [also makes little sense to have RadT>=FrameCount].
Additional benefit of above requirement, is that start/end mirrored frames probably NEARLY ALWAYS better result [or more likely RARELY/NEVER Worse].

Needs tester, I'm not set up for anything more than a very basic script [so below untested, should either work or throw error]

/*
SpotLess v1.07. temporal spot/noise remover, by StainlessS @ Doom9. https://forum.doom9.org/showthread.php?t=181777
Original idea from Didée post :- https://forum.doom9.org/showthread.php?p=1402690#post1402690
Req:- Pinterf MvTools2(), Pinterf Medianblur2() with MedianBlurTemporal rather than the MedianBlurT() from Didée post.
With appropriate plugins, will be AVS+ colorspace and x64 compatible.
Fine using Avs+ Prefetch() so long as current Pinterf plugins, and Frame Accurate source. RequestLinear() following the source filter might suffice as frame accurate source filter.

NOT FOR cartoon/anime, live video only, sorry.

v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
v1.02, Add some stuff.
v1.03, Frame size auto config of args removed (found instances where caused problems). Added Glob and bBlur args.
v1.04, Script breaking changes, I guess is more flexible if can also set ThSAD, inserted ThSAD 3rd arg. RadT now default 1, was 2, dont over denoise unless requested.
v1.05, Additional checks on args.
v1.06, Glob Default true, Almost always better.
v1.07, Modified to filter first&last frame too. (Selur) : FrameCount MUST be greater than radT.

SpotLess(clip c,int "RadT"=1,int "ThSAD"=10000,int "ThSAD2"=ThSAD,int "pel"=2,bool "chroma"=true, int "BlkSz"=8,Int "Olap"=BlkSz/2,
\ bool "tm"=true,Bool "glob"=True,Float "bBlur"=0.0)

RadT, Default 1. 1 <= RadT < FrameCount. Removes Spots on up to RadT [Temporal Radius] consecutive frames.
v1.07 new requirement RadT MUST be less than FrameCount.
RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

ThSAD, Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences].
ThSAD and ThSAD2 suggested absolute minimum of maybe about 400.
ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
in the result MCompensated frame].
Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
or some other mixture.
A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
middle pixel value.
I've just tested real bad double frame, full frame luma and chroma corruption, with below line:
SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
as we still had the temporal median filter to fall back on and pick the middle pixel value.

From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

ThSAD2, Default ThSAD, 0 < ThSAD2 < 16320(8*8*255), Lower removes fewer spots, but less chance of blurring.
ThSAD2 sets the SAD [Sum of Absolute Differences] threshold for most distant frame from current_frame at distance RadT, with those frames that are distances in-between 1 and RadT
acquiring a SAD threshold linearly interpolated between the two.
From mvtools2 docs:
ThSAD2:
Defines the SAD soft threshold for the furthest frames at current_frame +- RadT.
The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame)
and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
EDIT: Although I have said that SAD threshold being too high could result in blurred frames, that is really taken from above "risk of bluring" line from mvtools docs,
however, that warning says "temporal denoising filter", which might suggest pixel averaging, whereas we are using pixel median. I'm not sure that blurring would be the result
of having too high a SAD threshold.

Pel, Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)

Chroma, Default True. MAnalyse chroma arg. If true, use chroma in block matching when creating vectors. Maybe use False if source B&W or color noise.

BlkSz, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]

OLap, Default half BlkSz, Block overlap.

Tm, TrueMotion Default True. Some folk swear truemotion=false is better.

Glob, Default True (True v1.06, was same as Tm, true almost always better), Allow set MAnalyse(global) independently of TrueMotion.
From MvTools2 docs for MAnalyse,
global
Estimate global motion (at every level) and use it as an additional predictor.
Only pan shift is estimated (no zoom and rotation).
Use false to disable, use true to enable.

bBlur, Default 0.0 [OFF]. If used, Suggest about 0.6, where MAnalyse create vectors is performed on denoised (blurred) super clip.
*/

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg) : New Requirement v1.07, 1 <= RadT < c.FrameCount
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
Bblur = Default(bblur,0.0) # Default OFF
c
# Do validation checks first
Assert(1 <= RadT < FrameCount,myName + " 1 <= RadT < FrameCount")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")

# Always Add mirrored frames to either end of clip
Last.Reverse(Trim(1,-radT)) + Last + Last.Reverse(Trim(Framecount-1-radT,-radT))

pad = max(BlkSz,8)
sup = (bBlur<=0.0 ? Last : Last.blur(bblur)).MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2)
sup_rend = (bBlur<=0.0) ? sup : Last.MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2,levels=1) # Only 1 Level required where not MAnalyse-ing.
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
Last.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
Last.MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
Last.SelectEvery(RadT*2+1,RadT) # extract middle frame of every 2*RadT+1 Frames

# Remove mirrored frames
Last.Trim(RadT,-(Framecount-2*RadT))
return Last
}


The only real reason that optional mirroring might have been a good idea was to allow switch off mirroring when RadT >= Framecount where
script error may be likely [Trim() could/would produce script error, RadT < Framecount avoids this error].

[joka]

The reverse pre-clip is maybe not the correct solution.

For example (RadT=1) if the current frame is frame 1 the frames 0, 1, 2 are used. For frame 0 the frames -1, 0, 1 will be used, but MCompensate can not provide a compensated frame -1. In such a case it simply gives the current frame (here frame 0) back. Hence medium from frames 0, 0, 1 is in any case frame 0. So frame is unprocessed.
If frame -1 is provided as pre-clip like proposed above (frame 1) this results in medium 1, 0, 1 -> in any case 1. Means the result is the unfiltered (but motion compensated) frame 1.
So the pre clip in case of RadT=1 is proposed as frame 2.

For RadT=2 the pre-clip should be frame 3 + frame 4.
3, 4, 0, 1, 2, 3, 4, 5, 6, .... results in
current frame = frame 0 -- medium (3, 4, 0, 1, 2)
current frame = frame 1 -- medium (4, 0, 1, 2, 3)
current frame = frame 2 -- medium (0, 1, 2, 3, 4)
current frame = frame 3 -- medium (1, 2, 3, 4, 5), etc.
All of course motion compensated to the respective current frame.

Means Reverse(Trim(1,-radT)) should be replaced with Trim(RadT+1,-radT).

The post clip has to be adapted too.

[StainlessS]

Since about July 2023 (broke/dislocated ankle), I'm not often on the D9 now, I just pop in every day (well nearly every day) for a couple of minutes to view "todays posts".
I aint really touched avisynth much at all in more than a year (gonna have 3rd operation to remove lumps of metal from my foot in Jan).
However, I am still (when I've got time) gonna change the Spotless script as it dont quite work proper.

There is already a Spotless v1.07, https://forum.doom9.org/showthread.php?t=181777
that version number function already exists.

SpotLess(clip c,int "RadT"=1,int "ThSAD"=255*(8/2 * 8/2),int "ThSAD2"=ThSAD-(8*8),int "pel"=2,bool "chroma"=true, int "BlkSz"=8,Int "Olap"=BlkSz/2,
\ bool "tm"=true,Bool "glob"=True,Float "bBlur"=0.3, clip "dc"=Undefined)

Where dc is a user prefiltered clip alternative to using bBlur.
This dc clip would no longer work as desired with the mirrored thingy, so am gonna change (script breaking change to existing v1.07) to drop 'dc' & have a
maybe "String Prefilter" which might be user set with something like "RemoveGrain(1).Blur(0.3,0.3)" and applied internally to any
non_mirrored/mirrored input c clip.

I've got some stuff to do today, then I've gotta go out, but I'll try get time to go back to Spotless whotsit.

Your post is unclear to me, I need read it a couple of dozen more times to try figure it out.

Here is a test script similar to that posted earler, perhaps you can use/modify it to further your argument.


INLEN = 5 # Length of un-mirrored test clip
Colorbars().BilinearResize(80,60).Killaudio.Trim(0,-INLEN)
ScriptClip("""Subtitle(String(current_frame),size=64,align=2)""") # INLEN frames, numbered 0 to INLEN-1
#Return Last
#Return show()

RadT=2 # test with 1,2,3
c=last

preClip = Trim(c, 1, -radT)
postClip = Trim(c, c.Framecount-1-radT, -radT)

Reverse(preClip) + c + Reverse(postClip)
#Return show()

#Trim(radT, -(last.FrameCount-(2*radT)) ) # uncomment to test trimmed result
Return show()
return last

Function Show(clip c) {o=0 for(i=0,c.FrameCount-1) {q=c.Trim(i,-1) o=(o.IsInt)?q:o.StackHorizontal(q)} return o}

:EDITED:

Produces single frame result like this [RadT=1/2/3],
[ClickMe]
https://i.postimg.cc/LJb71kp0/s-00.jpg (https://postimg.cc/LJb71kp0)

https://i.postimg.cc/bDQFWK2K/s-01.jpg (https://postimg.cc/bDQFWK2K)

https://i.postimg.cc/B84Yrh4y/s-00.jpg (https://postimg.cc/B84Yrh4y)

EDIT: also maybe others weigh in on Joka comment, wadya think about it.

EDIT:
For example (RadT=1) if the current frame is frame 1 the frames 0, 1, 2 are used. For frame 0 the frames -1, 0, 1 will be used, but MCompensate can not provide a compensated frame -1. In such a case it simply gives the current frame (here frame 0) back.
But that RESULTING frame will be trimmed off anyway, and is not part of the final result. [I hope I understand you]
Without mirroring and RadT=1, frame 0 is unprocessed, = NOT_GOOD.
With mirroring, unmirrored frame 0 shifts & becomes now frame 1, and unprocessed NOT_GOOD frame 0 will be triimed off for result, and processed frame 1 becomes result frame 0.
HOPE that makes some sense.

input RadT=1
https://i.postimg.cc/wR9wtzqg/s-02.jpg (https://postimg.cc/wR9wtzqg)

mirrored, first and last mirrored frames are MC processing input ONLY, output of MC processing has dummy first and last frames which will be later removed.
https://i.postimg.cc/LJb71kp0/s-00.jpg (https://postimg.cc/LJb71kp0)

MC final result, dummy frames trimmed off.
https://i.postimg.cc/wR9wtzqg/s-02.jpg (https://postimg.cc/wR9wtzqg)

[poisondeathray]

Perhaps consider implementing an adjustable offset, because the farther away the frames, the more temporal artifacts . When you go more than a few frames, there are sometimes very severe artifacts

The way I usually did it in the past is 2 frames away, because the frame in interest would be surrounded by duplicates

e.g if it was the first frame (frame zero) , you would insert maybe frame 2 or 3 before frame zero to "sandwich" the frame you want affect

3 2 0 1 2

I was surprised that 2 1 0 1 2 as implemented in that current function worked decently on several film scans

[StainlessS]

Sounds a bit crazy to me :)
Can you post a short sample where you think it would work better especially as you say "because the farther away the frames, the more temporal artifacts".
You mean you purposely want to ignore the further away artefacts ? (and also ignore/skip some nearer 'good frames' even when not dupes ?)

EDIT: I would I think prefer to use a higher RadT (higher by number of consecutive dupes expected).
Also, with some kind of offset, then would have to figure out how to further limit minimum length allowed for input clip, relative to RadT+Offset or similar.
In most cases it would not matter, but cant really allow to fail on a trim() with short input clip. [without some kind of sensible error message]

EDIT: Not sure, I think trim would fail with out of range trim start and/or end, or trim frame count.

[poisondeathray]

Sounds a bit crazy to me :)
Can you post a short sample where you think it would work better especially as you say "because the farther away the frames, the more temporal artifacts".
You mean you purposely want to ignore the further away artefacts ? (and also ignore/skip some nearer 'good frames' when not dupes ?)

I'm saying you want sandwiched neighboring frames that are close , but not too far. I thought surrounding duplicates wouldn't work well at all ie. 1 0 1 , but it works ... go figure .

But if you use a completely different set of frames from another scene, or another movie, it's going to cause different sorts of temporal problems right ? something like 127 0 1 is way to far - that's what I meant by "the farther away the frames, the more temporal artifacts"

eg. for TR of 1
0 0 1 doesn't work, because adjacent duplicate
1 0 1 is the way it is now, and it works (I didn't think it would work well)
2 0 1 is what I initially suggested , offset farther, so you aren't surrounded by the same frame . In general this cleans better but has stronger temporal artifacts

But sometimes, artifacts linger or are similar on adjacent frames . By having an adjustable offset, you can tweak for better results . As you know it's a tradeoff between detail loss /temporal artifacts vs. cleaning

[StainlessS]

but it works ... go figure
EDIT: Its the magic of Didee's original script function with fixed RadT equivalent to 1.

ThSAD, Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences].
ThSAD and ThSAD2 suggested absolute minimum of maybe about 400.
ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
in the result MCompensated frame].
Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
or some other mixture.
A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
middle pixel value.
I've just tested real bad double frame, full frame luma and chroma corruption [EDIT: I think its called VHS dropout], with below line:
SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
as we still had the temporal median filter to fall back on and pick the middle [EDIT: Median] pixel value.

From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).


Also, do you have any thoughts on what default ThSAD should be.
In Spotless thread v1.07, I've used, (instead of original 10,000),


4080 : 255*(8/2 * 8/2) : 1/4 of 8x8 block @ max error 255.

but dont know if I should restore original 10,000, it seems too high really.

ThSAD, Default 10000 (Almost off). 0 < ThSAD < 16320 (ie 8*8*255).
8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences]
Some numnbers (Default 10,000 is nearly OFF, maybe Too High, suggest always try lower).
8160 : 255*(8/2 * 8) : 1/2 of 8x8 block @ max error 255.
4096 : 128*(8/2 * 8) : 1/2 of 8x8 block @ max error 128.
4080 : 255*(8/2 * 8/2) : 1/4 of 8x8 block @ max error 255.
2048 : 128*(8/2 * 8/2) : 1/4 of 8x8 block @ max error 128.
1536 : 24*(8 * 8) : Whole 8x8 block @ max error 24. Maybe general low noise.
1024 : 64*(8/2 * 8/2) : 1/4 of 8x8 block @ max error 64.
512 : 32*(8/2 * 8/2) : 1/4 of 8x8 block @ max error 32.
384 : 24*(8/2 * 8/2) : 1/4 of 8x8 block @ max error 24. Maybe no lower than this.

ThSAD and ThSAD2 suggested absolute minimum of maybe about 384.
ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
in the result MCompensated frame].
Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
or some other mixture.
A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
middle pixel value.
I've just tested real bad double frame, full frame luma and chroma corruption, with below line:
SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
as we still had the temporal median filter to fall back on and pick the middle pixel value.

From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

ThSAD2, Default ThSAD-(8*8), 0 < ThSAD2 <= ThSAD < 16320(8*8*255), Lower removes fewer spots, but less chance of blurring.
ThSAD2 sets the SAD [Sum of Absolute Differences] threshold for most distant frame from current_frame at distance RadT, with those frames that are distances in-between 1 and RadT
acquiring a SAD threshold linearly interpolated between the two.
From mvtools2 docs:
ThSAD2:
Defines the SAD soft threshold for the furthest frames at current_frame +- RadT.
The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame)
and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
EDIT: Although I have said that SAD threshold being too high could result in blurred frames, that is really taken from above "risk of bluring" line from mvtools docs,
however, that warning says "temporal denoising filter", which might suggest pixel averaging, whereas we are using pixel median. I'm not sure that blurring would be the result
of having too high a SAD threshold.


Above, some recent edits [yesterday] for ThSAD suggestions.

EDIT:
Also,

ThSAD2, Default ThSAD-(8*8), 0 < ThSAD2 <= ThSAD < 16320(8*8*255), Lower removes fewer spots, but less chance of blurring.
ThSAD2 sets the SAD [Sum of Absolute Differences] threshold for most distant frame from current_frame at distance RadT, with those frames that are distances in-between 1 and RadT
acquiring a SAD threshold linearly interpolated between the two.
From mvtools2 docs:

I been trying to figure out a good default for ThSAD2, dont know if I should make it something like
ThSAD-RadT*(8*8) or ThSAD-RadT*(8/2 * 8) or ThSAD-RadT*(8/2 * 8/2)
DTL said (somewhere) that it should be smaller, but not be TOO much smaller than ThSAD.