SpotLess - Temporal denoise

[StainlessS]

EDIT: See also, related threads:::
SpotRemover - clean video from spots:- https://forum.doom9.org/showthread.php?t=182407
[film transfer] Remove dirt but keep grain? :- https://forum.doom9.org/showthread.php?t=182831

Script filter to remove spots and noise using mvTools2 and Medianblur2.
Copes with spots on multiple consecutive frames, setting RadT accordingly.
Mvtools2/MedianBlur2 by Pinterf, so AVS+ colorspace and x64 compatible.
Based on an idea by Didée.
Originally posted here:- https://forum.doom9.org/showthread.php?p=1921583#post1921583

Spotless.avsi v1.07

/*
SpotLess v1.07. temporal spot/noise remover, by StainlessS @ Doom9. https://forum.doom9.org/showthread.php?t=181777
Original idea from Didée post :- https://forum.doom9.org/showthread.php?p=1402690#post1402690
Req:- Pinterf MvTools2(), Pinterf Medianblur2() with MedianBlurTemporal rather than the MedianBlurT() from Didée post.
With appropriate plugins, will be AVS+ colorspace and x64 compatible.
Fine using Avs+ Prefetch() so long as current Pinterf plugins, and Frame Accurate source. RequestLinear() following the source filter might suffice as frame accurate source filter.

NOT FOR cartoon/anime, live video only, sorry.

v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
v1.02, Add some stuff.
v1.03, Frame size auto config of args removed (found instances where caused problems). Added Glob and bBlur args.
v1.04, Script breaking changes, I guess is more flexible if can also set ThSAD, inserted ThSAD 3rd arg. RadT now default 1, was 2, dont over denoise unless requested.
v1.05, Additional checks on args.
v1.06, Glob Default true, Almost always better.
v1.07, bBlur Default changed from 0.0 OFF, to 0.3, slower but better for mc analysis. Maybe try 0.6 for HD to overcome HD 'grittiness'.
ThSAD default changed from 10,000 to 255 * (8/2 * 8/2), ie 1/4 of 8x8 block at 255, ie 1/4 BlkSz spots @ max SAD. Also ThSAD2 a bit less than ThSAD.
Added dc arg, detect clip.

SpotLess(clip c,int "RadT"=1,int "ThSAD"=255*(8/2 * 8/2),int "ThSAD2"=ThSAD-(8*8),int "pel"=2,bool "chroma"=true, int "BlkSz"=8,Int "Olap"=BlkSz/2,
\ bool "tm"=true,Bool "glob"=True,Float "bBlur"=0.3, clip "dc"=Undefined)

RadT, 1 or more, Default 1. Removes Spots on up to RadT [Temporal Radius] consecutive frames.
RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

ThSAD, Default 255 * (8/2 * 8/2) ie 4080 <ie 1/4 of 8x8 block at 255, ie 1/4 BlkSz spots @ max SAD>.
0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences].
ThSAD and ThSAD2 suggested absolute minimum of maybe about 400.
ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
in the result MCompensated frame].
Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
or some other mixture.
A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
middle pixel value.
I've just tested real bad double frame, full frame luma and chroma corruption, with below line:
SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
as we still had the temporal median filter to fall back on and pick the middle pixel value.

From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

ThSAD2, Default ThSAD-(8*8), 0 < ThSAD2 <= ThSAD < 16320(8*8*255), Lower removes fewer spots, but less chance of blurring.
ThSAD2 sets the SAD [Sum of Absolute Differences] threshold for most distant frame from current_frame at distance RadT, with those frames that are distances in-between 1 and RadT
acquiring a SAD threshold linearly interpolated between the two.
From mvtools2 docs:
ThSAD2:
Defines the SAD soft threshold for the furthest frames at current_frame +- RadT.
The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame)
and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
EDIT: Although I have said that SAD threshold being too high could result in blurred frames, that is really taken from above "risk of bluring" line from mvtools docs,
however, that warning says "temporal denoising filter", which might suggest pixel averaging, whereas we are using pixel median. I'm not sure that blurring would be the result
of having too high a SAD threshold.

Pel, Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)

Chroma, Default True. MAnalyse chroma arg. If true, use chroma in block matching when creating vectors. Maybe use False if source B&W or color noise.

BlkSz, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]

OLap, Default half BlkSz, Block overlap.

Tm, TrueMotion Default True. Some folk swear truemotion=false is better.

Glob, Default True (True v1.06, was same as Tm, true almost always better), Allow set MAnalyse(global) independently of TrueMotion.
From MvTools2 docs for MAnalyse,
global
Estimate global motion (at every level) and use it as an additional predictor.
Only pan shift is estimated (no zoom and rotation).
Use false to disable, use true to enable.

bBlur, Default 0.3. If used, Suggest about 0.3, where MAnalyse create vectors is performed on denoised (blurred) super clip
for better motion analysis. Maybe try 0.6 for HD clip to counteract HD 'grittiness'. bBlur ignored if dc clip specified,
If providing bBlur, then ALWAYS specify as named argument ie bBlur=bBlur, we will likely insert any additional MvTools args
before bBlur to keep them together.

dc, Default UnDefined. If dc Specified, then must be same size and colorspace as source clip.
If detection clip specified then bBlur prefilter arg is ignored, and analysis is performed on
dc.MSuper() clip [ instead of c.Blur(bBlur).MSuper() ]. Allows to provide your own prefiltered clip.
If providing dc clip, then ALWAYS specify as named argument ie dc=DC, we will likely insert any additional MvTools args
before bBlur to keep them together.

*/

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",
\ Float "bBlur", clip "dc" ) {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,255*(8/2 * 8/2)) # SAD threshold at radius 1. Here is max'ed out at 255 * 1/4 of 8*8 block <ie 4080> (MvTools Default 10,000 is nearly OFF, maybe way Too High).
# Maybe above 255*8/2*8/2 still a bit high, maybe 128*(8/2 * 8/2) Better <ie 2048>. For General noise perhaps somethimg like 24 * (8 * 8) <ie 1536>.
ThSAD2 = Default(ThSAD2,ThSAD-(8*8)) # SAD threshold at radius RadT(a bit less than ThSAD).
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.3) # Default 0.3. Suggest about 0.3 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}


Hacked this for a clip with spots on double frames, did not know if would be much better, but as it turns out is fandabbydozy.

Chosen function name would seem to be a very good choice, not perfect, but very nearly is.

Adjective: spotless
1) Completely neat and clean

You should also try spotless as a prefilter in smdegrain. Worked very good for me with almost zero artifacts left from spotless.

yes, it's do better job than any methods ...

This is a wonderful tool. As it happens, I've been cleaning a VHS cap and the silly video has quite a lot of spots and scratches in every other field. SpotLess cleans them up nicely and also takes care of most of the chroma noise too.. I haven't checked things with longer samples but looks good anyway.

Here Vapoursynth version of Spotless, posted later in this thread by MysteryX:- https://forum.doom9.org/showthread.php?p=1955170#post1955170
In VapourSynth, it currently only works with the 32-bit library. The dubhater library has a bug.

Find also Spotless_v1.07_20220928.zip @ MediaFire in sig below this post.

[StainlessS]

Perhaps someone would like to test SpotLess() on 'old cartoon' which is subject of this thread.
I dont usually touch cartoon/anime at all, and even less 'old cartoon'.
I not sure that Mvtools copes well with cartoon at all.

It doesn't work on anime, it wiped out all moving objects in motion scenes.

So not much good for cartoon/anime, sorry.

[StainlessS]

If you've got a clip with a big dollop of black crap (spot) at frame edge which is not removed by SpotLess, may possibly
be removed by eg AddBorders(32,32,32,32,$808080) prior to SpotLess and Crop(32,32,-32,-32) afterwards.
I had one of 4 frame duration which was successfully removed with RadT=6, [maybe I should up RadT max to 8 or 10 or something, for one-offs].
Will likely not work if 1st or last spot near scene change.

EDIT: My spot would seem to be a film burn hole [dont know what they are called].
EDIT: MSuper( ..., hpad=32,vpad=32) [ EDIT: inside of Spotless] dont do it, gotta Addborders pad with grey or white to get rid of black spot).
EDIT: Weird, was working fine, now only removes 2 of the 4 spots, ???
EDIT: I may have modified prior color correction since the time it removed all 4 spots, still only removing 2 spots.

[EDIT: Frame Top Right Hand corner] Source spot, burn in near vacinity of other dark object
https://i.postimg.cc/V0mKqdXM/Play-1-B-01.jpg (https://postimg.cc/V0mKqdXM)
Spotless fixed
https://i.postimg.cc/Dmhhj6gQ/Play-1-B-02.jpg (https://postimg.cc/Dmhhj6gQ)
Next frame, Not Fixed (well modified a bit but) is too close now to dark object
https://i.postimg.cc/7bNkpzsF/Play-1-B-00.jpg (https://postimg.cc/7bNkpzsF)
The color correction that I changed somehow previously allowed fix, but not now, forgot what prev color correction was.

[Boulder]

EDIT: My spot would seem to be a film burn hole [dont know what they are called]

I think that's called "a cigarette burn" :) I wouldn't be surprised if they actually were those back in the day.

[StainlessS]

Change from

MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=8,overlap=4,search=4)

To

MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=32,overlap=16,search=4)

Allows detect and removal of above posted spot, I'll make Blocksize selectable.
Spot from above bottom image Now gone [as is the last of the 4 spots also removed, so all 4 spots now gone].
https://i.postimg.cc/jwL7GKK9/Play-1-B-Copy-00.jpg (https://postimg.cc/jwL7GKK9)

EDIT
Cigarette burn, well I thought could happen when reel stopped spinning and projector lamp still hot.

[StainlessS]

1st post updated script.

v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.


EDIT:
Adding hpad, vpad greater than 8, assists when changing framerate, avoids oscillating effects at frame edge sometimes, will not do any harm to also have it here.

[Boulder]

EDIT
Cigarette burn, well I thought could happen when reel stopped spinning and projector lamp still hot.

My thought was more nostalgic, like people cutting the film while smoking heavily and dropping ashes and sparks all over the place :)

[StainlessS]

Yeh, you're problably correct in my particular spot instance, think projector lamp burns were not always totally symmetrical.

Nostalgia, its not what it used to be. [stolen from some D9 guys sig].

Maybe I should set hpad/vpad to same as BlkSz, when above 8.

Also, ClipClop, [or ReplaceFramesSimple] could be used to apply special instance SpotLess args to awkward ranges where spots resist fixing.

[StainlessS]

Clipclop script

AviSource(".\Play_1.avi")
O=Last
SpotLess() # Apply Spotless(Radt=2,Blksz=8) to entire clip

FIX1=O.AddBorders(32,32,32,32,$808080).Spotless(RadT=4,BlkSz=32).Crop(32,32,-32,-32) # Special clip index 1
FIX2=O.AddBorders(32,32,32,32,$808080).Spotless(RadT=5,BlkSz=32).Crop(32,32,-32,-32) # Special clip index 2

SCMD= """
1 28251,28254 # Replace fix range 28251,28254 using FIX1 [clip index 1]
2 30001,30005 # Replace fix range 30001,30005 using FIX2 [clip index 2]

"""

ClipClop(Last,FIX1,FIX2,scmd=SCMD) # also has file commands. Max 255 replacement clips [unlimited replacment ranges]
StackHorizontal(O,Last)


ClipClop:- https://forum.doom9.org/showthread.php?t=162266&highlight=Clipclop

EDIT: Its like magic :) Now to go back and try fix non removed large frame edge spot that I did yesterday.

Or mod


AviSource(".\Play_1.avi")
O=Last
SpotLess(2) # Apply Spotless(Radt=2,Blksz=8) to entire clip

FIX1=O.EdgeKludge(4) # Special clip index 1
FIX2=O.EdgeKludge(5) # Special clip index 2

SCMD= """
1 28251,28254 # Replace fix range 28251,28254 using FIX1 [clip index 1]
2 30001,30005 # Replace fix range 30001,30005 using FIX2 [clip index 2]

"""

ClipClop(Last,FIX1,FIX2,scmd=SCMD) # also has file commands # BUGFIX, ClipClop(Last ... NOT ... ClipClop(O
StackHorizontal(O,Last)

Return Last

Function EdgeKludge(clip c,Int RadT,Int "BlkSz", Int "Border") {
BlkSz=Default(BlkSz,32)
Border=Default(Border,32)
Return c.AddBorders(Border,Border,Border,Border,$808080).Spotless(RadT=RadT,BlkSz=BlkSz).Crop(Border,Border,-Border,-Border)
}

[FranceBB]

I think that's called "a cigarette burn" :) I wouldn't be surprised if they actually were those back in the day.

Is it?
I don't know whether we're talking about the same thing and I might be wrong but to me it looks like a cue mark (https://en.wikipedia.org/wiki/Cue_mark), which was used to let the projectionist know that the reel was close to the end or that it reached a certain length. Even though those things were used way before I was even born, I actually found 'em in some of the very old contents I'm Remastering and in the recording-sheet they were saying like "Cue Mark at " and so on

[Boulder]

Is it?
I don't know whether we're talking about the same thing and I might be wrong but to me it looks like a cue mark (https://en.wikipedia.org/wiki/Cue_mark), which was used to let the projectionist know that the reel was close to the end or that it reached a certain length. Even though those things were used way before I was even born, I actually found 'em in some of the very old contents I'm Remastering and in the recording-sheet they were saying like "Cue Mark at " and so on

You could well be right, especially if you've run into those like that :)

[Frank62]

I actually found 'em in some of the very old contents

I find these every week in for-me-not-so-old-content. Round or sometimes square. If the content is worth it, we fix it by hand, but this way looks quite elegant. Will give it a try. Thanks @StainlessS again! :)

[StainlessS]

There were so many fag burns, all in same-ish position, and all 4 frames duration, so FBB probably correct.
The 4 frame replacements in the script are nearly all, 'fag burns'.


The 'border color' prob, imported a few pixels of the gray addborders into where the burn was, probably due to burn surround color being mid green (close to mid grey),
so I changed the kludge fliter to use YUV PC Levels white, and that fixed the grey imports.
One of the spots fixes caused a little bit of blurring on fast moving obect, but way better than the burn, so left it [maybe could do special fix to apply just to spot 1/4 of frame,
or through mask, but probably gonna leave it, so hard to notice.
At one point, had mid frame bad full frame width vhs breakup, maybe 30 or 40 pixels tall, and also full frame desaturation, kludge fixed both perfectly, hehe.

Script used

AviSource(".\Play_1.avi")
O=Last
SpotLess(2) # Apply Spotless(Radt=2,Blksz=8) to entire clip

FIX1 = O.EdgeKludge(1)
FIX2 = O.EdgeKludge(2)
FIX3 = O.EdgeKludge(3)
FIX4 = O.EdgeKludge(4)
FIX5 = O.EdgeKludge(5)
FIX6 = O.EdgeKludge(6)
FIX7 = O.EdgeKludge(7)
FIX8 = O.EdgeKludge(8)
FIX9 = O.EdgeKludge(9)

SCMD= """
3 1756,-3 # ripple [-3 means 3 frames, instead of start,end frames]
4 28251,-4 #
1 28352 # single frame
4 28426,-4
4 51911,-4
4 78032,-4
4 78208,-4
4 105201,-4 # attn, change to border color = YUV PC levels white
4 105377,-4
7 122341,-7
4 127242,-4 # attn, desat
4 127418,-4
8 130004,-8 # film stratch
4 133844,-4
"""

ClipClop(Last,FIX1,FIX2,FIX3,FIX4,FIX5,FIX6,FIX7,FIX8,FIX9,scmd=SCMD) # also has file commands : BUGFIX: NOT ClipClop(O, but ClipClop(Last,
StackHorizontal(O,Last)

Return Last

Function EdgeKludge(clip c,Int RadT,Int "BlkSz", Int "Border",int "ColYUV") {
BlkSz=Default(BlkSz,32)
Border=Default(Border,32)
ColYUV=Default(ColYUV,$FF8080) # YUV PC RANGE WHITE

Return c.AddBorders(Border,Border,Border,Border,Color_yuv=ColYUV).Spotless(RadT=RadT,BlkSz=BlkSz).Crop(Border,Border,-Border,-Border)
}


EDIT: Not sure, but I think the original SpotLess(RadT=2) also fixed a single full frame dropout, I did not notice it in the fixed version.

Didee really should have pushed his idea more, this thing is bloody marvelous.
EDIT: GMJCZP did KillerSpots() function, but only single frame fixes.

EDIT: Above I used EdgeKludge() funcs, even for non edge problems, just saved time not doing separate functions.

EDIT: BUGFIX in above script, did not matter to me as I had already called Spotless(2) creating the input AVI

[StainlessS]

SpotLess Update v1.02. See 1st post.

SpotLess v1.02. temporal spot/noise remover, by StainlessS @ Doom9. https://forum.doom9.org/showthread.php?t=181777
Original idea from Didée post :- https://forum.doom9.org/showthread.php?p=1402690#post1402690
Req:- Pinterf MvTools2, Pinterf Medianblur2() with MedianBlurTemporal rather than the MedianBlurT() from Didée post.
With appropriate plugins, will be AVS+ colorspace and x64 compatible.

NOT FOR cartoon/anime, live video only, sorry.
v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
v1.02, Add some stuff.

RadT, 1 or more, Default 2. Removes Spots on up to RadT consecutive frames.
RadT > 2 will usually be overkill. Setting too high could possibly result in blurring [no problems encountered so far at RadT=6].
ThSAD2, 400 <= ThSAD2 <= 10000, Default 10000=NEARLY OFF. Lower removes fewer spots, but less chance of blurring.
Reduce if produces blurred frames, absolute minimum of maybe about 400, probably pretty good as is.
Lower reduces effect of bad non matching blocks, reduced more so on blocks from more distant radii frames.
ThSAD for frame @ distance 1 defaults to 10,000 [and we let it default], ThSAD2 sets the SAD [Sum of Absolute Differences] threshold
for most distant frame at distance RadT, with those frames that are distances in-between being given a SAD threshold linearly spread between the two.
Wild guess that maybe about 10000 at RadT=1, and 5000 (10000/2) where RadT=6, might be a good compromise if concerned about blurring.
Suggested, based on wild guesses above :) [if you experience blurred results, otherwise leave as is]
Where below RadT is 6, and ThSAD2=5000, then blocks @ frames current_frame +- 6 gets weighted SAD threshold value of 5000 (same as ThSAD2),
@ current_frame +- 3, gets distance weighted SAD threshold value 8000, etc
Radius SAD threshold : Where RadT=1, SAD threshold=ThSAD2. Where RadT>1; SAD threshold=(RadT-Radius)*((10000-ThSAD2)/(RadT-1))+ThSAD2; (6-Radius)*((10000-5000)/(6-1))+5000.
1 10000 10000, Almost switches OFF SAD thresholding at frame distance 1. [Ignores blocks of only really high SAD differences]
2 9000
3 8000 In between frames acquire SAD threshold value linearly between 10000 @ distance 1, and ThSAD2[here 5,000] at distance RadT[here 6].
4 7000
5 6000
6 5000 Where RadT=6 and ThSAD2 = 5000, so frame @ distance 6 gets SAD threshold value of ThSAD2[5000]
And to weight similar to above where eg RadT=3, set RadT=3, ThSAD2=8000 to set as above Radius=3,SAD threshold=8000. Radii 2 and 1 will acquire 9000 and 10000 resp.
From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).
ThSAD2:
Defines the SAD soft threshold for the furthest frames.
The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame, EDIT: Default 10000)
and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
In SpotLess, we dont actually set mvtools ThSAD arg, only ThSAD2, ThSAD will default 10000, but where RadT=1 and you set ThSAD2 to eg 5000 it should (I hope)
set soft SAD threshold at radii 1 to ThSAD2 [and override the ThSAD default 10000].
If anybody can explain above any better, then please do. I hate doing docs and as you can see, I aint very good at it.
Pel, Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma, Default True. MAnalyse chroma arg. If set to true. Use chroma in block matching.
BlkSz, Default c.width>960 ? 16 : 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips (Maybe on occasion 24 or 32).
OLap, Default half BlkSz, Block overlap.
tm, TrueMotion, False if BlkSz>8, else true. Some folk swear truemotion=false is better, NOTE MAnalyse Global will always be true.


EDIT: From MvTools2 docs for MAnalyse,
global

Estimate global motion (at every level) and use it as an additional predictor.
Only pan shift is estimated (no zoom and rotation).
Use false to disable, use true to enable. Default is like truemotion.
We always set Global=true, even when Truemotion turned OFF.

[StainlessS]

SpotLess v1.03, update, see 1st post.


SpotLess v1.03. temporal spot/noise remover, by StainlessS @ Doom9. https://forum.doom9.org/showthread.php?t=181777
Original idea from Didée post :- https://forum.doom9.org/showthread.php?p=1402690#post1402690
Req:- Pinterf MvTools2, Pinterf Medianblur2() with MedianBlurTemporal rather than the MedianBlurT() from Didée post.
With appropriate plugins, will be AVS+ colorspace and x64 compatible.

NOT FOR cartoon/anime, live video only, sorry.
v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
v1.02, Add some stuff.
v1.03, Frame size auto config of args removed (found instances where caused problems). Added Glob and bBlur args.

SpotLess(clip c,int "RadT"=2,int "ThSAD2"=10000,int "pel"=2,bool "chroma"=true, int "BlkSz"=8,Int "Olap"=BlkSz/2,bool "tm"=true,Bool "glob"=Tm,Float "bBlur"=0.0)

RadT, 1 or more, Default 2. Removes Spots on up to RadT consecutive frames.
RadT > 2 will usually be overkill. Setting too high could possibly result in blurring [no problems encountered so far at RadT=6].
ThSAD2, 400 <= ThSAD2 <= 10000, Default 10000=NEARLY OFF. Lower removes fewer spots, but less chance of blurring.
Reduce if produces blurred frames, absolute minimum of maybe about 400, probably pretty good as is.
Lower reduces effect of bad non matching blocks, reduced more so on blocks from more distant radii frames.
ThSAD for frame @ distance 1 defaults to 10,000 [and we let it default], ThSAD2 sets the SAD [Sum of Absolute Differences] threshold
for most distant frame at distance RadT, with those frames that are distances in-between being given a SAD threshold linearly spread between the two.
Wild guess that maybe about 10000 at RadT=1, and 5000 (10000/2) where RadT=6, might be a good compromise if concerned about blurring.
Suggested, based on wild guesses above :) [if you experience blurred results, otherwise leave as is]
Where below RadT is 6, and ThSAD2=5000, then blocks @ frames current_frame +- 6 gets weighted SAD threshold value of 5000 (same as ThSAD2),
@ current_frame +- 3, gets distance weighted SAD threshold value 8000, etc
Radius SAD threshold : Where RadT=1, SAD threshold=ThSAD2. Where RadT>1; SAD threshold=(RadT-Radius)*((10000-ThSAD2)/(RadT-1))+ThSAD2; (6-Radius)*((10000-5000)/(6-1))+5000.
1 10000 10000, Almost switches OFF SAD thresholding at frame distance 1. [Ignores blocks of only really high SAD differences]
2 9000
3 8000 In between frames acquire SAD threshold value linearly between 10000 @ distance 1, and ThSAD2[here 5,000] at distance RadT[here 6].
4 7000
5 6000
6 5000 Where RadT=6 and ThSAD2 = 5000, so frame @ distance 6 gets SAD threshold value of ThSAD2[5000]
And to weight similar to above where eg RadT=3, set RadT=3, ThSAD2=8000 to set as above Radius=3,SAD threshold=8000. Radii 2 and 1 will acquire 9000 and 10000 resp.
From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).
ThSAD2:
Defines the SAD soft threshold for the furthest frames.
The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame, EDIT: Default 10000)
and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
In SpotLess, we dont actually set mvtools ThSAD arg, only ThSAD2, ThSAD will default 10000, but where RadT=1 and you set ThSAD2 to eg 5000 it should (I hope)
set soft SAD threshold at radii 1 to ThSAD2 [and override the ThSAD default 10000].
If anybody can explain above any better, then please do. I hate doing docs and as you can see, I aint very good at it.
Pel, Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma, Default True. MAnalyse chroma arg. If set to true. Use chroma in block matching.
BlkSz, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips.
OLap, Default half BlkSz, Block overlap.
Tm, TrueMotion Default True. Some folk swear truemotion=false is better.
Glob, Default Tm, Allow set MAnalyse(global) independently of TrueMotion.
From MvTools2 docs for MAnalyse,
global
Estimate global motion (at every level) and use it as an additional predictor.
Only pan shift is estimated (no zoom and rotation).
Use false to disable, use true to enable. Default is like truemotion.
bBlur, Default 0.0 [OFF]. If used, Suggest about 0.6, where MAnalyse create vectors performed on denoised (blurred) super clip.



Reverted some args, BlkSz=8, tm(truemotion)=true.
Found instances where fast-ish moving objects got swallowed up [a swift 'heil hitler' would do it], so reverted to safer defaults.
Added Glob, and 'bBlur args.

EDIT: From MvTools2 docs for MAnalyse,
global

Estimate global motion (at every level) and use it as an additional predictor.
Only pan shift is estimated (no zoom and rotation).
Use false to disable, use true to enable. Default is like truemotion.

[StainlessS]

Here script that I've been using for SingleRate deinterlace of mixed Progressive / Interlaced PAL.
Removed from DBSC thread last post, as have changed it here.
I used SpotLess on pre-deinterlaced source, it seemed like a good idea at the time but causes fuzziness around eg eyes
and sort of prevents deinterlace where fuzzy.
So, moved SpotLess to after deinterlace, much better detail and acts as a sort of residual combing fixer.
I only use below for single rate deinterlacing [ie OUTPUT=0], the other modes being experimental.
Many oldish [80's] PAL TV sourced DVD's are mixes of interlaced and progressive, the one I've recently been 'doin' spotless for is interlaced video
color for indoor scenes, progressive for outdoor scenes, and progressive B&W for ancient footage.
Such PAL hybrid progressive/interlaced is quite often cursed with changing borders, I use this script prior to removing dynamically changing borders
with DBSC.

DGIndex template, DVD_PAL_Hybrid_DeintSingleRate.avs

### DGIndex Template, PAL_Hybrid_DeintDoubleRate.avs
VideoFileName ="__vid__"
AudioFileName ="__aud__"
AudioDelay =Value("__del__")

###############
# PAL Hybrid Interlaced/Progressive mix, output to singlerate or doublerate.
###############
MPEG2Source(VideoFileName)
AudioExt=RT_GetFileExtension(AudioFileName)

Audio= (AudioExt==".ac3") ? NICAC3Source(AudioFileName,channels=2,DRC=0)
\ : (AudioExt==".mpa"||AudioExt=="mp1"||AudioExt==".mp2"||AudioExt==".mp3") ? NicMPG123Source(AudioFileName,Normalize=False)
\ : (AudioExt==".wav") ? RaWavSource(AudioFileName)
\ : (AudioExt==".dts") ? NicDTSSource(AudioFileName)
\ : 0

Assert(!isInt(Audio),"NO AUDIO")

(!isInt(Audio)) ? AudioDub(Audio).DelayAudio(AudioDelay).Trim(0,0) : NOP # Trim, chop/add audio to length

(!isInt(Audio) && AudioRate() <> 44100) ? ResampleAudio(44100) : NOP
####### CONFIG ##########

AssumeTFF # Whatever (check) : Almost always TFF for PAL DVD. Default field order is BFF.

# Uncomment below if has any Perverse Telecine [eg Babylon5].
TFM(pp=0)
O=Last
# and a bit of color tweaking, PRE-MC stuff (else VERY SLOW)
#AutoAdjust(Auto_Gain=true).Merge(O,0.6) # un-comment if wished

OUTPUT = 3 # 0=SingleRate 25FPS Progressive. Progressive src frames as input, interlaced src frames are even frames from dbl rate deinterlacing.
# 1=SingleRate 25FPS Hybrid. Progressive src frames as input, interlaced src frames dbl rate deinterlaced, reinterlaced to 25FPS after processing.
# 2=DblRate 50fps. Prog=ChangeFPS. Progressive src frames doubled with ChangeFPS, interlaced src frames are dbl rate deinterlaced.
# 3=DblRate 50fps. Prog=Interpolate. Progressive src frames double rate interpolated with JohnFPS [ie MvTools], interlaced src frames are dbl rate deinterlaced.

UNSPOT_P=true # Use SplotLess() on Progressive parts to remove spots and noise.
UNSPOT_I=False # Use SplotLess() on Interlaced parts to remove spots and noise.

RADT_P = 2 # SpotLess temporal radius (spots on 2 consecutive frames)
ThSAD_P = 5000 # Almost OFF.
ThSAD2_P = 5000 # Almost OFF.
BlkSz_P = 16
Tm_P = false
Glob_P = true
bBlur_P = 0.6

FRAMES_P=1 # Progressive McDegrainSharp(frames=FRAMES_P), (0=Dont McDegrainSharp)
CSHARP_P=0.5

RADT_I = 2 # SpotLess temporal radius (spots on 2 consecutive frames)
ThSAD_I = 5000 # Almost OFF.
ThSAD2_I = 5000 # Almost OFF.
BlkSz_I = 16
Tm_I = false
Glob_I = true
bBlur_I = 0.6

FRAMES_I=2 # Interlaced McDegrainSharp(frames=FRAMES_I), (Maybe double FRAMES_P for detail migration, although max possible for McDegrainSharp=3). (0=Dont McDegrainSharp)
CSHARP_I=CSHARP_P

# IsCombedTIVTC args, Defaults, 1st after comment hash.
CTHRESH = 12 # 10), Area combing threshold.
# Valid settings are from -1 (every pixel will be detected as combed) to 255 (no pixel will be detected as combed).
# This is basically a pixel difference value. A good range is between 8 to 12.
MI = 80 # 80), Number of combed pixels inside any of the blocky by blockx size blocks on the frame for the frame to be detected as combed.
CHROMA = True # True), Whether or not chroma is considered in the combed frame decision. [We want chroma as interlaced parts in color]
BLOCKX = 16 # 16), X-axis size of the window used during combed frame detection.
BLOCKY = 16 # 16), Y-axis size of the window used during combed frame detection.
# So in any single block, if 85[MI] out of 256[BLOCKX x BLOCKY] pixels pass CThresh as combed, then is Interlaced.
METRIC = 1 # 0) Detecto method: 0=Old, 1=New

# Deinterlace mode
BOBMOD=1 # 0=BOB, 1=YadifMod2, 2=QTGMC : QTGMC here is way memory/CPU intensive, req x64. Suggest YadifMod2

BOB_FUNC_0 = """Bob()""" # Cheap & cheerful Doublerate simple Bob.
BOB_FUNC_1 = """YadifMod2(Order=-1,Mode=1,edeint=NNedi3(Field=-2, nns=2))""" # Doublerate Yadifmod2 filter.
# # Field-2=double rate, internal parity value to start : Order-1=Internal Parity. Mode 1=double rate do spactial check (3=no spatial check).

BOB_FUNC_2 = """QTGMC(Preset="fast",NoisePreset="Fast",SourceMatch=2,Lossless=2,EZKeepGrain=0.4,Sharpness=0.1, tr2=0)""" # Doublerate QTGMC filter.

FUNC = Select(BOBMOD, BOB_FUNC_0, BOB_FUNC_1, BOB_FUNC_2)
FNAME = Select(BOBMOD,"BOB", "YADIF", "QTGMC")


PRE_FETCH=6 # 1 = No Prefetch, Else eg 4 = Prefetch(4) ie MT mode

### # Display stuff
SUBS = True # Show where Deinterlaced/ReInterlaced
STACK = True # Show Src and Result
TITLE = True # Show Title Bar
TITNUM = True # Show Frame Number on Title Bar (Only if TITLE)
FINAL = True # True, switch off SUBS, STACK, and Title

##### End Config ##########

Src=Last

SUBS = (!FINAL&&SUBS)
TITLE = (!FINAL&&TITLE)
STACK = (!FINAL&&STACK)


### IsCombedTIVTC with args, as a string eg "IsCombedTIVTC(cthresh=10,mi=85,chroma=true,blockx=16,blocky=16,metric=1)"
COMBDET = RT_String("IsCombedTIVTC(cthresh=%d,mi=%d,chroma=%s,blockx=%d,blocky=%d,metric=%d)",CTHRESH,MI,CHROMA,BLOCKX,BLOCKY,METRIC)
SUBTXT = (OUTPUT==0) ? FNAME+"_DEINTERLACED-EVEN" : (OUTPUT==1) ? FNAME+"_RE-INTERLACED" : FNAME+"_DblRate-DEINTERLACED"

# Interlaced parts
# DeSpot, Deinterlace to doublerate, McDeGrainSharp to migrate detail between original fields[now frames].
Deint = Src.Eval(FUNC) # Double Rate deinterlace
Deint = (OUTPUT==0) ? Deint.SelectEven : Deint # If (OUTPUT==0) McDegrainSharp & SpotLess done after Deinterlace.SelectEven, Else after doublerate deinterlace.
Deint = (UNSPOT_I) ? Deint.SpotLess(Radt=RADT_I,thsad=ThSAD_I,thsad2=ThSAD2_I,blksz=BLKSZ_I,tm=Tm_I,glob=Glob_I,bBlur=bBlur_I) : Deint
Deint = (FRAMES_I>0) ? Deint.McDeGrainSharp(frames=FRAMES_I,csharp=CSHARP_I) : Deint
Deint = (OUTPUT==1) ? Deint.SeparateFields.SelectEvery(4,0,3).Weave : Deint # If singlerate Re-Interlacing
Deint = (SUBS) ? Deint.Subtitle(SUBTXT,Align=5) : Deint

# Progressive parts, Despot, Degrain sharp done still @ singlerate
Prog = Src
Prog = (UNSPOT_P) ? Prog.SpotLess(Radt=RADT_P,thsad=ThSAD_P,thsad2=ThSAD2_P,blksz=BLKSZ_P,tm=Tm_P,glob=Glob_P,bBlur=bBlur_P) : Prog
Prog = (FRAMES_P>0) ? Prog.McDeGrainSharp(frames=FRAMES_P,csharp=CSHARP_P) : Prog
Prog = (OUTPUT==2) ? Prog.ChangeFPS(Src.FrameRateNumerator*2,Src.FrameRateDenominator) : (OUTPUT==3) ? Prog.JohnFPS() : Prog # Progressive to O/P rate

SrcTest = (OUTPUT<=1) ? Src : Src.ChangeFPS(Src.FrameRateNumerator*2,Src.FrameRateDenominator)
# Select Progressive / Deinterlaced frame, based on IsCombedTIVTC detection on original source clip frame.
Result=ConditionalFilter(SrcTest, Deint, Prog, COMBDET,"=","true")

SrcOut = (!TITLE)?SrcTest:(OUTPUT<=1) ? Src.TSub("Source",TITNUM) : Src.TSub("Source DblRate via ChangeFPS, ie dupes",TITNUM).ChangeFPS(Src.FrameRateNumerator*2,Src.FrameRateDenominator)

Result = (!TITLE)?Result :(OUTPUT<=1) ? Result.TSub(String(OUTPUT,"%.0f) Result SingleRate"),TITNUM) : (OUTPUT==2) ? Result.TSub("2) Result DblRate (Progressive frames via ChangeFPS)",TITNUM)
\ : Result.TSub("3) Result DblRate (Progressive frames via JohnFPS, ie MvTools2)",TITNUM)

(STACK) ? Stackhorizontal(SrcOut, Result) : Result

(PRE_FETCH>1) ? Prefetch(PRE_FETCH) : NOP # I had problems using x64 and Prefetch [Access Violation], culprit was old IsCombedTIVTC, fixed via Pinterf version.

Return Last

#####################
# Stack Overhead Subtitle Text, with optional FrameNumber shown.
Function TSub(clip c,string Tit,Bool "ShowFrameNo",Int "Col"){
c.BlankClip(height=20,Color=Default(Col,0))
(Default(ShowFrameNo,False))?ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""))"""):Trim(0,-1).Subtitle(Tit)
Return StackVertical(c).AudioDubEx(c)
}

EDIT: Above Edited, now Doublerate, and uses TFM(pp=0) to remove perverse telecine as for Babylon5. Added in BLUE

EDIT: Dont ,miss previous post where Spotless update to v1.03.
EDIT: Also, with above script I found that is better with ThSAD2=5000, rather than default 10000.
EDIT: Perhaps good guess for ThSAD2 would be 10,000 / RadT, so
RadT=2 then 5000 : RadT=6 then 1666 : RadT=10 then 1000 : RadT=20 then 500

[FranceBB]

I just tried SpotLess and it's absolutely Spot On xD
No, seriously, it's coming in handy as I'm cleaning the 1948 Olympics. ;)

[StainlessS]

Dont tell anybody your secret, you'll be able to ask for a pay rise :)

[Frank62]

Also a big thank you to @StainlessS (and Didee), I am using it too, with a very dirty and scratchy source, and it works very well!
I agree to those who use it with truemotion=false.
Fast big things in the foreground give far less artefacts - I have no idea why, just tested.

[johnmeyer]

I decided to take SpotLessS (note my improved spelling) out for a spin, hoping that it might be a more modern alternative to RemoveDirt.

I am very encouraged by the initial trials and, if I can figure out how to tune it, it might just be a good replacement.

I did run into two problems with my "torture clip." First, SpotLessS is slow. I made no attempt to run it with multithreading enabled because some of the posts made it sound like that wasn't going to work. For me, getting it to work multi-threaded will be mandatory if I am to use it. I didn't want to waste time on that effort until I am first convinced that it is something I can really use.

The second issue was its tendency to remove far more "good things" than does RemoveDirt. Let me explain by referring to my torture clip.

Here is a single, fairly long play, from an early 1960s American football game, recorded on 24 fps B&W movie film. This is cut directly from the DVD that was supplied to me (i.e., it is as far up the food chain as I can go, since I do not have access to the film itself):

http://www.mediafire.com/file/fgtsy99n439mj6z/test+clip_02.mpg/file

This film was horribly damaged, and I spent a full day repairing it, mostly by hand, by masking in good video from adjacent frames to replace the huge chemical spots (they look like nitrate film deterioration, but this film seems too modern to have been nitrate). I was still left with lots of dirt to remove, and the RemoveDirt portion of my film restoration script did a really good job with that.

Here is the script I used to test SpotLessS:

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\Latest Version\mvtools2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MedianBlur2.dll")
source = AVISource("e:\fs.avi").ConvertToYV12(interlaced=true).killaudio()
output = SpotLess(source)
#return output

Interleave(
\ source
\ , output
\ )As I stepped through each frame using the Interleave code (which shows me the original frame followed immediately by the restored frame) I began to note that an awful lot of arms and legs were disappearing. Having the ball disappear in flight is something I have never been able to avoid with RemoveDirt, but arms and legs seldom disappear.

Then, with this particular clip, I noticed a huge number of hash marks were also disappearing. For those not familiar with American football, hash marks are the short lines on the field which are paced exactly one yards apart, using for spotting the ball for the next play. I have never seen these disappear in all the dozens of hours of football plays I've restored for a major sports film collector using my version of VideoFred's film restoration scripts to which I added RemoveDirt.

I'm hoping that both problems can be fixed by tweaking a few variables, but I am asking for just a few hints as to which variables I should focus on.

StainlessS, you may find this torture clip pretty useful because it has a LOT of things going on: motion in the frame, as well as horizontal camera motion; high contrast and low contrast dirt; massive dirt stains (some of which SpotLessS removes almost perfectly!!); and of course, "spots!!"
------------

[edit]
I'm having some success by massively reducing ThSAD2 (1000 instead of 10000). Chroma=false also helps a lot (source is B&W). BlkSz massively changes the results: 16 makes everything disappear, but 4 significantly degrades the spot removal. The sensitivity to block size is way more pronounced than I have ever seen with other MVTools2 scripts and may provide hints on what other things to tune.

[edit2]
OMG, the true motion variable is the secret: set that to false and really great things happen. Wow, wow, wow!

These are the variables that created magic:

output = SpotLess(source,ThSAD2=1000,RadT=2,Chroma=true,BlkSz=8,OLap=4,Tm=false)

Makes me wonder if I should experiment with that for other MVTools2 scripts I have.

I'll play around with this tomorrow when I'm fresher (age kills energy late in the day, I'm finding). However, as of right now, this may be the best AVISynth script or function I've used in the last year.

Way to go Mr. Lack of Stain.

[StainlessS]

SplotLess() v1.04, update. See 1st post.

I made the executive decision to make some script breaking changes, sorry.


SpotLess v1.04. temporal spot/noise remover, by StainlessS @ Doom9. https://forum.doom9.org/showthread.php?t=181777
Original idea from Didée post :- https://forum.doom9.org/showthread.php?p=1402690#post1402690
Req:- Pinterf MvTools2(), Pinterf Medianblur2() with MedianBlurTemporal rather than the MedianBlurT() from Didée post.
With appropriate plugins, will be AVS+ colorspace and x64 compatible.
Fine using Avs+ Prefetch() so long as current Pinterf plugins, and Frame Accurate source. RequestLinear() following the source filter might suffice as frame accurate source filter.

NOT FOR cartoon/anime, live video only, sorry.

v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
v1.02, Add some stuff.
v1.03, Frame size auto config of args removed (found instances where caused problems). Added Glob and bBlur args.
v1.04, Script breaking changes, I guess is more flexible if can also set ThSAD, inserted ThSAD 3rd arg. RadT now default 1, was 2, dont over denoise unless requested.

SpotLess(clip c,int "RadT"=1,int "ThSAD"=10000,int "ThSAD2"=ThSAD,int "pel"=2,bool "chroma"=true, int "BlkSz"=8,Int "Olap"=BlkSz/2,bool "tm"=true,Bool "glob"=Tm,Float "bBlur"=0.0)

RadT, 1 or more, Default 1. Removes Spots on up to RadT [Temporal Radius] consecutive frames.
RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

ThSAD, Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences].
ThSAD and ThSAD2 suggested absolute minimum of maybe about 400.
ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
in the result MCompensated frame].
Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
or some other mixture.
A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
middle pixel value.
I've just tested real bad double frame, full frame luma and chroma corruption, with below line:
SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
as we still had the temporal median filter to fall back on and pick the middle pixel value.

From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

ThSAD2, Default ThSAD, 0 < ThSAD2 < 16320(8*8*255), Lower removes fewer spots, but less chance of blurring.
ThSAD2 sets the SAD [Sum of Absolute Differences] threshold for most distant frame from current_frame at distance RadT, with those frames that are distances in-between 1 and RadT
acquiring a SAD threshold linearly interpolated between the two.
From mvtools2 docs:
ThSAD2:
Defines the SAD soft threshold for the furthest frames at current_frame +- RadT.
The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame)
and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
EDIT: Although I have said that SAD threshold being too high could result in blurred frames, that is really taken from above "risk of bluring" line from mvtools docs,
however, that warning says "temporal denoising filter", which might suggest pixel averaging, whereas we are using pixel median. I'm not sure that blurring would be the result
of having too high a SAD threshold.

Pel, Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)

Chroma, Default True. MAnalyse chroma arg. If true, use chroma in block matching when creating vectors. Maybe use False if source B&W or color noise.

BlkSz, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]

OLap, Default half BlkSz, Block overlap.

Tm, TrueMotion Default False. Some folk swear truemotion=false is better.

Glob, Default Tm, Allow set MAnalyse(global) independently of TrueMotion.
From MvTools2 docs for MAnalyse,
global
Estimate global motion (at every level) and use it as an additional predictor.
Only pan shift is estimated (no zoom and rotation).
Use false to disable, use true to enable. Default is like truemotion.

bBlur, Default 0.0 [OFF]. If used, Suggest about 0.6, where MAnalyse create vectors is performed on denoised (blurred) super clip.


EDIT: RequestLinear() following the source filter might suffice as frame accurate source filter.

[johnmeyer]

I just tried the new v1.04 on the full version of that B&W football game, but didn't see any significant differences.

I'm still getting a little too much removal of legs, etc., but I'm still trying to experiment with all the settings.

[StainlessS]

I have not as yet tried with that clip, I'm a bit knackered and probably heading for bed soon.
Maybe I should have given moniker of this filter StainLessS(), then my name would be remembered down the millennia,
me and Achilles, yes they would remember us.

EDIT: John, your torture clip is 29.976FPS with dupes, so assume 23.976 FPS.
You need to at least remove duplicates, they will mess stuff up, ie count bad frames twice and mess with median,
things look tricky enough without having same error counted multiple times.

[johnmeyer]

John, your torture clip is 29.976FPS with dupes, so assume 23.976 FPS.
You need to at least remove duplicates, they will mess stuff up, ie count bad frames twice and mess with median,
things look tricky enough without having same error counted multiple times.Let me go back and look at that. I just checked and it looks like proper 23.976 progressive without dups, but maybe something got screwed up along the way.

[edit]I just looked at the original MPEG video, as well as the clip that I losslessly extracted using Womble's MPEG Video Wizard, and both are indeed 23.976 progreesive without dups or drops. I looked at the test clip I posted in MediaInfo and it reports 23.976, with no hint of 29.97 w/ pulldown. My NLE (Vegas) also reports 23.976 and when I play the clip in Vegas, as well as when I serve from Vegas into your script, I see no evidence of mismatched frame rates.

Have another look tomorrow morning and see if something got glitched on your end.

[Boulder]

Have you tested if doing MRecalculate (or multiple recalculations) with a smaller blocksize on the vectors would help with those disappearing human parts issues?

[joka]

Spotless() is a motion compensated temporal median filter of n frames (n=2*RadT+1). In case of n=5 this is a very heavy filter that removes a lot of dirt, spots, grain, ...; but removes details and (parts of) moving objects too.
Such a filter but with n=3 (like the script proposed by Didee) is the cleaning part of RemoveDirtMC. In RemoveDirtMC the cleaning filter is followed by a motion area restoring part ("RestoreMotionBlocks") to remove the artifacts.

In my opinion spotless() is a try worth to use it as cleaning filter in RemoveDirtMC. For a stand alone usage there are too much articfacts.

However, as I understood the original intend was to remove overlapping spots on 2 consecutive frames. Spotless() can do it in no-motion areas, but the required "RestoreMotionBlocks" will restore it.

[johnmeyer]

Have you tested if doing MRecalculate (or multiple recalculations) with a smaller blocksize on the vectors would help with those disappearing human parts issues?I never thought of doing that. I've re-written quite a few MVTools2 scripts to use MRecalculate so, if time permits today I'll try that with StainlessS' code and see what happens.

I made the observation, in my earlier post, that block size had the strongest impact on the filter of any variable and this impact was far larger than I'd ever seen in other uses of MVTools2. It never occurred to me to use MRecalculate, which I normally think of as a way to get the precision of the smaller block size, but with the speed of the larger blocks, but which also, I guess, has other impacts.

[wonkey_monkey]

Spotless() is a motion compensated temporal median filter of n frames (n=2*RadT+1).

Huh. There was me skimming this thread, not really paying much attention, seeing the video and thinking "What this really needs is a motion compensated temporal median filter of, ooh, around 5 frames." Great minds think alike :cool:

[Boulder]

I never thought of doing that. I've re-written quite a few MVTools2 scripts to use MRecalculate so, if time permits today I'll try that with StainlessS' code and see what happens.

I made the observation, in my earlier post, that block size had the strongest impact on the filter of any variable and this impact was far larger than I'd ever seen in other uses of MVTools2. It never occurred to me to use MRecalculate, which I normally think of as a way to get the precision of the smaller block size, but with the speed of the larger blocks, but which also, I guess, has other impacts.
I have no idea if it does help here, but I find it stabilizes the vector field quite a bit. Maybe SSS will add an MShow-related parameter to display the vectors to help determine the correct settings.

[johnmeyer]

I was unable to get MRecalculate to work with this script. I suspect that the problem is that the "delta" variable in MAnalyze is not supported by MRecalculate. This script uses large values for Delta. Once I used MRecalculate, the resulting video was from the wrong moment in time.

[StainlessS]

John, I suspect it [Mrecalculate thingy] just needs vectors=Multivec, and tr=RadT.

MRecalculate

MRecalculate (
clip super,
clip vectors,
int thSAD (200),
int smooth (1),
int blksize,
int blksizeV,
int search,
int searchparam,
int lambda,
bool chroma,
bool truemotion,
int pnew,
int overlap,
int overlapV,
string outfile,
int dct,
int divide,
int sadx264,
bool isse,
int tr
int scaleCSAD (0)
)

Refines and recalculates motion data of previously estimated (by MAnalyse) motion vectors with different super clip or new parameters set (e.g. lesser block size), after divide, etc. The two-stage method may be also useful for more stable (robust) motion estimation. The refining is at finest hierarchical level only. Interpolated vectors of old blocks are used as predictors for new vectors, with recalculation of SAD. Only bad quality new vectors with SAD above threshold thSAD will be re-estimated by search. thSAD value is scaled to 8x8 block size. Good vectors are not changed, but their SAD will be re-calculated and updated.

You must set the tr variable if you process "multi" motion vector clips.

Parameters not described below have the same meaning as in MAnalyse, but you can use other values. Their default settings are same as in MAnalyse.

smooth

This is method for dividing coarse blocks into smaller ones.
0 Use motion of nearest block.
1 Bilinear interpolation of 4 neighbors.

tr

This is the temporal radius for motion vector clips generated by MAnalyse with multi = true. Default 0 (normal vector clip).


EDIT:
vectors is not an optional named argument for Mrecalculate, so skip the "vectors=".

MAnalyse "c[blksize]i[blksizeV]i[levels]i[search]i[searchparam]i[pelsearch]i[isb]b[lambda]i[chroma]b[delta]i[truemotion]b[lsad]i[plevel]i[global]b[pnew]i[pzero]i[pglobal]i
"[overlap]i[overlapV]i[outfile]s[dct]i[divide]i[sadx264]i[badSAD]i[badrange]i[isse]b[meander]b[temporal]b[trymany]b[multi]b[mt]b[scaleCSAD]i"
MBlockFps "cccc[num]i[den]i[mode]i[ml]i[blend]b[thSCD1]i[thSCD2]i[isse]b[planar]b[mt]b"
MCompensate "ccc[scbehavior]b[recursion]f[thSAD]i[fields]b[time]f[thSCD1]i[thSCD2]i[isse]b[planar]b[mt]b[tr]i[center]b[cclip]c[thSAD2]i"
MDegrain1 "cccc[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[mt]b[out16]b"
MDegrain2 "cccccc[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[mt]b[out16]b"
MDegrain3 "cccccccc[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[mt]b[out16]b"
MDegrain4 "cccccccccc[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[mt]b[out16]b"
MDegrain5 "cccccccccccc[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[mt]b[out16]b"
MDegrain6 "cccccccccccccc[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[mt]b[out16]b"
MDegrainN "ccci[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[thsad2]i[thsadc2]i[mt]b[out16]b"
MDepan "cc[mask]c[zoom]b[rot]b[pixaspect]f[error]f[info]b[log]s[wrong]f[zerow]f[range]i[thSCD1]i[thSCD2]i[isse]b[planar]b"
MFlow "ccc[time]f[mode]i[fields]b[thSCD1]i[thSCD2]i[isse]b[planar]b[tclip]c"
MFlowBlur "cccc[blur]f[prec]i[thSCD1]i[thSCD2]i[isse]b[planar]b"
MFlowFps "cccc[num]i[den]i[mask]i[ml]f[blend]b[thSCD1]i[thSCD2]i[isse]b[planar]b[optDebug]i"
MFlowInter "cccc[time]f[ml]f[blend]b[thSCD1]i[thSCD2]i[isse]b[planar]b[tclip]c"
MMask "cc[ml]f[gamma]f[kind]i[time]f[Ysc]i[thSCD1]i[thSCD2]i[isse]b[planar]b"

MRecalculate "cc[thsad]i[smooth]i[blksize]i[blksizeV]i[search]i[searchparam]i[lambda]i[chroma]b[truemotion]b[pnew]i[overlap]i[overlapV]i[outfile]s[dct]i[divide]i[sadx264]i[isse]b[meander]b[tr]i[mt]b[scaleCSAD]i"
^
|


MRestoreVect "c[index]i"
MScaleVect "c[scale]f[scaleV]f[mode]i[flip]b[adjustSubPel]b[bits]i"
MSCDetection "cc[Ysc]i[thSCD1]i[thSCD2]i[isse]b"
MShow "cc[scale]i[sil]i[tol]i[showsad]b[number]i[thSCD1]i[thSCD2]i[isse]b[planar]b"
MStoreVect "c+[vccs]s"
MSuper "c[hpad]i[vpad]i[pel]i[levels]i[chroma]b[sharp]i[rfilter]i[pelclip]c[isse]b[planar]b[mt]b"

[johnmeyer]

I thought that's what I did, but this code didn't work.

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
RadT = Default(RadT,2) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,10000) # Setting lower, sets SAD thresold to limit blurring effect of more distant reference frame BAD matching blocks. (MCompensate arg)
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true. Use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,Tm) # Default Tm, Allow set MAnalyse(global) independently of TrueMotion.
Bblur = Default(bblur,0.0) # Default OFF
Assert(1 <= RadT,"SpotLess: 1 <= RadT")
pad = max(BlkSz,8)
sup = (bBlur<=0.0 ? c : c.blur(bblur)).MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2)
sup_rend = (bBlur<=0.0) ? sup : c.MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2,levels=1) # Only 1 Level required where not MAnalyse-ing.
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,search=4,chroma=Chroma,truemotion=Tm,global=Glob)

MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,search=4,chroma=Chroma,truemotion=Tm,global=Glob)
MultiVec_re = MRecalculate(sup,MultiVec, chroma=Chroma,blksize=BlkSz/2, overlap=OLap/2,truemotion=Tm,tr=1)
c.MCompensate(sup_rend,MultiVec_re,tr=RadT,thSad=ThSAD,thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

[StainlessS]

MultiVec_re = MRecalculate(sup,MultiVec, chroma=Chroma,blksize=BlkSz/2, overlap=OLap/2,truemotion=Tm,tr=RadT)

[johnmeyer]

That works. Thanks!

I'll play around with the variables and see if I can come up with something which cleans as well or better than RemoveDirt, and removes the same or fewer arms and legs.

[edit]If you ever decide to play with my torture clip, this set of variables did a pretty good job of cleaning the basic dirt (forget about the chemical stains because they are too big to be dealt with automatically). It also didn't completely amputate arms and legs, although socks and wristbands sometimes disappear. As long as that doesn't happen too often, most people won't notice.

This is getting tantalizingly close to being one of the truly great scripts.

SpotLess(source,ThSAD=5000,ThSAD2=5000,RadT=1,Chroma=false,BlkSz=8,OLap=4,Tm=false)

[StainlessS]

I will have a play with your clip, I'm just about done recovering a working linux after moving Linux / Windows onto SSHD (Solid State Hybrid Drive), it killed/crippled my setup for some days [well a week].
I had to look, I'm currently on Linux.

EDIT: This,

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
RadT = Default(RadT,2) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,10000) # Setting lower, sets SAD thresold to limit blurring effect of more distant reference frame BAD matching blocks. (MCompensate arg)


Should be

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # Setting lower, sets SAD thresold to limit blurring effect of more distant reference frame BAD matching blocks. (MCompensate arg)

[johnmeyer]

Yeah, changing RadT to 1 is what made it behave the way I expected. Of course RadT>1 is part of what makes the script so powerful for other applications, like when you removed that massive color cast, so I'm sure I'll be using larger numbers for other applications.

[FranceBB]

And this is where things went completely wrong.
Sadly those athletes were all black and SpotlessS clearly thought they were big giant spots and tried its best to remove them xD

https://i.imgur.com/sSBKO4g.png
https://i.imgur.com/MzpZr9S.png
https://i.imgur.com/FM0ocbw.png


StainlessS, I think your filter might be bit racist xD

[kedautinh12]

And this is where things went completely wrong.
Sadly those athletes were all black and SpotlessS clearly thought they were big giant spots and tried its best to remove them xD

https://i.imgur.com/sSBKO4g.png
https://i.imgur.com/MzpZr9S.png
https://i.imgur.com/FM0ocbw.png


StainlessS, I think your filter might be bit racist xD

I think they are ghost :) :) :)

[StainlessS]

StainlessS, I think your filter might be bit racist

Yeh well in this crazy mixed up PC (EDIT: Black Legs Matter) world, you gotta have a bit of balance somewhere.

EDIT: Although the guy in yellow at the back seems to have lost a bit more than his legs.

[johnmeyer]

And this is where things went completely wrong.What settings did you use? Without that information, your observations don't mean much.

I too noticed the same thing (limb amputation), and posted about it earlier in this thread. I then spent some time to learn and understand the filter, and found settings that completely avoid the disappearance of arms and legs that you posted.

Posting that, without providing any information about settings is exactly like complaining that a noise reduction filter eliminates all detail after you turn all the settings to "max." Such an observation does a disservice to the people reading the thread who may think that the filter/plugin has no utility and is not worth much.

SpotlessS is most definitely a very worthwhile script.

[FranceBB]

I know, I wasn't complaining, I was just pointing out that I had the very same issues you had on some scenes, that's all.
Anyway, I'll provide the script if it helps others.

[johnmeyer]

I know, I wasn't complaining, I was just pointing out that I had the very same issues you had on some scenes, that's all.
Anyway, I'll provide the script if it helps others.Got it. Thanks!

[StainlessS]

Just noticed, mistake in docs [fixed]

Tm, TrueMotion Default True. Some folk swear truemotion=false is better. # Was False

[johnmeyer]

I was able to eliminate a lot of the rainbows in the clip provided in a thread over at Videohelp. This post shows the SpotlessS script I used, along with one "before/after" snapshot:

https://forum.videohelp.com/threads/398864-Need-help-on-Avisynth?p=2595644&viewfull=1#post2595644

[kedautinh12]

I was able to eliminate a lot of the rainbows in the clip provided in a thread over at Videohelp. This post shows the SpotlessS script I used, along with one "before/after" snapshot:

https://forum.videohelp.com/threads/398864-Need-help-on-Avisynth?p=2595644&viewfull=1#post2595644

Can you try ASTDR derainbow??
https://github.com/realfinder/AVS-Stuff/blob/master/avs%202.5%20and%20up/ASTDR.avsi

[johnmeyer]

Can you try ASTDR derainbow??
https://github.com/realfinder/AVS-Stuff/blob/master/avs%202.5%20and%20up/ASTDR.avsiI'll try to remember to do that when I get back to my office computer tomorrow. I really only wanted to kill the rainbows, but not do any denoising, and SpotlessS uses a function which desnoises as well as averages. I'll let you know if it works.

[StainlessS]

John, you posted on VideoHelp, non current script,


RadT = Default(RadT,2) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,10000) # Setting lower, sets SAD thresold to limit blurring effect of more distant reference frame BAD matching blocks. (MCompensate arg)


should be (and for good reason)

RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # Setting lower, sets SAD thresold to limit blurring effect of more distant reference frame BAD matching blocks. (MCompensate arg)


Changing ThSAD also changes default ThSAD2.

We dont really want gazllions of versions of the function floating about, there are already more than enough official ones.

[johnmeyer]

John, you posted on VideoHelp, non current script, <snip>

We dont really want gazllions of versions of the function floating about, there are already more than enough official ones.Sorry about that. I don't think it will make any difference, since I fed my own numbers into the script so the new presets in your current version will be overridden.

[StainlessS]

Update SpotLess(), v1.06, see 1st post.


v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
v1.02, Add some stuff.
v1.03, Frame size auto config of args removed (found instances where caused problems). Added Glob and bBlur args.
v1.04, Script breaking changes, I guess is more flexible if can also set ThSAD, inserted ThSAD 3rd arg. RadT now default 1, was 2, dont over denoise unless requested.
v1.05, Additional checks on args.
v1.06, Glob Default true, Almost always better.

[StainlessS]

Dont know if this is of any use at all (I usually like to get rid of interlacing).


Function SpotLessI(clip c,String "Bobber",int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
/*
SpotLess use on Interlaced material.
Input requires correct parity, use AssumeTFF/AssumeBFF before use.
RadT etc applied to Bobbed fields rather than frames, Might want to use at least RadT=2 (default is 1).
*/
c
Bobber = Default(Bobber,"Bob") # Your Favourite Bob deinterlacer whotsit
Last.Eval(Bobber)
SpotLess(RadT,ThSAD,ThSAD2,pel,chroma,BlkSz,Olap,tm,glob,bBlur)
SeparateFields.SelectEvery(4,0,3).Weave
Return Last
}


Have I done it correctly, any comments and testing welcome.
If is of value, I'll add to main script, otherwise if no response I'll give it a miss.

EDIT: Or any suggestions on single/doublerate use (maybe skip last step SeparateFields.weave).
EDIT: Or could use above as a template.
EDIT: Possibly of best use where original clip was progressive with noise before artificial conversion to Interlaced.

EDIT: Alternative.

Function SpotLess2i2(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",Float "bBlur") {
/*
SpotLess use on Interlaced material, fields filtered separately.
Input requires correct parity, use AssumeTFF/AssumeBFF before use.
*/
c
e=Separatefields.SelectEven.AssumeFrameBased.SpotLess(RadT,ThSAD,ThSAD2,pel,chroma,BlkSz,Olap,tm,glob,bBlur)
o=Separatefields.SelectOdd.AssumeFrameBased.SpotLess(RadT,ThSAD,ThSAD2,pel,chroma,BlkSz,Olap,tm,glob,bBlur)
Return interleave(e.AssumeFieldBased,o.AssumeFieldBased).Weave
}