SpotLess - Temporal denoise

[coolgit]

After a few days of experiments, I thought I will post my findings.

Sometimes I used

Despot(mthres=16,p1=24,p2=12,pwidth=4,pheight=4,ranked=true,sign=-1,maxpts=0,p1percent=10,show=0)

to remove small white star(ish) shaped defects. It worked to a point and fast too. However when you have outdoors scenes with trees, bushes, grass, etc, then the small white areas in the trees, for example, get removed and the trees will now flicker insanely when played back. Spotless doesn't have this problem because it use previous and next frames in its calculation.

The experiment is to get Spotless to do what DeSpot does.

The answer is

SpotLess(BlkSz=8, OLap=8/2, pel=1, Tm=False, Bblur=0.0, ThSAD=200, RadT=1)

Yes ThSAD is 200 and it is not a typing error ok. You can go as low as 150 for

Despot(mthres=16,p1=24,p2=12,pwidth=3,pheight=3,ranked=true,sign=-1,maxpts=0,p1percent=10,show=0)

StainLesss, is it possible to add a 'sign' syntax like Despot has. Currently DeSpot 'sign' syntax is

sign = 0 - any spots and outliers (default)
sign = 1 - only black (dark) spots and outliers
sign = -1 - only white (light) spots and outliers
sign = 2 - only black (dark) spots, any outliers
sign = -2 - only white (light) spots, any outliers

Just curious if white or black only options, and white and black option, totalling 3 options. The rest of the colours are ignored since I have never come across coloured star(ish) shaped defects that flickers.

[StainlessS]

I would not really like to include that in SpotLess() function,
best I can suggest is make some kind of mask by combining source chroma close to 128, and luma hi/low, and use only results
of Spotless where masks comply with required.

There are others here way better than I at masking stuff, perhaps someone else can assist on that,
I also got my hands quite full with non AVS related stuff.

[coolgit]

What about doing luma or chroma only.
The thing is when i used Spotless to mimic DeSpot, it takes 30 mins extra to compress for an hour video clip.
I just thought if it does luma only then it would be faster.

[ChaosKing]

Try Chroma = false
Have you tried spotless with Prefetch() ?

[coolgit]

Prefetch yes as always.

Not sure what to do with chroma=false?

I am looking for a filter that separates luma and chroma, then i could use Spotless on luma, then merge new luma with chroma.

[coolgit]

Source whatever...

Luma=SpotLess(src, BlkSz=8, OLap=8/2, pel=1, Tm=False, Bblur=0.0, ThSAD=900, RadT=1)
MergeLuma(src, Luma)
StackHorizontal(src,last)

Tried the above and seems to work. ThSAD needs to be 800 to 1000 to do the same as SpotLess(BlkSz=8, OLap=8/2, pel=1, Tm=False, Bblur=0.0, ThSAD=200, RadT=1).

Testing to see if it will work faster.

EDIT: Much later... The above took 2 minutes longer than SpotLess(BlkSz=8, OLap=8/2, pel=1, Tm=False, Bblur=0.0, ThSAD=200, RadT=1). Crap idea.

[Mounir]

converttoyv12(matrix="rec601",interlaced=true) #
separatefields()
input=last
U = UToY().SpotLess() #
V=VToY().SpotLess() #
YToUV(U,V,last)
weave()

[coolgit]

Originally Posted by Mounir

converttoyv12(matrix="rec601",interlaced=true) #
separatefields()
input=last
U = UToY().SpotLess() #
V=VToY().SpotLess() #
YToUV(U,V,last)
weave()

Tried it and didn't work.

[coolgit]

Converting image to greyscale doesn't reduce Spotless processing time. DeSpot works differently and i am not sure why.
Despot took 1 minute extra to compressed whereas SpotLess took 30 to 32 minutes extra.

[Lathe]

Originally Posted by StainlessS

I would not really like to include that in SpotLess() function,
best I can suggest is make some kind of mask by combining source chroma close to 128, and luma hi/low, and use only results
of Spotless where masks comply with required.

There are others here way better than I at masking stuff, perhaps someone else can assist on that,
I also got my hands quite full with non AVS related stuff.

Hi there! I'm kind of a newbie and I'm trying to employ your SpotLess script with 32bit Avisynth 2.6. I have your dll and the two support dll's that I believe were mentioned awhile back (mvtools2 and MedianBlur2) I tried running this script and I got a really weird error both when trying to preview with AVSMod and actually run the x264 CMD line. This is the script I was trying to run:

DirectShowSource("GC.mkv")
Import("C:\Program Files (x86)\AviSynth\plugins\Spotless.avsi")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\MSharpen.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\MedianBlur2.dll")
assumeTFF()
TFM().TDecimate()
crop(24,12,-16,-20) #crop the crud off
Spotless()
MSharpen()
#greyscale() #convert to B&W
addborders(20,16,20,16) #pad to 720x480

This was the error the preview showed:

Traceback (most recent call last):
File "avsp.pyo", line 9119, in OnMenuVideoPlay
File "avsp.pyo", line 14672, in PlayPauseVideo
File "avsp.pyo", line 13855, in ShowVideoFrame
File "avisynth.pyo", line 462, in GetFrame
WindowsError: exception: access violation reading 0xFFFFFFFF

I really appreciate any help because I would LOVE to try your script if I can get it to run, thanks!

[Sharc]

Lathe, please don't hijack and litter this thread as well. The problem is not with Spotless(), it's a problem between your chair and your keyboard. See all your other posts here and over at the VideoHelp forum. You call yourself a newbie with 1077 posts at doom9 in almost 20 years? Start with learning the basics of video and AviSynth. Sorry to be a bit harsh

[Arx1meD]

Lathe, try opening your video with L-SMASH-Works. DirectShowSource is often buggy.

[coolgit]

Originally Posted by Lathe

crop(24,12,-16,-20) #crop the crud off
Spotless()

Spotless()
crop(24,12,-16,-20) #crop the crud off

[Bluedraft]

Hi, it's possible to remove the vertical lines and spots from this video using spotless? I tried with the default setting and using RadT=2 but could not eliminate them

sample

https://drive.google.com/file/d/1ybNWoe2Td_EvRTOdvQxcfMEoG54MpUjf/view?usp=drive_link

[johnmeyer]

Probably not vertical lines.

Couldn't access your Google Drive. You need to make the link public.

[Bluedraft]

Sample

https://mega.nz/file/Y8oQyK5a#Y14zTh259pxceNMqcLQx5dvMN_afcdWe-Ywnb1Are6Q

[Selur]

" I tried with the default setting and using RadT=2 "
For me, SpotLess seems to remove those scratches (works in Avisynth and Vapoursynth)
Script used:

ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\LSMASHSource.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\mvtools2.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\MedianBlur2.dll")
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\RgTools.dll")
Import("F:\Hybrid\64bit\Avisynth\avisynthPlugins\mtmodes.avsi")
Import("F:\Hybrid\64bit\Avisynth\avisynthPlugins\SpotLess.avs")
# loading source: C:\Users\Selur\Desktop\sample.mkv
# color sampling YV12@8, matrix: bt709, scantyp: progressive, luminance scale: limited
LWLibavVideoSource("C:\Users\Selur\Desktop\sample.mkv",cache=false,format="YUV420P8", prefer_hw=0,repeat=true)
# current resolution: 1920x1080
# despotting using SpotLess
SpotLess()
# Converting from 8 to 10bit for encoder
ConvertBits(10)
# setting output fps to 24.000fps
AssumeFPS(24,1)
PreFetch(16)
#  output: color sampling YV12@10, matrix: bt709, scantyp: progressive, luminance scale: limited
return last

[Bluedraft]

Wonderful! I'll try it! Thanks a lot Selur!

[Bluedraft]

You seems to be using spotless by default, and looking at the pictures you upload it it works perfectly, I'm doing the same but from staxrip and using FFVideoSource to open the file instead LSMASHSource (because otherwise I get error) and it's like spotless was not being applied, the scratches still there.

SetFilterMTMode("DEFAULT_MT_MODE", 2)
tcFile = "%temp_file%_timestamps.txt" # timestamps file path
Exist(tcFile) ? FFVideoSource("%source_file%", cachefile="%source_temp_file%.ffindex", timecodes=tcFile) : FFVideoSource("%source_file%", cachefile="%source_temp_file%.ffindex")
LoadPlugin("C:\Users\Bluedraft\Downloads\Portables\StaxRip 2.12\Apps\Plugins\AVS\MVTools2\mvtools2.dll")
LoadPlugin("C:\Users\Bluedraft\Downloads\Portables\StaxRip 2.12\Apps\Plugins\AVS\MedianBlur2\MedianBlur2.dll")
LoadPlugin("C:\Users\Bluedraft\Downloads\Portables\StaxRip 2.12\Apps\Plugins\AVS\RgTools\RgTools.dll")
Import("C:\Users\Bluedraft\Downloads\Portables\StaxRip 2.12\Apps\Plugins\AVS\Spotless\Spotless.avsi")
Spotless()

PreFetch(16)

What I'm doing wrong?

[Selur]

No, clue, seems fine to me, but I have never used StaxRip.
SpotLess script I use: https://pastebin.com/2UDMcJw5