SpotLess - Temporal denoise

[FranceBB]

Thanks for the insight, I'm gonna try Delta Restore.

If Spotless is the bread, Delta Restore is the knife, zorr is the butter and Stainless is Marmite

[StainlessS]

Zorr,
I note that in a number of posts in spotless related thread
eg here:- https://forum.doom9.org/showthread.p...87#post1944487

 function RemoveSmallSpots(clip mask, int spotSize) {
	mask_orig = mask

	# remove too small spots ----------------------------------------------------
	for (i = 1, spotSize) {
		mask = mask.mt_inpand()
	}
	mask = mt_hysteresis(mask, mask_orig, chroma="-127") 
	return mask
}

think maybe above chroma="-127" should be chroma="-128",
Appears quite a few times in the linked thread.

From Masktools 2 original docs.

When defined, the value contained in this string will overwrite the u & v processing modes. This is a nice addition proposed by mg262 that makes the filter more user friendly. Allowed values for chroma are :

"process" : set u = v = 3.
"copy" or "copy first" : set u = v = 2.
"copy second" : set u = v = 4.
"copy third" : set u = v = 5.
"xxx", where xxx is a number : set u = v = -xxx.

EDIT: I recently found a script posted some time ago that now throws an error in recent Pinterf MaskTools, I had wrongly used
chroma="128" [instead of chroma="-128"]
and it now throws a wobbly, says should be setting chroma to -ve ie -128 [although original script did seem to work OK in older masktools].

EDIT: To below: Thankyou kind sir [you've gotta develop thick skin on the D9]

[zorr]

Originally Posted by StainlessS

Zorr,
I note that in a number of posts in spotless related thread
...
think maybe above chroma="-127" should be chroma="-128",

Yes you're right. I'll fix it when I release the "official" Delta Restore version. It doesn't have any effect on the final result so it's nothing to worry about.

[coolgit]

Many times I had spots around the edges of the frames and always put back in by delta restore.

Check the coding and zpadding seems to be responsible for it. Don't know why it was there but I had changed it to zeros.

Spots now removed. Yippeeeee.

[zorr]

Originally Posted by coolgit

Many times I had spots around the edges of the frames and always put back in by delta restore.

Check the coding and zpadding seems to be responsible for it. Don't know why it was there but I had changed it to zeros.

Spots now removed. Yippeeeee.

Zpadding was used in the original case to restrict the edge detector from detecting the frame borders (the source was scanned film). For most material Zpadding is not needed.

[coolgit]

Hey StainlessS, what does olap do?

Had a big defect and was experimenting. Nothing worked 100% even when BlkSz,32. For a laugh I changed olap to BlkSz/3 and boooom, it work 100%.

[johnmeyer]

Originally Posted by coolgit

Hey StainlessS, what does olap do?

It is the overlap parameter used by the MAnalyze function of MVTools2, which is the motion estimation plugin that is at the heart of Spotless. In MVTools2 you can specify the size of the block used to "track" objects in the video. Typical sizes are 4, 8, and 16. To improve the quality of certain operations the algorithm can overlap adjacent blocks. This sometimes improves noise reduction or synthesis of intermediate frames, but you don't know until you try, as you did in fact do.

[StainlessS]

OLap is the block [Blksz] Overlap.
Popular choices are sometimes BlkSz/2 or 2.

[MysteryX]

lol I'm trying Spotless on a clip while driving, and it's doing a slightly better job than VS mClean at removing noise (at higher speed!). And -- trees while driving are disappearing. Can I call it Tree-less?

[StainlessS]

lol I'm trying Spotless on a clip while driving, and it's doing a slightly better job than VS mClean at removing noise (at higher speed!). And -- trees while driving are disappearing. Can I call it Tree-less?

You can indeed call it TreeLessS,
However I have to point out that it is very dangerous to try out Spotless whilst you are driving.

EDIT: Especially at higher speed!

EDIT: Zorr added some recovery stuff for disappearing trees, arms, legs, balls, etc, here:- https://forum.doom9.org/showthread.php?t=182831

[MysteryX]

That ain't faster anymore. Waiting half-an-hour for the first frame of a 5K clip to come out. I give up.

Another way to restore details: run this as a prefilter to BM3D.

Edit: LOL it was resizing up by a factor of 3x

[MysteryX]

I'm comparing this to xClean. I'm very surprised by the results. Trees are back but....... look different. They actually look more like trees than in the source??? How is that possible? Image details are also a lot clearer. This is looking damn good. It's just that -- the way it completely alters moving trees is a bit risqué

Runs at 0.4fps on 5K

Compared to xClean, noise is about the same, image is clearer/sharper, and moving objects are... transformed, for better or worse. Haven't tested whether it works well for all types of content. On this test clip, Spotless gives better visible result (perhaps a bit too sharp), while xClean better respects the source.

[coolgit]

Originally Posted by johnmeyer

It is the overlap parameter used by the MAnalyze function of MVTools2, which is the motion estimation plugin that is at the heart of Spotless. In MVTools2 you can specify the size of the block used to "track" objects in the video. Typical sizes are 4, 8, and 16. To improve the quality of certain operations the algorithm can overlap adjacent blocks. This sometimes improves noise reduction or synthesis of intermediate frames, but you don't know until you try, as you did in fact do.

Is there an image example somewhere explaining how it works?

Are you saying that if a block is currently analyse the overlap can do the next block to exist the current block.

How does BlkSz/3 do a better job than BlkSz/2 - assuming / means divide.

[MysteryX]

Originally Posted by coolgit

Is there an image example somewhere explaining how it works?

Are you saying that if a block is currently analyse the overlap can do the next block to exist the current block.

How does BlkSz/3 do a better job than BlkSz/2 - assuming / means divide.

For blksize=16, if you set an overlap of 8, half-way through each block begins the next block.

[johnmeyer]

Originally Posted by coolgit

Is there an image example somewhere explaining how it works?

Are you saying that if a block is currently analyse the overlap can do the next block to exist the current block.

How does BlkSz/3 do a better job than BlkSz/2 - assuming / means divide.

http://avisynth.org.ru/mvtools/mvtools2.html

https://forum.doom9.org/showthread.p...686#post564686

Search them both for "overlap."

[StainlessS]

Assuming just 8 bit depth,
I nearly always avoid LSMash and ffms2, too many probs.
Maybe this of use [in a demo format]

DEMO.BAT

REM https://forum.doom9.org/showthread.php?p=1908619#post1908619
REM We DO NOT LIKE SPACES IN FILE NAMES (REM == REMark ie comment)
REM We DO NOT LIKE ACCENTS IN FILE NAMES.


setlocal

REM Where to Find ffmpeg
set FFMPEG="C:\BIN\ffmpeg.exe"

REM Where to get INPUT files, No terminating Backslash, "." = current directory (ie same as dir .bat file)
set INDIR="."

REM Where to place OUTPUT files, No terminating Backslash. "." would be same as .bat file, ".\OUT" = OUT folder in same directory as bat file.
set OUTDIR="D:"

REM Below, can add INPUT extensions as eg *.WEBM (SPACE separated, Batch Processes all INPUT type files in INDIR)
FOR %%A IN (*.AVI *.MKV *.MP4 *.MOV *.QT *.3GP *.DVB *.VOB *.MPG *.MPEG *M2P *.PS *.TS *.DIVX *.XVID *.FLV *.WMV *.ASF *.MXF) DO (

REM ****** Un-REM [ie SELECT] ONLY one of below lines, Comment out ie REM the remaining lines.
    %FFMPEG% -i "%INDIR%\%%A" -vcodec utvideo  -acodec pcm_s16le       "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec huffyuv  -acodec pcm_s16le       "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec magicyuv -acodec pcm_s16le       "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec v410     -acodec pcm_s16le       "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec r210     -acodec pcm_s16le       "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec copy     -acodec copy      -sn   "%OUTDIR%\%%~nxA.MKV"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec utvideo  -acodec copy            "%OUTDIR%\%%~nxA.MKV"
REM %FFMPEG% -i "%INDIR%\%%A" -vn              -acodec pcm_s16le       "%OUTDIR%\%%~nxA.WAV"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec rawvideo -f rawvideo  -an        "%OUTDIR%\%%~nxA.rawvideo"


REM *****************************************************************************************.

)
Pause

REM ... Above Command lines, What they do if UnREM'ed (and all others REM'ed, UnREM=UnCOMMENT ) :
REM      (1) Convert Video to UtVideo  lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM      (2) Convert Video to HuffYUV  lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM      (3) Convert Video to MagicYUV lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM      (4) Convert Video to v410 Uncompressed 4:4:4 10-bit lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM      (5) Convert Video to r210 Uncompressed RGB 10-bit lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM      (6) Remux NO SUBTITLES, copy both video and audio but NOT subtitles(output MKV - see file extension at the end of the Remux line, ie MKV).
REM      (7) Convert Video to UtVideo lossless, copy audio (output MKV).
REM      (8) Skip any video, Convert Audio to 16 bit PCM (output WAV).
REM      (9) Convert Video to rawvideo uncompressed, Convert Audio to 16 bit PCM audio (output y4m).
REM *****************************************************************************************.
REM    In the UnREM'ed command Line [ie without a preceding REM]:-
REM      '-vcodec utvideo' means convert video using utvideo codec.
REM      '-vcodec copy' means copy video rather than convert.
REM      '-vn' means no video output, use instead of eg '-vcodec utvideo'
REM      '-acodec pcm_s16le' means convert audio using pcm_s16le codec.
REM      '-acodec copy' means copy audio rather than convert.
REM      '-an' means no audio output, use instead of eg '-acodec pcm_s16le'
REM      The file extension at end of the line determines output container, eg '.AVI'
REM
REM      From Command line with ffmpeg somewhere in your environment PATH:-
REM        'ffmpeg -codecs >D:\ffmpeg_codecs.txt'
REM          Writes a txt file of ffmpeg available codecs to D:\
REM          At start of txt file it shows a 'legend' or 'key' for codecs that can be used for video and audio,
REM          use only ENCODING SUPPORTED codecs, prefereably LOSSLESS.
REM          Supported codecs will vary with ffmpeg version.
REM *****************************************************************************************.

I have a dedicated conversion directory where the script lives.
Copy video files to that directory.
Click the bat, batch converts all in directory.
Job done. Use reliable AviSource for the lot.

[MysteryX]

I had accidently posted the ffsm2 issue in the wrong thread; but since you answered... not sure converting 5K video to UT Video is a great idea. It would need enormous space!

Edit: for a 94MB video of 12 seconds, it produces a 2.2GB video and lists these errors:
- Using non-standard frame rate 29/1
- deprecated pixel format used, make sure you did set range correctly
- Duplicate POC in a sequence
- Error parsing NAL unit #0
- Error while decoding stream #0:0: Invalid data found when processing input

Video is only missing the last frame and colours are wrong. lol.

[StainlessS]

Maybe, but they would work without problems.

I think GoPro would be mp4, just use LSmashVideoSource, that one is usually fine.

[MysteryX]

Originally Posted by StainlessS

Maybe, but they would work without problems.

I think GoPro would be mp4, just use LSmashVideoSource, that one is usually fine.

I think I'll just denoise the videos and let my video editor fix remaining glitches. Next time, I'll encode in h264 instead of h265, with High-Bit-Depth enabled, and re-encode every clip while transferring to my computer at a desirable bitrate. I hope they have less bug in their H264 encoder.

Also taking GoPro pictures -- need to set sharpening to LOW, and then add sharpening and contrast with Photoshop and it then looks as good or better than my cellphone. GoPro is peculiar to use...

[MysteryX]

I tried Spotless+DR on other clips. As I suspected, it works great on some content and not so great on other types (by the nature of MVTools). Main issue is that MVTools blurs out what I call "fine textures", which is why xClean passes MVTools through BM3D which restores the textures; and then through KNLMeans which always improves upon the prefilter.

Other than lost textures, Spotless has nearly identical noise reduction as xClean. Sharpening is too high!