TemporalDegrainV2

[StainlessS]

Blurring (I believe) occurs where vectors are not so good and done to hide problems/errors.
This script function deliberately blurs bad areas and sharpens good areas, I use it one helluva lot, for almost every encode.
McDegrainSharp:- https://forum.doom9.org/showthread.p...45#post1737045

EDIT: I usually use Frames=1(default, EDIT: well usually all defaulted).

EDIT:

Function MCDegrainSharp(clip c, int "frames", float "bblur", float "csharp", bool "bsrch",bool "Precise") {
  # From:- http://forum.doom9.org/showthread.php?p=1737045#post1737045
  # Based on MCDegrain By Didee, http://forum.doom9.org/showthread.php?t=161594
  # Also based on DiDee observations in this thread: http://forum.doom9.org/showthread.php?t=161580 
  # "Denoise with MDegrainX, do slight sharpening where motionmatch is good, do slight blurring where motionmatch is bad"
  # In areas where MAnalyse cannot find good matches, the blur() will be dominant.
  # In areas where good matches are found, the sharpen()'ed pixels will overweight the blur()'ed pixels
  # when the pixel averaging is performed.
  #

[gispos]

The code uses Undot() but is not specified in the required plugins.
Is there an x64 version of Undot.dll?
I just deleted the line with Undot()

thanks

[ErazorTT]

@gispos

yes, right thanks for spotting that!
I know that there is a 64bits undot.dll coming with the 64bit MeGui. Which you can get here. But I do not know where they have that from, I would guess they have compiled it themselves. Or does someone know where to get a 64bit undot.dll from?
Anyways, I have put the call to undot after a check that this function actually exists using FunctionExists. Checkout the newest version 2.0.2.

[ChaosKing]

Undot == RemoveGrain(mode=1), just replace it

[ErazorTT]

@ChaosKing
that was quick and easy, great!

==> checkout new version 2.0.3

[hello_hello]

Originally Posted by StainlessS

Blurring (I believe) occurs where vectors are not so good and done to hide problems/errors.
This script function deliberately blurs bad areas and sharpens good areas, I use it one helluva lot, for almost every encode.
McDegrainSharp:- https://forum.doom9.org/showthread.p...45#post1737045

I added the TrueMotion argument to MCDegrainSharp where appropriate so I could compare the result with Truemotion enabled and disabled. I don't know if other arguments should have different defaults when it's disabled, but it did make a difference. Not so much for the first frame from post #9, probably because the McDegrainSharp defaults don't remove as much noise as TemporalDegrain does by default, however for the third frame disabling TrueMotion seems like an improvement.

Original



MCDegrainSharp(TrueMotion=false)



MCDegrainSharp(TrueMotion=true)


As a side note, I assume not too many people are using the FastDegrain script. The version here (there's a link for it at the bottom of the TemporalDegrain page), has an incorrect list of required plugins, the "overshoot=oshot" argument for the LimitedSharpenFaster function should be "overshoot=oshoot", and the MDegrain functions don't include the thSAD option.
The only other versions of the script I could find are here and they're the same.

[StainlessS]

disabling TrueMotion seems like an improvement.

That is a surprise, TrueMotion = true has been MvTools default since v1.4.10.
As TrueMotion is a group setting for some other args, might be interesting to see what Zorr's Avisynth Optimizer thing makes of it (altering individual args).
TemporalDegrain perhaps has higher frame range than McDegrainSharp which uses only 1 frame range by default[EDIT: Actually 2], I like only
a light touch and so am usually happy with single frame range.

YouTube DarkHorse musvid clip is quite amazing video, but adjusting levels to produce 16->235 (think was maybe ~16->255), and applying
McDegrainSharp(frames=1) and NonlinUSM(z=3, pow=1.1, str=0.25, rad=9) or thereabouts produced a clip that knocked the socks off
the YT effort, but demands about 50% higher bitrate than original. (I think I may also have added a very light FFT3DFilter(sigma=1.5) or thereabouts after
the degrain).

Wonder how TemporalDegrain would fair in comparison.

[ErazorTT]

Originally Posted by StainlessS

That is a surprise, TrueMotion = true has been MvTools default since v1.4.10.

In QTGMC it is however disabled by default. so perhaps due to the same reason...

[Groucho2004]

Originally Posted by StainlessS

That is a surprise, TrueMotion = true has been MvTools default since v1.4.10.

Didée apparently was never a big fan of this parameter in a denoising context.

[StainlessS]

And would HD resolution in this case be as anything with width > 1100 or height > 600 as sort of agreed upon in another thread.
Any argument there.

EDIT: Changed '>' to '>=' above.
EDIT: NOPE, had it right the first time, width > 1100 or height > 600

EDIT: And larger block size be about 24 and above.

[hello_hello]

Originally Posted by StainlessS

That is a surprise, TrueMotion = true has been MvTools default since v1.4.10.

Best as I can tell QTGMC uses the default values for options changed by TrueMotion when TrueMotion=true.
When TrueMotion=false two options are set to values other than the MVTools2 defaults. They're lambda and pnew.

I added them to the MCDegrainSharp script but the difference is quite small, and after looking at a bunch of frames I decided the MVTools2 defaults were probably better, at least for MCDegrainSharp. I'll try the same with TemporalDegrain2 later.

Anyhow... here's a version of MCDegrainSharp with TrueMotion disabled. The lambda and pnew defaults when TrueMotion=false are easy to change if you want to try them yourself.

# ========== MCDegrainSharp =====================================================
#
#  From: http://forum.doom9.org/showthread.php?p=1737045#post1737045
#  Based on MCDegrain By Didee:
#  http://forum.doom9.org/showthread.php?t=161594
#  Also based on Didee observations in this thread:
#  http://forum.doom9.org/showthread.php?t=161580 
#  Denoise with MDegrainX, do slight sharpening where motionmatch is good,
#  do slight blurring where motionmatch is bad.
#  In areas where MAnalyse cannot find good matches, the blur() will be dominant.
#  In areas where good matches are found, the sharpen()'ed pixels will
#  overweight the blur()'ed pixels when the pixel averaging is performed.
#
#  3 Sept 2015 - Mod by StainlessS to add Precise.
#  24 Oct 2018 - Mod by Hello_Hello to disable TrueMotion by default.
#  https://forum.doom9.org/showthread.php?p=1855907#post1855907
#
# ===============================================================================

function MCDegrainSharp(clip c, int "frames", float "bblur", float "csharp", bool "bsrch", bool "precise", \
bool "truemotion", int "lambda", int "pnew")     {

frames = default(frames, 2)
bblur = default(bblur, 0.6)
csharp = default(csharp, 0.6)
bsrch = default(bsrch, true)
precise = default(precise, false)  # Use MRecalculate
truemotion = default(truemotion, false)
bs = (c.width() > 960) ? 16 : 8 
c2 = c.blur(bblur)

# Uncommenting the two lines below results in the same truemotion settings as QTGMC when truemotion=false
# lambda = default(lambda, (truemotion ? 1000 : 100 ) * BSize * BSize / 64)
# pnew = default(pnew, truemotion ? 50 : 25)

# Only 1 level required for sharpened Super (not MAnalyse-ing)
super = (frames > 0) ? (bsrch ? c2.MSuper(pel=2, sharp=1) : c.MSuper(pel=2, sharp=1)) : nop()
super_rend = (frames > 0) ? (c.sharpen(csharp).MSuper(pel=2, sharp=1, levels=1)) : nop()

bvec3 = \
(frames > 2) ? MAnalyse(super, isb=true, delta=3, blksize=bs, overlap=bs/2, \
truemotion=truemotion, lambda=lambda, pnew=pnew) : nop()
bvec2 = \
(frames > 1) ? MAnalyse(super, isb=true, delta=2, blksize=bs, overlap=bs/2, \
truemotion=truemotion, lambda=lambda, pnew=pnew) : nop()
bvec1 = \
(frames > 0) ? MAnalyse(super, isb=true, delta=1, blksize=bs, overlap=bs/2, \
truemotion=truemotion, lambda=lambda, pnew=pnew) : nop()
fvec1 = \
(frames > 0) ? MAnalyse(super, isb=false, delta=1, blksize=bs, overlap=bs/2, \
truemotion=truemotion, lambda=lambda, pnew=pnew) : nop()
fvec2 = \
(frames > 1) ? MAnalyse(super, isb=false, delta=2, blksize=bs, overlap=bs/2, \
truemotion=truemotion, lambda=lambda, pnew=pnew) : nop()
fvec3 = \
(frames > 2) ? MAnalyse(super, isb=false, delta=3, blksize=bs, overlap=bs/2, \
truemotion=truemotion, lambda=lambda, pnew=pnew) : nop()

# If Precise, then recalculate on Prefiltered (blurred) Super (NOT the sharpened render super)

bvec3 = \
(frames > 2) ? (!Precise ? bvec3 : \
MRecalculate(super, bvec3, blksize=bs/2, overlap=bs/4, thSAD=100, \
truemotion=truemotion, lambda=lambda, pnew=pnew)) : nop()
bvec2 = \
(frames > 1) ? (!Precise ? bvec2 : \
MRecalculate(super, bvec2, blksize=bs/2, overlap=bs/4, thSAD=100, \
truemotion=truemotion, lambda=lambda, pnew=pnew)) : nop()
bvec1 = \
(frames > 0) ? (!Precise ? bvec1 : \
MRecalculate(super, bvec1, blksize=bs/2, overlap=bs/4, thSAD=100, \
truemotion=truemotion, lambda=lambda, pnew=pnew)) : nop()
fvec1 = \
(frames > 0) ? (!Precise ? fvec1 : \
MRecalculate(super, fvec1, blksize=bs/2, overlap=bs/4, thSAD=100, \
truemotion=truemotion, lambda=lambda, pnew=pnew)) : nop()
fvec2 = \
(frames > 1) ? (!Precise ? fvec2 : \
MRecalculate(super, fvec2, blksize=bs/2, overlap=bs/4, thSAD=100, \
truemotion=truemotion, lambda=lambda, pnew=pnew)) : nop()
fvec3 = \
(frames > 2) ? (!Precise ? fvec3 : \
MRecalculate(super, fvec3, blksize=bs/2, overlap=bs/4, thSAD=100, \
truemotion=truemotion, lambda=lambda, pnew=pnew)) : nop()

return (frames == 0) ? c : \
(frames == 1) ? c2.MDegrain1(super_rend, bvec1, fvec1, thSAD=400) : \
(frames == 2) ? c2.MDegrain2(super_rend, bvec1, fvec1, bvec2, fvec2, thSAD=400) : \
c2.MDegrain3(super_rend, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=400)     }

# ===============================================================================

[zorr]

Originally Posted by StainlessS

As TrueMotion is a group setting for some other args, might be interesting to see what Zorr's Avisynth Optimizer thing makes of it (altering individual args).

Sure! The challenge is coming up with an objective and reliable way to measure the quality of the grain removal. I gave it some thought and came up with something that may work...

So first we need to be able to calculate the "graininess" of the frame. Some googling and I found this, which looks like something that is possible to implement in Avisynth using MaskTools. I'm not sure about how to sum the pixel values after convolution though, perhaps by doing recursive bilinear downscaling in 16bit or 32bit per pixel colorspace in order not to lose too much precision. And hopefully there's a way to read a single pixel's color value into a script variable.

We also need another criteria, if we only measure graininess and consider it bad then the optimizer is going to find the blurriest image possible... We could measure the maximum absolute pixel color difference to the original unprocessed frame based on the idea that the degraining should not make a picture that has large deviations from the original.

With these two measurements the optimizer can create a two dimensional pareto front. After the optimizer is done a human would see which of the pareto front members is the best compromise. That would be a fun experiment to try... can anyone help me with the graininess measurement?

[StainlessS]

Unfortunately RT_Stats is only 8 bit and only really for Luma Y or RGB, I have no idea how 16/32 bit works in Avs+, never touched the stuff.
No idea how to force convert to some kind of 16->8 Least Significant Byte, Most significant Byte format so that RT could be used.

EDIT: Here are available COMBINED functions for Y, each func available as individual function as well. [down to single 8 bit pixel]

RT_Ystats(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
      float "threshold"=0.0,int "lo"=128,int "hi"=lo,int "flgs"=255,string "prefix"="YS_",
        float "mu"=0.0,int "d"=1,int "p"=1,int "u"=1, int "Matrix"=(Width>1100||Height>600?3:2),clip "mask"=NOT_USED,int "MaskMin"=128,"MaskMax"=255)
  Returns multiple results as for above single frame Luma sampling functions as Local Variables (prefixed with the prefix string arg).
  The args up to "interlaced", are as for all other clip functions, "threshold" used only for "RT_YPlaneMin", "RT_YPlaneMax" and
  "RT_YPlaneMinMaxDifference" equivalent routines with same functionality.
  "lo" and "hi" are used only with the "RT_YInRange" equivalent routine with same functionality.
  "mu" and "d" and "p" and "u" are used only with the "RT_YPNorm" equivalent routine with same functionality.
  The new arg "Flgs" selects which results you want returned and the string "Prefix" that is prepended
  to the returned Local variable names.
  The actual return result is a copy of the flgs arg with any non valid bits reset, ie the Local variables that were set.
  Local variables are NOT altered for any function not selected in flgs.
  Returns 0 if no pixels found in search area of mask within MaskMin and MaskMax.

  Flgs_Bit_Number   Add_To_Flgs     Equivalent_Function             Local_Var_Set_Excluding_Prefix
     0                 1($01)        RT_YPlaneMin()                     "yMin"        (0->255)
     1                 2($02)        RT_YPlaneMax()                     "yMax"        (0->255)
     2                 4($04)        RT_YPlaneMinMaxDifference()        "yMinMaxDiff" (0->255)
     3                 8($08)        RT_YPlaneMedian()                  "yMed"        (0->255)
     4                16($10)        RT_AverageLuma()                   "yAve"        (0.0->255.0)
     5                32($20)        RT_YPlaneStdev()                   "yStdev"      (0.0->255.0)
     6                64($40)        RT_YInRange()                      "yInRng"      (0.0->1.0)
     7               128($80)        RT_YPNorm()                        "yPNorm"      (0.0->??? depends upon d and u)
  RT_Ystats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
  routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
  The Default flgs=255($FF) are all bits set and so sets ALL Local vars at once.
  RT_Ystats(flgs=1+2+16) would set Local vars "YS_yMin", "YS_yMax" and "YS_yAve" for full frame current_frame.

  In addition to above Local Variables, RT_YStats() sets an int Local variable (where default prefix) of "YS_PixelCount" being
  the number of pixels in mask area X,Y,W,H between MaskMin and MaskMax inclusive, or pixels scanned in X,Y,W,H area where mask
  not used.
  NOTE, If no valid flg bits set (eg $FF00), then returns 0, YS_PixelCount and all other variables remain as before call.
  Assuming some valid flg bits, if no valid pixels were found in mask then function returns 0, and only YS_PixelCount would be set
  to 0, no other variables are touched (remain as before call, undefined if not previously existing).
  Example usage:
    ScriptClip("""
      got = RT_Ystats(c,mask=Mask,flgs=$10) # AverageLuma
      (got != 0) ? RT_debug("AverageLuma = " + String(YS_yAve) + "PixelCount = " + String(YS_PixelCount)) : RT_Debug("NO VALID PIXELS FOUND")
    """)

RT_YPNorm, is a weird one as requested by martin53, [below is the individual routine, also available in above combined results routine]

RT_YPNorm(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
      float "mu"=0.0,int "d"=1,int "p"=1,int "u"=1,int "Matrix"=(Width>1100||Height>600?3:2),clip "mask"=NOT_USED,int "MaskMin"=128,"MaskMax"=255)
  Return int -1 if no valid pixels in Mask clip.
  Returns FLOAT value greater or equal to 0.0, being the "Minkowski P-norm" (range depends upon values of 'd' and 'u') for frame(n+delta)
  area x,y,w,h.
  mu, Float, default 0.0 (0.0 -> 255.0)
  d,  int,   default 1 (1 -> 255)       # downscale
  p,  int,   default 1 (1 -> 16)        # power
  u,  int,   default 1 (1 -> 255)       # final upscale before returning result (experimental)
  Formula is: sum_over_pixels[ ((pixel-mu)/d)^p ]^(1/p) * u
  or in words: d and u are scaling aids. The differences between the pixel values and mu are scaled, taken to the power of p and added up
  over the frame. The sum is taken to the p-th root and finally rescaled.
  mu=0, d=1, p=1, u=1 yields the average.
  mu=average, d=1, p=2, u=1 yields the standard deviation (uncorrected sample standard deviation).
  Implemented as requested by Martin53 (thankyou). *** EXPERIMENTAL ***
  http://en.wikipedia.org/wiki/P-norm#The_p-norm_in_finite_dimensions
  http://en.wikipedia.org/wiki/Minkowski_distance

EDIT: Fastest will be RT_AverageLuma (even for single pixel, but with int result as float 0.0->255.0, ie no fractional part), as others create count arrays,
whereas RT_AverageLuma sums on-the-fly [same for combined RT_YStats where AverageLuma is sole requested result].

[ErazorTT]

@zorr: I guess the easiest AND most precise way would be to start with a very clean clip, add noise to it and use this as input for the denoiser. The output of the denoiser and the original ground truth are than be subtracted pixel by pixel. Finally all the pixel differences are summed frame by frame. The best denoiser setting will have smallest departure from the original.

For this to work there is thus no need for any measure of noise/grain. Just summing up the difference of all pixels in a frame gives the answer.

[StainlessS]

Good suggestion methinks.
Would Tears Of Steel be a good test source, with both live and CGI content.

Source from Avisynth Usage Forum Sticky, "Consolidated list of test video clip resources",
https://media.xiph.org/

We have a lossless version of the short film Tears of Steel. (17620 frames 2.40:1)

[ y4m 4k (67 GB compressed) ]
[ png 1920x800 (30 GB) ]
[ tiff 4k 16 bit (598 GB) ]

[ flac stereo (80 MB) | 5.1 surround (331 MB) ]

[ exr frames by scene (226 GB) | cleaned footage (330 GB) | raw footage (4 TB) ]

Bit big mind you. Could probably find shorter smaller sequence somewhere on net.

Some Lossy Compressed Versions, various compression / sizes:- https://mango.blender.org/download/
EDIT: Maybe someone already has big download and could spare a 30 second snippit for testing.

[VoodooFX]

Originally Posted by ErazorTT

And now the comparison of why I think it was time for an improved motion estimation over v1 of TemproralDegrain.
original from Breaking Bad s1e1, the camera moves following the car:


v1.23: TemporalDegrain(degrain=2)

The motion of the low contrast tarmac is not captured well and thus the details are being washed away.

v2.03: TemporalDegrain2(degrainTR=2)

More of the details of the tarmac are preserved.

Very nice overall improvement. Why v2.03 fails on this part?:

original vs v1.23 vs v2.03

[ErazorTT]

@VoodooFX
This one is tricky, since it results from the motion estimation I took from QTGMC. I tried to tweak the motion estimation in a way to improve the detection of moving fine detail as the rim in this example but without putting either the motion detection of big objects or the degraining/denoising at risk.

original, v1.23, v2.03, v2.10


As you can see v2.10 has gotten a little better when compared to v2.03 however it cannot match the detail of v1.23 in this case.
I would consider this issue a corner case, since there are a couple of factors driving this: the very fast movement of this detail, the total time the detail is visible and its size.

[VoodooFX]

Not the same as v1.23, but still huge improvement. Fast moving details not that important so better not to compromise with slow moving or static details.

[tormento]

Strangely enough I get the following error, both in VirtualDub2 and in Simple x264 launcher:

[2018-10-28][10:56:47] error: Script error: There is no function named 'ConvertToStacked'.
[2018-10-28][10:56:47] (D:/Programmi/Media/AviSynth+/plugins64/TemporalDegrain-2.1.1—ErazorTT.avsi, line 236)

My script:

PHP Code:

SetMemoryMax(8000)
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\eseguibili\media\DGDecNV\x64\DGDecodeNV.dll")
DGSource("E:\in\2_01 sacrificio del cervo sacro, Il\sacrificio.dgi")
CompTest(1)
TemporalDegrain2(degrainTR=4,postFFT=4,postSigma=3,postDither=-1)
Prefetch(6) 


EDIT: solved.

[ErazorTT]

You will need Avisynth+ in a recent version.