VDub's Temporal smoother To Avisynth ?

[Neil Lee]

VDub's Temporal smoother To Avisynth ?

I've started to ultilize MVTool's motion compensated frames,
and feed them to VirtualDub's Temporal Smoother, tiresome and
slow but the result is pretty amazing. Sorry to say, I haven't
been able to reproduce half the quality using many of avisynth's
temporal smoother.

IMHO, the reasons lie in two parts...
First, Vdub's Temporal smoother uses up to 7 frames for averaging.
it's algorithm is pretty sufficicated. Second it uses sum of RGB Dif rather
than YUV format, which i think is a much reasonable approach to
determine the threshold. Avisynth's smoothers can't seem to get enough
of a smoother result when dealing with very noisy videos, I can lower
the threshold, but noise's UV values can vary just a bit too much..

I'm very new to avisynth and just wonder if the Vdub's smoother algo
can be achieved in avisynth simply by scripting?

I dug up the algorithm posted by Vdub's author avery lee in his forum.

Maybe someone is interested..?

http://forums.virtualdub.org/index.php?act=ST&f=7&t=713&hl=&

Temporal smoother is a seven-frame centered, finite-response filter with
per-pixel adaptive weighting. For each pixel position and for each pixel in
that position in the seven-frame window, compute the 2-norm squared
absolute RGB difference from the center window pixel to
the current window pixel (d = |r|^2 + |g|^2 + |b|^2). Shift d right by
N bits (divide by 2^N) , where N is the filter power, then subtract it from
16 with clamping to zero. Compute a new vector (d, r*d, g*d, b*d).

Accumulate all such vectors for all seven frames in the window,
then divide the RGB terms by the alpha term to get noise reduced RGB for
that pixel position in the center window frame. The alpha sum will never be
zero since the center window pixel always matches itself perfectly. Finally,
cycle a new source pixel into the window -- this will be three frames behind
the currently output pixel. The output of the filter should never cycle back
into the window, or else you will have an incorrect, recursive filter.

The source is uncommented because I basically prototyped the algorithm
directly in MMX and never bothered to write a psuedocode or scalar version
.

[Revgen]

...it's algorithm is pretty sufficicated...

SPELLING ALERT!

sufficicated = sophisticated



Sorry, I had to do it. :) Happy New Year!

[stickboy]

There's already an AviSynth version of TemporalSmoother (http://users.win.be/dividee/avisynth.html).

However, last time I tried it, it had some issues (http://forum.doom9.org/showthread.php?p=189610#post189610).

[Didée]

The closest we have for YV12 is TTempsmooth, as it can do weighted averaging. I've posted a basic possible utilization (http://forum.doom9.org/showthread.php?p=752196#post752196) of it for motion compensated denoising. It's really basic, but works rather OK. Smarter (and slower) ways of processing are in the works.

[phædrus]

I am a fan of the TemporalSmoother function in VDubMod also -- it is better than some other AVS denoisers I've tried. And I would like a way to utilize it in my AVS scripts for encoding to mpeg1 or mpeg2, not just avi as I've been doing in VirtualDub.

But I am not an AVS expert, either. Can somebody who is familiar with TTempsmooth suggest script parameters which would approximate the effects of TemporalSmoother?

[Didée]

Quickly put together, hardly tested, but result seems not too bad ;) (except for speed :D )

Perhaps the parameter calculation could be done better ... I'm already sleeping ;)

source = last # whatever you have at this point

frames = 5 # No. of frames to include into averaging (fwd & bckwd)
noiselevel = 8 # estimated noise variance to be removed

#----------------------
fr1 = int(round( frames/2.0 ))
fr2 = frames
th1_1 = noiselevel*2
th1_2 = int(round( noiselevel/2.0 ))
th2_1 = int(round( pow(noiselevel,0.25) - 1.0 ))
th2_2 = th2_1-1
th2_2 = (th2_2 < 0) ? 0 : th2_2
#----------------------

x1 = source.ttempsmooth(fr1,th1_1,th1_1,th2_1,th2_1,fr1-1)

ttempsmooth(fr2,th1_2,th1_2,th2_2,th2_2,fr2-1,pfclip=x1)
# FluxSmoothT(4) # perhaps reduce 'noiselevel' a little, and activate this.

Not that it necessariliy had to be done this way ... it's just a thought wrapped into a script, and the result seems comparable to Vdub's Temporal Smoother ... perhaps it even slightly has the edge. Just try and see. Regarding speed, it looses big time, no doubt.
For me it's bedtime now ;)

[phædrus]

Thank you, Didée. I will try this out and see if I can get it to work for me. I haven't used this filter before, so there may be a learning curve.

I assume that if frames = 5 were reduced to 3, and if noiselevel = 8 were reduced to 5, it would run faster. Well, we shall see.

[Mug Funky]

it uses sum of RGB Dif rather
than YUV format, which i think is a much reasonable approach to
determine the threshold.

actually, from a video signal point of view, the error is usually much higher in the chroma planes. temporalsoften is useful in that it allows you to specify a threshold for y, uv and scenechange, which is very good for stuff with noisy chroma but clean luma.

[phædrus]

Quickly put together, hardly tested, but result seems not too bad ;) (except for speed :D )

Perhaps the parameter calculation could be done better ... I'm already sleeping ;)

source = last # whatever you have at this point

frames = 5 # No. of frames to include into averaging (fwd & bckwd)
noiselevel = 8 # estimated noise variance to be removed

#----------------------
fr1 = int(round( frames/2.0 ))
fr2 = frames
th1_1 = noiselevel*2
th1_2 = int(round( noiselevel/2.0 ))
th2_1 = int(round( pow(noiselevel,0.25) - 1.0 ))
th2_2 = th2_1-1
th2_2 = (th2_2 < 0) ? 0 : th2_2
#----------------------

x1 = source.ttempsmooth(fr1,th1_1,th1_1,th2_1,th2_1,fr1-1)

ttempsmooth(fr2,th1_2,th1_2,th2_2,th2_2,fr2-1,pfclip=x1)
# FluxSmoothT(4) # perhaps reduce 'noiselevel' a little, and activate this.

Not that it necessariliy had to be done this way ... it's just a thought wrapped into a script, and the result seems comparable to Vdub's Temporal Smoother ... perhaps it even slightly has the edge. Just try and see. Regarding speed, it looses big time, no doubt.
For me it's bedtime now ;)


I appreciate your work on this, Didée, but being only moderately comfortable with basic avs scripting, I decided to try just TTempSmooth() with the default values as a test. I thought the result was good, but I only tried it on one source.

Can I just cut and paste your code into my script? i.e. It is not understandable to me, really, but I know how to cut and paste. :)

So, I would try this next -- your code pasted into my avs script -- and if I have made some terrible stupid newbie error, please point it out.

LoadPlugin("C:\Program Files\dgmpgdec140\DGDecode.dll")
LoadPlugin("C:\Program Files\TTempSmooth\TTempSmooth.dll")
LoadPlugin("C:\Program Files\Decomb521\decomb521.dll")
mpeg2source("D:\TEMPRIP\movie.d2v")

Telecide(Order=1,Guide=1).Decimate()

LanczosResize(640,352)

source = last # whatever you have at this point
frames = 5 # No. of frames to include into averaging (fwd & bckwd)
noiselevel = 8 # estimated noise variance to be removed

#----------------------
fr1 = int(round( frames/2.0 ))
fr2 = frames
th1_1 = noiselevel*2
th1_2 = int(round( noiselevel/2.0 ))
th2_1 = int(round( pow(noiselevel,0.25) - 1.0 ))
th2_2 = th2_1-1
th2_2 = (th2_2 < 0) ? 0 : th2_2
#----------------------

x1 = source.ttempsmooth(fr1,th1_1,th1_1,th2_1,th2_1,fr1-1)

ttempsmooth(fr2,th1_2,th1_2,th2_2,th2_2,fr2-1,pfclip=x1)

I don't want to impinge on your time, but since you already went to the trouble of hashing this out, I feel it is only proper courtesy to make a good attempt to test it out here. And I do appreciate your effort.

phædrus