Advanced Denoising [Two new functions from mf]

[Soulhunter]

Ok, here are two new functions from mf:

# MotionThresh by mf
# Simple scenechange-proof motion threshold
# Tile outputs 16x16 clip for speed
# Use tile=true for conditional filtering, tile=false for masking

function MotionThresh(clip input, float thresh, bool "tile") {
tile = Default(tile, false)

black = BlankClip(input, width=16, height=16)
white = BlankClip(input, width=16, height=16, color=$FFFFFF)
cond1 = ConditionalFilter(input, white, black, "YDifferenceFromPrevious()", "greaterthan", String(thresh))
cond2 = ConditionalFilter(input, white, black, "YDifferenceToNext()", "greaterthan", String(thresh))
Overlay(cond1, cond2, mode="darken")
tile ? last : PointResize(input.width, input.height)
}

# MotionRamp by mf
# Average motion soft-thresholding based on 5 thresholds
# Dependancies: MotionThresh, BlendMulti

function MotionRamp(clip input, int thresh1, int thresh2, int thresh3, int thresh4, int thresh5, int "min", int "max", int "floor", int "ceil", int "radius", bool "tile") {
min = Default(min, 0)
max = Default(max, 255)
floor = Default(floor, 0)
ceil = Default(ceil, 255)
radius = Default(radius, 2)
tile = Default(tile, false)

input
BlendMulti(MotionThresh(thresh1, tile=true), MotionThresh(thresh2, tile=true), MotionThresh(thresh3, tile=true), MotionThresh(thresh4, tile=true), MotionThresh(thresh5, tile=true))
TemporalSoften(radius,255,0,255,2)
Levels(floor, 1, ceil, min, max)
ColorYUV(levels="TV->PC")
tile ? last : PointResize(input.width, input.height)
}

Basically they translate the motion amount of a frame -> a luminance value which can be used to remote adjust filters... This can be used to raise the strength of a denoiser more n more as higher the motion gets [very useful to get some extra % compressibility as you wont notice the loss of fine details in high motion scenes anyway] :]


# Example:
# More Motion -> More Denoising via FFT3DFilter

Input=Last
Input

MotionRamp(5,10,15,20,25,Max=255,Tile=True)
MotionRamp=Last

Input

ScriptClip("Overlay(Last,Input,Opacity=AverageLuma(MotionRamp)/100)")
ScriptClip("FFT3DFilter(Sigma=(AverageLuma(MotionRamp)/2/2/2/2))")

You can also use this functions to raise denoising strength depending on the noise amount... Useful for sources where the noise amount changes from scene to scene!


# Example:
# More Noise -> More Denoising via FFT3DFilter

Input=Last

Deen("a2d",3,10,10)
Subtract(Last,Input)
Levels(120,1,137,0,255)

MotionRamp(5,10,15,20,25,Max=255,Tile=True)
MotionRamp=Last

Input

#ScriptClip("Subtitle(String(Round(AverageLuma(ramp))))")

ScriptClip("Overlay(Last,Input,Opacity=AverageLuma(MotionRamp)/100)")
ScriptClip("FFT3DFilter(Sigma=(AverageLuma(MotionRamp)/2/2/2/2))")

But there are much more possibilitys, just play around with it!

Btw, the BlendMulti plugin can be found here... (http://mf.creations.nl/avs/filters/)


Have fun ~Bye

[Pookie]

Thanks :)

Yup, it's an mf function.... :D :D

http://img87.imageshack.us/img87/5400/98887639mq7.th.jpg (http://img87.imageshack.us/my.php?image=98887639mq7.jpg)

[Soulhunter]

LOL! What was the script you used? Both? XD

Maybe try it with a different denoiser than FFT3DFilter!


Bye

[elguaxo]

Wow, sounds great! Now the n00b question to understand this better. In your first example, what would be the minimum and maximum possible values for Sigma?
FFT3DFilter(Sigma=(AverageLuma(motionramp)/2/2/2/2))

Thanks!

[foxyshadis]

0 to 16, by the math, but in practice it would probably never get that high.

One error I notice in that line is that while sigma is a float, the division will give you an integer, losing a lot of precision... it should be something like
sigma=(AverageLuma(motionramp)/16.0)

[Soulhunter]

It was just a example to demonstrate how to make use of mfs scripts... of coz you can use whatever you like! Btw, you see a way to make a nice fast dll out of MotionThresh/MotionRamp? Maybe we can speed stuff up a little bit by using optimized code and/or using a different method to generate the values for the denoiser... :]


Bye

[Soulhunter]

So, someone tested this stuff already? >.>


Bye

[AssassiNBG]

It sounds great! I'll see if I can find some time these days to try those two out.

Wondering why this thread didn't get much attention...

[Didée]

*cough* FMF & QMF by HomieFR *cough*

[Soulhunter]

Links plz! :P

Can only find posts on some french board... (http://www.unite-video.com/phpbb/viewtopic.php?=1&t=2433&start=0&postdays=0&postorder=asc&vote=viewresult) >.>

Actualy the board MarcFD used to hang around, lol!


Tia n' Bye

[Soulhunter]

Ok, found a link to a zip with a HomieFR.dll (http://www.geocities.com/manao47/Filters/homiefr-v0.1.zip) plus a french readme...

What does the DLL? Spitting out motion values? Is it faster than mf's script?


Tia n' Bye

[gzarkadas]

Links plz! :P

I think Didée refers to the script contained inside the "Advanced conditional filtering: part III" section of the ConditionalFilter 's Avisynth docs page.

[Wilbert]

@gzarkadas,

It looks like someone converted the idea of QMF into a plugin. It's the first time i see that plugin though.

@all,

Did anyone try this plugin and compared it to the script? Does anyone know who the author is of the plugin?

@soulhunter,

What does the DLL? Spitting out motion values?
Just look at the script (and its intro) that gzarkadas refers to. The plugin does something similar. It defines low/medium/high motion which can be used in the conditional environment (thus everything happens on frame basis) in combination with QMF:

Some adaptive motion/resizing filters appeared on the forums. These filters discriminate between low, medium and high motion in a clip (on frame basis). By doing that, different filters can be used for different kind of motion in the clip. In general, one should use temporal smoothing in low motion scenes, spatial smoothing in high motion scenes and use spatio-temporal smoothing in medium motion scenes.

[Soulhunter]

Oh, I see... Thanks for the hint! :] Seems its similar to the "More Motion -> More Denoising" example I posted... Just that it works with just 3 motion levels [low, mid, high] instead of 256, and it picks one of 3 filter methods based on this instead of adjusting a filters strength, right?

EDIT: Missed Wilberts reply... ^^


Thx n' Bye

[Dreassica]

Nice! Been working with qmf for a while now and I look forward to playing with this one :) A shame there is no debug option in the dll version.

[Didée]

Indeed I was thinking of this thread (http://forum.doom9.org/showthread.php?t=56051), dating back in 2003, so the idea in itself is not exactly new.

Generally, I'm not too convinced by the method. It's the old game of making a basic temporal evaluation of a basic spatial measurement. This leaves, generally, the old problem of making a reliable distinction between "noise", "detail", and "motion".

Did a few (very quick) tests several weeks ago. It seems that for the idea of "more motion -> more filtering" it works out for the most part.
Still, I see a problem when parts of the frame a low motion, and parts of the frame have high or changing motion: if, e.g., the left part of a frame changes from low to high motion, then the whole frame will be treated as being of higher motion, where only the left half should. That's the point I also mentioned to HomiE in the early stages of QFM.

The idea of "adapt filter strength to changing noise levels" I could not find practical at all. With methods similar to Soulhunter's suggestion, the whole thing is way more sensible to changing motion than it is to changing noise. In effect, if I tried to raise filter strength with increasing noise, the whole frame changed from sharp to overfiltered as soon as a static scene started to move by even very slight slight camera pan. A no-go.

My attitude stays the same: "measuring" and distincting-between noise/detail or detail/motion is not that easy. Trying to to that measurement by "simple" means is not reliable. For more reliable results, the usage of slow motion vector search is hardly to avoid.

Of course ... if you want to use a weak FFT3D(sigma=0.9) for low motion/low noise, and a strong sledgehammer like FFT3D(sigma=1.1) for high motion/high noise, then it works perfectly. ;)


Edit: BTW, the functions could be simplified & sped up quite a bit. A pity that mf still doesn't like masktools too much, as it seems.

[foxyshadis]

So basically, put the output of mvmask(kind=0) through a few different luts to create masks for the weights each filter should have on the output? Sounds simple enough, the trick mainly being determining the cutoffs. Two maskedmerges in a row shouldn't cause much roundoff error. It shouldn't be any slower than motionramp, might be faster. Masking probably doesn't need to be extremely accurate, unless you happen to be Didée or Chainmax, so you could get away with a rather faster mvanalyse. :p