Advanced Denoising [Two new functions from mf]

Soulhunter · 26th February 2007, 00:30

Ok, here are two new functions from mf:

Code:

# MotionThresh by mf
# Simple scenechange-proof motion threshold
# Tile outputs 16x16 clip for speed
# Use tile=true for conditional filtering, tile=false for masking
 
function MotionThresh(clip input, float thresh, bool "tile") {
tile = Default(tile, false)
 
black = BlankClip(input, width=16, height=16)
white = BlankClip(input, width=16, height=16, color=$FFFFFF)
cond1 = ConditionalFilter(input, white, black, "YDifferenceFromPrevious()", "greaterthan", String(thresh))
cond2 = ConditionalFilter(input, white, black, "YDifferenceToNext()", "greaterthan", String(thresh))
Overlay(cond1, cond2, mode="darken")
tile ? last : PointResize(input.width, input.height)
}

Code:

# MotionRamp by mf
# Average motion soft-thresholding based on 5 thresholds
# Dependancies: MotionThresh, BlendMulti
 
function MotionRamp(clip input, int thresh1, int thresh2, int thresh3, int thresh4, int thresh5, int "min", int "max", int "floor", int "ceil", int "radius", bool "tile") {
min = Default(min, 0)
max = Default(max, 255)
floor = Default(floor, 0)
ceil  = Default(ceil, 255)
radius = Default(radius, 2)
tile = Default(tile, false)
 
input
BlendMulti(MotionThresh(thresh1, tile=true), MotionThresh(thresh2, tile=true), MotionThresh(thresh3, tile=true), MotionThresh(thresh4, tile=true), MotionThresh(thresh5, tile=true))
TemporalSoften(radius,255,0,255,2)
Levels(floor, 1, ceil, min, max)
ColorYUV(levels="TV->PC")
tile ? last : PointResize(input.width, input.height)
}

Basically they translate the motion amount of a frame -> a luminance value which can be used to remote adjust filters... This can be used to raise the strength of a denoiser more n more as higher the motion gets [very useful to get some extra % compressibility as you wont notice the loss of fine details in high motion scenes anyway] :]

Code:

# Example:
# More Motion -> More Denoising via FFT3DFilter

Input=Last
Input

MotionRamp(5,10,15,20,25,Max=255,Tile=True)
MotionRamp=Last

Input

ScriptClip("Overlay(Last,Input,Opacity=AverageLuma(MotionRamp)/100)")
ScriptClip("FFT3DFilter(Sigma=(AverageLuma(MotionRamp)/2/2/2/2))")

You can also use this functions to raise denoising strength depending on the noise amount... Useful for sources where the noise amount changes from scene to scene!

Code:

# Example:
# More Noise -> More Denoising via FFT3DFilter

Input=Last

Deen("a2d",3,10,10)
Subtract(Last,Input)
Levels(120,1,137,0,255)

MotionRamp(5,10,15,20,25,Max=255,Tile=True)
MotionRamp=Last

Input

#ScriptClip("Subtitle(String(Round(AverageLuma(ramp))))")

ScriptClip("Overlay(Last,Input,Opacity=AverageLuma(MotionRamp)/100)")
ScriptClip("FFT3DFilter(Sigma=(AverageLuma(MotionRamp)/2/2/2/2))")

But there are much more possibilitys, just play around with it!

Btw, the BlendMulti plugin can be found here...

Have fun ~Bye

Pookie · 26th February 2007, 01:09

Thanks

Yup, it's an mf function....

Soulhunter · 26th February 2007, 01:49

LOL! What was the script you used? Both? XD

Maybe try it with a different denoiser than FFT3DFilter!

Bye

elguaxo · 26th February 2007, 02:41

Wow, sounds great! Now the n00b question to understand this better. In your first example, what would be the minimum and maximum possible values for Sigma?

Code:

FFT3DFilter(Sigma=(AverageLuma(motionramp)/2/2/2/2))

Thanks!

foxyshadis · 26th February 2007, 03:10

0 to 16, by the math, but in practice it would probably never get that high.

One error I notice in that line is that while sigma is a float, the division will give you an integer, losing a lot of precision... it should be something like
sigma=(AverageLuma(motionramp)/16.0)

Soulhunter · 26th February 2007, 05:12

It was just a example to demonstrate how to make use of mfs scripts... of coz you can use whatever you like! Btw, you see a way to make a nice fast dll out of MotionThresh/MotionRamp? Maybe we can speed stuff up a little bit by using optimized code and/or using a different method to generate the values for the denoiser... :]

Bye

Soulhunter · 31st March 2007, 21:01

So, someone tested this stuff already? >.>

Bye

AssassiNBG · 5th April 2007, 20:10

It sounds great! I'll see if I can find some time these days to try those two out.

Wondering why this thread didn't get much attention...

Didée · 5th April 2007, 22:27

*cough* FMF & QMF by HomieFR *cough*

Soulhunter · 6th April 2007, 01:13

Links plz! :P

Can only find posts on some french board... >.>

Actualy the board MarcFD used to hang around, lol!

Tia n' Bye

Soulhunter · 6th April 2007, 01:35

Ok, found a link to a zip with a HomieFR.dll plus a french readme...

What does the DLL? Spitting out motion values? Is it faster than mf's script?

Tia n' Bye

gzarkadas · 6th April 2007, 09:51

Quote:

Originally Posted by Soulhunter

Links plz! :P

I think Didée refers to the script contained inside the "Advanced conditional filtering: part III" section of the ConditionalFilter 's Avisynth docs page.

Wilbert · 6th April 2007, 10:15

@gzarkadas,

It looks like someone converted the idea of QMF into a plugin. It's the first time i see that plugin though.

@all,

Did anyone try this plugin and compared it to the script? Does anyone know who the author is of the plugin?

@soulhunter,

Quote:

What does the DLL? Spitting out motion values?

Just look at the script (and its intro) that gzarkadas refers to. The plugin does something similar. It defines low/medium/high motion which can be used in the conditional environment (thus everything happens on frame basis) in combination with QMF:

Quote:

Some adaptive motion/resizing filters appeared on the forums. These filters discriminate between low, medium and high motion in a clip (on frame basis). By doing that, different filters can be used for different kind of motion in the clip. In general, one should use temporal smoothing in low motion scenes, spatial smoothing in high motion scenes and use spatio-temporal smoothing in medium motion scenes.

Soulhunter · 6th April 2007, 10:20

Oh, I see... Thanks for the hint! :] Seems its similar to the "More Motion -> More Denoising" example I posted... Just that it works with just 3 motion levels [low, mid, high] instead of 256, and it picks one of 3 filter methods based on this instead of adjusting a filters strength, right?

EDIT: Missed Wilberts reply... ^^

Thx n' Bye

Dreassica · 6th April 2007, 11:42

Nice! Been working with qmf for a while now and I look forward to playing with this one

A shame there is no debug option in the dll version.

Didée · 6th April 2007, 13:56

Indeed I was thinking of this thread, dating back in 2003, so the idea in itself is not exactly new.

Generally, I'm not too convinced by the method. It's the old game of making a basic temporal evaluation of a basic spatial measurement. This leaves, generally, the old problem of making a reliable distinction between "noise", "detail", and "motion".

Did a few (very quick) tests several weeks ago. It seems that for the idea of "more motion -> more filtering" it works out for the most part.
Still, I see a problem when parts of the frame a low motion, and parts of the frame have high or changing motion: if, e.g., the left part of a frame changes from low to high motion, then the whole frame will be treated as being of higher motion, where only the left half should. That's the point I also mentioned to HomiE in the early stages of QFM.

The idea of "adapt filter strength to changing noise levels" I could not find practical at all. With methods similar to Soulhunter's suggestion, the whole thing is way more sensible to changing motion than it is to changing noise. In effect, if I tried to raise filter strength with increasing noise, the whole frame changed from sharp to overfiltered as soon as a static scene started to move by even very slight slight camera pan. A no-go.

My attitude stays the same: "measuring" and distincting-between noise/detail or detail/motion is not that easy. Trying to to that measurement by "simple" means is not reliable. For more reliable results, the usage of slow motion vector search is hardly to avoid.

Of course ... if you want to use a weak FFT3D(sigma=0.9) for low motion/low noise, and a strong sledgehammer like FFT3D(sigma=1.1) for high motion/high noise, then it works perfectly.

Edit: BTW, the functions could be simplified & sped up quite a bit. A pity that mf still doesn't like masktools too much, as it seems.

foxyshadis · 6th April 2007, 15:00

So basically, put the output of mvmask(kind=0) through a few different luts to create masks for the weights each filter should have on the output? Sounds simple enough, the trick mainly being determining the cutoffs. Two maskedmerges in a row shouldn't cause much roundoff error. It shouldn't be any slower than motionramp, might be faster. Masking probably doesn't need to be extremely accurate, unless you happen to be Didée or Chainmax, so you could get away with a rather faster mvanalyse.

26th February 2007, 01:09	#2 \| Link
Pookie Registered User Join Date: Apr 2005 Posts: 1,339	Thanks Yup, it's an mf function.... Last edited by Pookie; 26th February 2007 at 01:43.

26th February 2007, 01:49	#3 \| Link
Soulhunter Confused Join Date: Apr 2003 Location: Euroland Posts: 2,820	LOL! What was the script you used? Both? XD Maybe try it with a different denoiser than FFT3DFilter! Bye Last edited by Soulhunter; 26th February 2007 at 01:52.

26th February 2007, 02:41	#4 \| Link
elguaxo Registered User Join Date: Jun 2006 Posts: 260	Wow, sounds great! Now the n00b question to understand this better. In your first example, what would be the minimum and maximum possible values for Sigma? Code: FFT3DFilter(Sigma=(AverageLuma(motionramp)/2/2/2/2)) Thanks!

26th February 2007, 05:12	#6 \| Link
Soulhunter Confused Join Date: Apr 2003 Location: Euroland Posts: 2,820	It was just a example to demonstrate how to make use of mfs scripts... of coz you can use whatever you like! Btw, you see a way to make a nice fast dll out of MotionThresh/MotionRamp? Maybe we can speed stuff up a little bit by using optimized code and/or using a different method to generate the values for the denoiser... :] Bye Last edited by Soulhunter; 26th February 2007 at 15:55.

31st March 2007, 21:01	#7 \| Link
Soulhunter Confused Join Date: Apr 2003 Location: Euroland Posts: 2,820	So, someone tested this stuff already? >.> Bye Last edited by Soulhunter; 31st March 2007 at 21:07.

26th February 2007, 03:10	#5 \| Link
foxyshadis Angel of Night Join Date: Nov 2004 Location: Tangled in the silks Posts: 9,549	0 to 16, by the math, but in practice it would probably never get that high. One error I notice in that line is that while sigma is a float, the division will give you an integer, losing a lot of precision... it should be something like sigma=(AverageLuma(motionramp)/16.0)

5th April 2007, 20:10	#8 \| Link
AssassiNBG Registered User Join Date: Mar 2005 Location: Pleven, Bulgaria, Europe. Posts: 45	It sounds great! I'll see if I can find some time these days to try those two out. Wondering why this thread didn't get much attention...

5th April 2007, 22:27	#9 \| Link
Didée Registered User Join Date: Apr 2002 Location: Germany Posts: 5,407	cough FMF & QMF by HomieFR cough __________________ - We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

6th April 2007, 01:13	#10 \| Link
Soulhunter Confused Join Date: Apr 2003 Location: Euroland Posts: 2,820	Links plz! :P Can only find posts on some french board... >.> Actualy the board MarcFD used to hang around, lol! Tia n' Bye Last edited by Soulhunter; 6th April 2007 at 01:18.

6th April 2007, 01:35	#11 \| Link
Soulhunter Confused Join Date: Apr 2003 Location: Euroland Posts: 2,820	Ok, found a link to a zip with a HomieFR.dll plus a french readme... What does the DLL? Spitting out motion values? Is it faster than mf's script? Tia n' Bye

6th April 2007, 10:20	#14 \| Link
Soulhunter Confused Join Date: Apr 2003 Location: Euroland Posts: 2,820	Oh, I see... Thanks for the hint! :] Seems its similar to the "More Motion -> More Denoising" example I posted... Just that it works with just 3 motion levels [low, mid, high] instead of 256, and it picks one of 3 filter methods based on this instead of adjusting a filters strength, right? EDIT: Missed Wilberts reply... ^^ Thx n' Bye

6th April 2007, 11:42	#15 \| Link
Dreassica Registered User Join Date: May 2002 Posts: 384	Nice! Been working with qmf for a while now and I look forward to playing with this one A shame there is no debug option in the dll version.

6th April 2007, 13:56	#16 \| Link
Didée Registered User Join Date: Apr 2002 Location: Germany Posts: 5,407	Indeed I was thinking of this thread, dating back in 2003, so the idea in itself is not exactly new. Generally, I'm not too convinced by the method. It's the old game of making a basic temporal evaluation of a basic spatial measurement. This leaves, generally, the old problem of making a reliable distinction between "noise", "detail", and "motion". Did a few (very quick) tests several weeks ago. It seems that for the idea of "more motion -> more filtering" it works out for the most part. Still, I see a problem when parts of the frame a low motion, and parts of the frame have high or changing motion: if, e.g., the left part of a frame changes from low to high motion, then the whole frame will be treated as being of higher motion, where only the left half should. That's the point I also mentioned to HomiE in the early stages of QFM. The idea of "adapt filter strength to changing noise levels" I could not find practical at all. With methods similar to Soulhunter's suggestion, the whole thing is way more sensible to changing motion than it is to changing noise. In effect, if I tried to raise filter strength with increasing noise, the whole frame changed from sharp to overfiltered as soon as a static scene started to move by even very slight slight camera pan. A no-go. My attitude stays the same: "measuring" and distincting-between noise/detail or detail/motion is not that easy. Trying to to that measurement by "simple" means is not reliable. For more reliable results, the usage of slow motion vector search is hardly to avoid. Of course ... if you want to use a weak FFT3D(sigma=0.9) for low motion/low noise, and a strong sledgehammer like FFT3D(sigma=1.1) for high motion/high noise, then it works perfectly. Edit: BTW, the functions could be simplified & sped up quite a bit. A pity that mf still doesn't like masktools too much, as it seems. __________________ - We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

6th April 2007, 15:00	#17 \| Link
foxyshadis Angel of Night Join Date: Nov 2004 Location: Tangled in the silks Posts: 9,549	So basically, put the output of mvmask(kind=0) through a few different luts to create masks for the weights each filter should have on the output? Sounds simple enough, the trick mainly being determining the cutoffs. Two maskedmerges in a row shouldn't cause much roundoff error. It shouldn't be any slower than motionramp, might be faster. Masking probably doesn't need to be extremely accurate, unless you happen to be Didée or Chainmax, so you could get away with a rather faster mvanalyse.