Suggestion for "TemporalDegrain" for HD contents

[moviefan]

Hi guys, especially Sagekilla and Didee who created TemporalDegrain, which produces absolutely amazing results for me. I applied this filter for HD content and noticed it is extremely slow (as also mentioned at the avisynth wiki page). Would there be a lot of speed and/or quality improvement, if you used RemoveGrainHD instead of RemoveGrain for a TemporalDegrainHD version? Or does RemoveGrain have only very little impact on quality and speed respectively? Anyway, I would like to use TD for HD content too, but it's much too slow!

[foxyshadis]

The vast majority of the time spent in TemporalDegrain is in MVDegrain, so no, it won't give you any speedup to do this. RemoveGrainHD is significantly different from RemoveGrain and actually much slower, in fact. (Though it may or may not improve actual results, but you can use it as a prefilter yourself if you want to see how it works.)

The parameters degrain, pel, and blksize have the most effect on speed.

[thetoof]

You could try using FastDegrain (http://avisynth.org/mediawiki/upload/7/7d/FastDegrain.avs).

[maxisvk]

i try tonight to replace fft3dfilter with FFT3DGPU for prefilter like this:

#only with Dual/Quad CPU
SetMTMode(5)
##########

# "srch" is a prefiltered clip on which the motion serach is done.
# Here, we simply use FFT3DFilter. You can also use your own. Basically, you shouldn't use
# a clip with "a tiny bit of filtering". The search clip should ideally be "dead calm."
filter = o
filter = defined(denoise) ? denoise : filter.fft3dgpu(sigma=sigma,sigma2=s2,sigma3=s3,sigma4=s4,bt=4,bw=bw,bh=bh,ow=ow,oh=oh)
filter = (HQ>=1) ? filter.HQdn3D(4,3,6,3) : filter
srch = filter

#only with Dual 2,4 or Quad CPU 2,6
SetMTMode(2,6)
###########

with source 720x576 on my system Q6600@3.0ghz and Radeon 3450 i got 8.5fps vs 6.0fps with fft3dfilter (bt=4 vs bt=5 only 0.3fps). i'don't know if with single cpu the increment is the same and i need to check if fft3dgpu with MT produce artifact. If anyone has suggestions...

[Didée]

A fast variant for prefiltering:

MinBlur(2).FluxSmoothT() # or even MedianBlur(2,2,2).FluxSmoothT()

It's crude, but not that bad for sources that really need so strong filters like TemporalDegrain().


You could try using FastDegrain (http://avisynth.org/mediawiki/upload/7/7d/FastDegrain.avs).
That function is in bad shape, short of saying it were b0rked.

[moviefan]

@Didée: Do you consider TemporalDegrain() a very strong degraining/denoising function? I apply it to in my eyes rather medium noisy videos and There is sometimes not enough noise filtered. Concerning 1080p content: Would it be a lot of work for you to create a much faster version of TemporalDegrain for 1080p content which specifically aims at this resolution? Probably block size of noise is different than for SD material and thus TemporalDegrain as it is now isn't efficient. Or could you quickly explain the difference between SD noise filtering and HD noise filtering?

[thetoof]

TemporalDegrain is a very powerful degraining/denoising tool, but the reference clip makes all the difference. If you are not satified with either speed or denoising strength, use your own external reference clip.

I talked about what I use in this (http://forum.doom9.org/showthread.php?t=136175) thread (but it's VERY slow), so if you want it to go faster, try Didée's suggestion.

source = whateversource("yourfile")
denoised = source.MinBlur(2).FluxSmoothT() # or denoised = source.MedianBlur(2,2,2).FluxSmoothT()
TemporalDegrain(source,denoised) # (or TemporalDegrainExternal if you took the script posted in the link above)

You could also lower search, pel and use MVDegrain1... but that would affect quality.

[Didée]

And regarding "There is sometimes not enough noise filtered", it should be noted that the versions hosted right now at the Wiki are buggy. They use idx=2 for both the 1st and the 2nd call of MVDegrain, which causes way less denoising power than what the script is supposed to produce. The 2nd one has to use idx=3.
Affected are both the "stable" version V1.20 (Mar 22, 2008) and the "beta" version V1.18 (Feb 14, 2008).

Regarding HD content, especially full 1080p: a blocksize of 8 most probably is too small for such resolutions. bs=8 is okay for SD content. For 720p, bs=8 is a borderline case - that's why I left truemotion=true in the original script, albeit 'false' is better for denoising. For 1080p content, a blocksize of 16 is already rather small ...


Edit: uploaded fixed versions of TemporalDegrain and TemporalDegrain_beta to the Wiki.

[thetoof]

Hmmm... from what I see, beta is fixed, but not stable...

I left truemotion=true in the original script, albeit 'false' is better for denoising
So, would it be better to use false? If that's the case, what would be "good" values of the other settings (lambda, lsad, etc.)? Are there any that can be used for most sources or it has to be changed every time?

[Terranigma]

Hmmm... from what I see, beta is fixed, but not stable...


So, would it be better to use false? If that's the case, what would be "good" values of the other settings (lambda, lsad, etc.)? Are there any that can be used for most sources or it has to be changed every time?

Well it may be stronger for denoising, but in the case of mvdegrain, truemotion is recommended. You can try all the settings of fixed motion you like, it won't give you better results as far as SAD is concerned.

[Didée]

Well it may be stronger for denoising, but in the case of mvdegrain, truemotion is recommended. You can try all the settings of fixed motion you like, it won't give you better results as far as SAD is concerned.
Be careful with betting money on that. :)

No matter if one uses MVDegrain (with automatic internal compensation) or another "manual" way with compensate/interleave/filter/selectevery - in either way, a successful motion compensation is needed. Without a succesful compensation, you can't do successful denoising.


Left: source frame -- right: frame to compensate:

http://img153.imageshack.us/img153/9661/pic1sourcefg6.th.png (http://img153.imageshack.us/my.php?image=pic1sourcefg6.png) - http://img245.imageshack.us/img245/1869/pic1sourcedelta2ha6.th.png (http://img245.imageshack.us/my.php?image=pic1sourcedelta2ha6.png)


Would you call this a difficult scene for the motion engine? To me, it doesn't look very complicated.


Result with truemotion=true: http://img153.imageshack.us/img153/9067/pic3amvcomptruemotionls3.th.png (http://img153.imageshack.us/my.php?image=pic3amvcomptruemotionls3.png)


Result with truemotion=false: http://img267.imageshack.us/img267/8703/pic3bmvcompnottruemotionn6.th.png (http://img267.imageshack.us/my.php?image=pic3bmvcompnottruemotionn6.png)


Oh-lala. :D If truemotion misses not-difficult motion that easily, then the arguing it were better for denoising gets weak.
(And that's a *typical* results example, not a seldom exception.)

Of course, it's not like truemotion is all evil, and switching it off is the holy grail ... surely not. But first of all, one must not let be blinded by the well-sounding name of TRUE MOTION. It's just a name for settings that tie very tightly, nothing else.

Both presets are kind of extremes. The most usable settings are somewhere in-between, it depends on lots of things, and there are no "best" settings.

[Terranigma]

Well, the reason why I was thinking truemotion was better, because everytime i'd do a comparison, i'd compare the actual filesizes rather than looking at the images, and truemotion always had the smaller file size, but after seeing your images and doing some visual tests, it seems that that was because truemotion was actually removing subtle details, like stars and dots.

So :goodpost:

[moviefan]

So for a first step towards a proper Full-HD motion denoising, would you suggest to turn off truemotion and set block size to 16? This should also give a little speed improvement against a block size of 8. Additionally I will try your suggestion for the prefiltered clip. Is there anything else to improve, in particular for Full-HD?

@Didée: Are you sure, the fixed version is in the wiki?

[Didée]

Yes, it's definetly in the Wiki. In case of doubt, clear your browser's cache.

Regarding speed, there's nothing groundbreaking that could be done. A noticeable speedup can only be achieved by using simplified settings and/or methods, wich will lead to worse results. And when turning down the quality settings far enough so that speed becomes tasty, it might be better to not use mocomp at all.

[moviefan]

Hm, is it possible that truemotion=false doesn't calm the clip as much as it would be with truemotion=true? For my eyes, my clip with truemotion=false looks more unstable... Can anyone confirm this?

[thetoof]

This is from the MVTools guide... read it and then tweak the other settings affected by truemotion to adapt them to your source

truemotion is a preset of these parameters values. It allows easy to switch default values of all "true motion" parameters at once. Set it true for true motion search (high vector coherence), set it false to search motion vectors with best SAD. Default is true since v1.4.10. In any case you can tune each parameter individually.

lambda : set the coherence of the field of vectors. The higher, the more coherent. However, if set too high, some best motion vectors can be missed. Values around 400 - 2000 (for block size 8) are strongly recommended. Internally it is a SAD penalty coefficient for vector squared difference from predictor (neighbors), scaled by 256.
Default is 0 for truemotion=false and 1000*blksize*blksizeV/64 for truemotion=true.

lsad: SAD limit for lambda using. Local lambda is reset to 0 for vector predictor (formed from neighbor blocks) with greater then the limit SAD value. It prevents bad predictors using but destroy motion coherence. Values above 1000 (for block size=8) are strongly recommended for true motion.
Default is 400*blksize*blksizeV/64 for truemotion=false and 1200*blksize*blksizeV/64 for truemotion=true.

pnew: relative penalty (scaled to 256) to SAD cost for new candidate vector. New candidate vector must be better will be accepted as new vector only if its SAD with penalty (SAD + SAD*pnew/256) is lower then predictor cost (old SAD). It prevent replacing of quite good predictors by new vector with a little better SAD but different length and direction.
Default is 0 for truemotion=false and 50 for truemotion=true.

plevel: penalty factor lambda level scaling mode. Value=0 - no scaling, 1 - linear, 2 - quadratic dependence from hierarchical level size. Note, that vector length is smaller at lower level.
Default is 0 for truemotion=false and 1 for truemotion=true

global: estimate global motion (at every level) and use it as an additional predictor. Only pan shift is estimated (no zoom and rotation). Use false to disable, use true to enable. Default is false for truemotion=false and true for truemotion=true.

[g-force]

I think the basic concept of TemporalDegrain is great, but since it uses MVdegrain(x), which is an averaging filter, it is not as efficient as if it were to use a median filter. Here is part of a script that I have been working with that is much more efficient than TemporalDegrain, but very similar results.

########## some 3D filtering, frequency based sharpening, smoothing to avoid oversharpening
FFT3DFilter(sigma=1.25,sharpen=.7)
BlindPP(cpu2="ooooxo")

########## get motion vectors and apply them
source=last
backward_vec1=source.MVAnalyse(isb=true, delta=1,pel=2,overlap=4,sharp=1,idx=1)
forward_vec1 =source.MVAnalyse(isb=false,delta=1,pel=2,overlap=4,sharp=1,idx=1)
bw1=source.MVCompensate(backward_vec1,idx=1,thSAD=900)
fw1=source.MVCompensate(forward_vec1, idx=1,thSAD=900)

########## local motion stabilization/degrain stage
inter=Interleave(bw1,bw1,source,fw1,fw1)
intersmooth=inter.Clense().TemporalSoften(1,255,255,25,2)
RestoreMotionBlocks(intersmooth,inter,neighbour=inter,alternative=inter,gmthreshold=100,dist=1,
\ dmode=0,debug=false,noise=10,noisy=15,grey=false,show=false)
SelectEvery(5,2)

########## temporal min/max sharpening stage
pmax=source.MT_Logic(bw1,"max").MT_Logic(fw1,"max")
pmin=source.MT_Logic(bw1,"min").MT_Logic(fw1,"min")
Sharpen(.6)
MT_Clamp(pmax,pmin,1,1,U=2,V=2)

########## restore stars stage
filtered=last
mask=filtered.MT_Binarize(threshold=16,upper=true).MT_Expand
MT_Merge(filtered,source,mask)

-G

[thetoof]

What I tried either wasn't strong enough or completely screwed up the gradients. But, I must admit that it's quite fast and that it could be very useful for low to medium noise sources.

[Didée]

much more efficient than TemporalDegrain, but very similar results
Mini-comparison: >clickme< (http://www.mediafire.com/?xxcgxwtmmpk)

Two different worlds ...

[g-force]

Mini-comparison: >clickme< (http://www.mediafire.com/?xxcgxwtmmpk)

Two different worlds ...

Didée, thanks for the comparison clip. Admittedly, the source that I'm currently using that script on has much less noise that the one you posted. For that much noise you are better off not using the post-sharpening, which you should know adds a lot of the grain back, because you (Didée) invented it!

Would you guys be willing to try one more comparison? (or have I completely blown my credibility?) I feel pretty strongly that a median filter in combination with motion compensation is more efficient than MVDegrain(x). I previously posted a function that I wrote that deals with the heavy noise case, but I don't think anyone tried it. I'd be curious to hear how it works.

# GTemporalDegrainFaster ver.1.00 6MAR08
# Function by G-force,"Limited" Concept by Didee
# Requires FFT3DFilter.dll, mt_masktools, MVtools.dll, RemoveGrain.dll

function GTemporalDegrainFaster (clip input,int "threads",int "sigma")
{
source = input
ncpu = default(threads,1) #max number of CPU threads to use in FFT calculation (int>0, default=1)
sigma = default(sigma, 16) #turn down for more limited denoising
s2 = floor (sigma * 0.625)
s3 = floor (sigma * 0.375)
s4 = floor (sigma * 0.250)

filter = source.fft3dfilter(ncpu=ncpu,sigma=sigma,sigma2=s2,sigma3=s3,sigma4=s4,bt=5,bw=16,bh=16,ow=8,oh=8)
filterD = mt_makediff(source,filter)

backward_vec1= filter.MVAnalyse(isb=true, delta=1,pel=2,overlap=4,sharp=1,idx=1)
forward_vec1 = filter.MVAnalyse(isb=false,delta=1,pel=2,overlap=4,sharp=1,idx=1)
bw1 = source.MVCompensate(backward_vec1,idx = 2) #idx = 2 since working on a different clip than MVAnalyse
fw1 = source.MVCompensate(forward_vec1, idx = 2)

#nice thing about this approach is that it doesn't necessarily use any pixels from current frame
#this gets rid of a lot more dirt, scratches etc. (things that last only one frame)
nr = Interleave(bw1,source,fw1).Clense().SelectEvery(3,1)

# Limit "nr" to not do more than what "filter" would do. -Didee
nrD = mt_makediff(source,nr)
DD = mt_lutxy(filterD,nrD,"x 128 - abs y 128 - abs < x y ?")
source.mt_makediff(DD,U=2,V=2)

output = last
return(output)
}

:thanks:-G

[g-force]

Never mind.

I tried it out myself on the clip that Didée posted. Wow, that's a lot of grain!

in comparison GTemporalDegrainFaster is:

1. less sharp
2. less denoised
3. no denoising happens on scene changes

bummer. I guess TemporalDegrain is just too good!

-G

[g-force]

Okay, this gets much closer to the results that Didée posted using TemporalDegrain. Let me know what you think.

-G

directshowsource("C:\Documents and Settings\Auritg\Desktop\avisynth\source.avi")

########## some 3D filtering, frequency based sharpening
FFT3DFilter(sigma=4,sharpen=.3)

########## get motion vectors and apply them
source=last
backward_vec1= source.MVAnalyse(isb=true, delta=1,pel=2,overlap=4,sharp=1,idx=1)
forward_vec1 = source.MVAnalyse(isb=false,delta=1,pel=2,overlap=4,sharp=1,idx=1)
bw1 = source.MVCompensate(backward_vec1,idx = 1)
fw1 = source.MVCompensate(forward_vec1, idx = 1)

########## local motion stabilization/degrain stage
inter=Interleave(bw1,source,fw1)
intersmooth=inter.Clense()
RestoreMotionBlocks(intersmooth,inter,neighbour=inter,alternative=inter,gmthreshold=100,dist=1,
\ dmode=0,debug=false,noise=10,noisy=15,grey=false,show=false)
SelectEvery(3,1)

########## temporal min/max sharpening stage
pmax=source.MT_Logic(bw1,"max").MT_Logic(fw1,"max")
pmin=source.MT_Logic(bw1,"min").MT_Logic(fw1,"min")
Sharpen(.5)
MT_Clamp(pmax,pmin,1,1,U=2,V=2)

[thetoof]

Heh... you want a real challenge against TemporalDegrain? Try this (http://www.mediafire.com/?3suvhrgslov)!

[Terranigma]

Heh... you want a real challenge against TemporalDegrain? Try this (http://www.mediafire.com/?3suvhrgslov)!

I'll download and use mc_spuds to combat temporaldegrain ;)
So, perhaps you can do an temporal degrain encode, and i'll do one using the ultra efficient mc_spuds.


__
nvm, I noticed that you included a sample. :)

[g-force]

Heh... you want a real challenge against TemporalDegrain? Try this (http://www.mediafire.com/?3suvhrgslov)!

Nice blends.
Yikes! I bumped up the sigma in my previous post to 7 and it looks pretty close to your "TemporalDegrain" clip, except for - now I know what you mean by "messes up the gradients". There's a TON of chroma ickyness going on in your clip!

I'm just presenting this idea as a cheap alternative to TemporalDegrain. I don't mean to knock the greatness of the original!

-G

[g-force]

thetoof,

turns out I just needed to process all of the planes in fft3d.

the following looks very much like the TemporalDegrain clip you posted:

########## some 3D filtering, frequency based sharpening
FFT3DFilter(sigma=8.5,sharpen=.3,plane=4)

########## get motion vectors and apply them
source=last
backward_vec1= source.MVAnalyse(isb=true, delta=1,pel=2,overlap=4,sharp=1,idx=1)
forward_vec1 = source.MVAnalyse(isb=false,delta=1,pel=2,overlap=4,sharp=1,idx=1)
bw1 = source.MVCompensate(backward_vec1,idx = 1)
fw1 = source.MVCompensate(forward_vec1, idx = 1)

########## local motion stabilization/degrain stage
inter=Interleave(bw1,source,fw1)
intersmooth=inter.Clense()
RestoreMotionBlocks(intersmooth,inter,neighbour=inter,alternative=inter,gmthreshold=100,dist=1,
\ dmode=0,debug=false,noise=10,noisy=15,grey=false,show=false)
SelectEvery(3,1)

########## temporal min/max sharpening stage
pmax=source.MT_Logic(bw1,"max").MT_Logic(fw1,"max")
pmin=source.MT_Logic(bw1,"min").MT_Logic(fw1,"min")
Sharpen(.5)
MT_Clamp(pmax,pmin,1,1,U=2,V=2)

give it a try and report back. You won't be dissapointed!

-G

[Terranigma]

I've posted some examples using mc_spuds. :)

Download (http://www.mediafire.com/?jdwvfwfbmzy)

MC_Spuds (Calmed Grain):
Noise removed, with grain that was kept and calmed.

MC_Spuds (I Hate Grain):
0 Grain/noise, which reveals banding, but can be combated easily with some dithering.

[g-force]

Okay, so I had to "productize" it:

# GTemporalDegrainFaster2 ver.2.00 8Apr08
# Function by G-force, temporal min/max sharpening by Didée
# Requires FFT3DFilter.dll, mt_masktools, mvtools.dll, RemoveGrain.dll, RemoveDirt.dll

function GTemporalDegrainFaster2 (clip input, float "sigma", float "presharp", float "plane", int "ncpu", int "thSAD", int "idx", float "postsharp")
{
source = input
sigma = default(sigma, 4) #FFT3D strength - turn down for low-noise sources, up for high noise sources
presharp = default(presharp, .3) #FFT3D sharpening strength
plane = default(plane, 0) #FFT3D processing planes (0=Luma, 1=U, 2=V, 3=U&V, 4=Luma&U&V)
ncpu = default(ncpu, 1) #max number of CPU threads to use in FFT3D calculation (int>0, default=1)
thSAD = default(thSAD, 10000) #MV thSAD value (turn down to about 900 for additional artifact protection)
idx = default(idx, 1) #MV idx value
postsharp = default(postsharp, .5) #temporal min/max sharpening value (<= 1)

########## some 3D filtering, frequency based pre-sharpening
source=source.FFT3DFilter(sigma=sigma,sharpen=presharp,plane=plane,ncpu=ncpu)

########## get motion vectors and apply them
backward_vec1=source.MVAnalyse(isb=true, delta=1,pel=2,overlap=4,sharp=1,idx=idx)
forward_vec1 =source.MVAnalyse(isb=false,delta=1,pel=2,overlap=4,sharp=1,idx=idx)
bw1=source.MVCompensate(backward_vec1,thSAD=thSAD,idx = idx)
fw1=source.MVCompensate(forward_vec1, thSAD=thSAD,idx = idx)

########## local motion stabilization/degrain stage
inter=Interleave(bw1,source,fw1)
intersmooth=inter.Clense()
RestoreMotionBlocks(intersmooth,inter,neighbour=inter,alternative=inter,gmthreshold=100,
\ dist=1,dmode=0,debug=false,noise=10,noisy=15,grey=false,show=false)
SelectEvery(3,1)

########## temporal min/max sharpening stage (thanks Didée)
pmax=source.MT_Logic(bw1,"max").MT_Logic(fw1,"max")
pmin=source.MT_Logic(bw1,"min").MT_Logic(fw1,"min")
Sharpen(postsharp)
MT_Clamp(pmax,pmin,1,1,U=2,V=2)

output = last
return(output)
}

call with the default settings to best match TemporalDegrain defaults:

GTemporalDegrainFaster2()

thetoof -
to match your settings of TemporalDegrain, call like this:

GTemporalDegrainFaster2(sigma=8.5,plane=4)

-G

[thetoof]

Hmm... to my eyes, mc_spuds has not enough detail retention.

g-force, I must say I'm impressed by the quality/speed ratio of your script! However, it has strong haloing (which is normal with such high values sigma values... but I also used something similar for the reference clip of TemporalDegrain and it didn't have that much, so TD helped on that), it lacks a bit of sharpness and, most important of all, it's not as efficient on high motion scenes. TemporalDegrain (http://img100.imageshack.us/img100/9665/temporaldegrainfft3dfilqi9.jpg)/g-force (http://img521.imageshack.us/img521/1489/gtemporaldegrain2xe4.jpg)

Here (http://www.mediafire.com/?ibdgdn0ymrc)'s 30 frames to play with ;)

Have fun!

[thetoof]

For that much noise you are better off not using the post-sharpening, which you should know adds a lot of the grain back, because you (Didée) invented it!

Just noticed that... from what I know, sharpening after proper denoising helps giving the video a more detailed look and will not "add" some grain out of nowhere since it should have been well removed before sharpening.

[g-force]

Here (http://www.mediafire.com/?ibdgdn0ymrc)'s 30 frames to play with ;)

Have fun!

Great! looks like another day of not getting any work done! Seriously though, if I get it too much closer to what you are looking for it may end up as slow as TD. ;)

-G

[thetoof]

Actually, you improve in every new post, so I wouldn't say it's of another day not getting any work done ;) and the speed improvement is SO good compared to TD that the only reason I'm "pushing" you like this is that it looks very promising.

[g-force]

it should have been well removed before sharpening.

depends on if you prefer a little grain, or none at all. ;)

-G

[g-force]

Actually, you improve in every new post, so I wouldn't say it's of another day not getting any work done ;) and the speed improvement is SO good compared to TD that the only reason I'm "pushing" you like this is that it looks very promising.

That's very kind words. Seems like we're cross-posting!

-G

[thetoof]

Well, to my eyes, there are two types of grain: annoying grain that makes the clip a pain to watch and "gradient grain" that helps having nice transitions between colors (i.e. dark brown to light brown) and if you remove the grainy look of the source (like it was done in the VERY GRAINY thread for 300, where TemporalDegrain was created) and then use some sharpening to have an overall more detailed image, I find that type of grain it brings up quite nice ^_^

Actually, what made me want to post about the last quote was that that you said "adds back" the grain, but it's actually about enhancing "good grain"... well... imo :p

edit: hehe, seems like we are :p

[Terranigma]

Hmm... to my eyes, mc_spuds has not enough detail retention.


Which clip are you talking about? If you're saying that the calmed grain clip is less detailed than your clip, then you need your eyes checked, because it actually has more detail than the video you posted, and your colors is all wrong too.

[g-force]

Which clip are you talking about? If you're saying that the calmed grain clip is less detailed than your clip, then you need your eyes checked, because it actually has more detail than the video you posted, and your colors is all wrong too.

I actually think that "calmed grain" clip looks pretty nice. I think thetoof is looking for a bit more grain reduction than that from what I've deduced, so he probably meant that the "hate grain" clip has a lack of detail. I think the "hate grain" has about the same amount of detail as the TemporalDegrain clip that thetoof posted, my problem with it is how different parts of the window seem to bounce around not in relation to eachother.

I don't see what you mean by the color is all wrong, they look pretty similar to me in that aspect.

I have to admit I'm a bit impressed with mc_spuds!

-G

[Terranigma]

I think the "hate grain" has about the same amount of detail as the TemporalDegrain clip that thetoof posted, my problem with it is how different parts of the window seem to bounce around not in relation to eachother.

I don't see what you mean by the color is all wrong, they look pretty similar to me in that aspect.

I have to admit I'm a bit impressed with mc_spuds!

-G

Yes,the details were left intact, but grainless.
Now, I don't like the bouncing effect either, but don't you think that's due to not having any grain there to hide the bouncing effect? If you look at the original or the calm clip, you can see that it's doing the same thing, except it's not really that much of an issue since the grain is making it less annoying.

Now what I meant by the colors are wrong phrase, was that thetoof's lossless source color seems to be a bit darker than the source. You have to look closely. :p

[thetoof]

'bout the bouncing, I plan to use a stabilizer later on.

lossless source color seems to be a bit darker than the source
I must admit that I don't understand what you mean...

Ok, now regarding detail.
Frame 14 (http://img397.imageshack.us/img397/8420/fullho6.jpg):Original (http://img90.imageshack.us/img90/1259/original2pz1.jpg)/TD (http://img241.imageshack.us/img241/7089/tdnz0.jpg)/Calm (http://img509.imageshack.us/img509/5733/calmedws1.jpg)/Hate (http://img384.imageshack.us/img384/4263/hategk2.jpg)/TD+chroma (http://img253.imageshack.us/img253/2989/tdchromajm1.jpg)
Frame 0 (http://img187.imageshack.us/img187/369/2full2co5.jpg):Original (http://img89.imageshack.us/img89/6451/originaldk5.jpg)/TD (http://img257.imageshack.us/img257/4231/td2nd6.jpg)/Calm (http://img150.imageshack.us/img150/11/calmed2ye9.jpg)/Hate (http://img354.imageshack.us/img354/6236/hate2ey7.jpg)/TD+chroma (http://img523.imageshack.us/img523/3472/td2chromatv3.jpg)

First example: We can't see the wall anymore through his curl in Calm (for me = detail loss), but in Hate it's a lot better and I had to add chroma denoise after TemporalDegrain to get closer to it.

Second example: the lines on the suit have almost disappeared in Calm and, in Hate, they got wider (which lead to the visual disappearance of a few of them, especially on their right end) and the colors changed (now it's greyish).

Overall: MC_Spuds looks better regarding chroma noise, but in Calm, everything got blurred and, in Hate, subtle detail was lost.

[Terranigma]

'bout the bouncing, I plan to use a stabilizer later on.


I must admit that I don't understand what you mean...

Ok, now regarding detail.
Frame 14:Original (http://img90.imageshack.us/img90/1259/original2pz1.jpg)/TD (http://img241.imageshack.us/img241/7089/tdnz0.jpg)/Calm (http://img509.imageshack.us/img509/5733/calmedws1.jpg)/Hate (http://img384.imageshack.us/img384/4263/hategk2.jpg)/TD+chroma (http://img253.imageshack.us/img253/2989/tdchromajm1.jpg)
Frame 0:Original (http://img89.imageshack.us/img89/6451/originaldk5.jpg)/TD (http://img257.imageshack.us/img257/4231/td2nd6.jpg)/Calm (http://img150.imageshack.us/img150/11/calmed2ye9.jpg)/Hate (http://img354.imageshack.us/img354/6236/hate2ey7.jpg)/TD+chroma (http://img523.imageshack.us/img523/3472/td2chromatv3.jpg)

First example: We can't see the wall anymore through his curl in Calm (for me = detail loss), but in Hate it's a lot better and I had to add chroma denoise after TemporalDegrain to get closer to it.

Second example: the lines on the suit have almost disappeared in Calm and, in Hate, they got wider (which lead to the visual disappearance of a few of them, especially on their right end) and the colors changed (now it's greyish).

Overall: MC_Spuds looks better regarding chroma noise, but in Calm, everything got blurred and, in Hate, subtle detail was lost.

Been to an optometrist? From the images you posted, there's no details that was lost with hate.. there's just no grain, so you might be fooled into thinking that; with calm, more grain was kept than temporaldegrain.

Overall, you can't compare a simple script like temporaldegrain, to a much more complicated script such as mc_spuds; it's just not possible.

mc_spuds just needs some tweaking, but it's the better of the 2 scripts.

mc_spuds' competition would be that overly long complicated script that temporaldegrain was spun from.

[Didée]

Y'all are discussing wardrobe while the Titanic is sinking.

FixBlendIVTC(dclip=last.medianblur(2,2,2))
tdecimate(0,1,5)
setcache(8) # AvsTimer
temporaldegrain(degrain=3) # (with contrasharp extended to radius=2)

Linkydinky (http://www.mediafire.com/?wxx7myxdhs9)

edit: TemporalDegrain was changed like so: [...]
s = NR2.minblur(1,1)
allD = mt_makediff(o,NR2)
ssD = mt_makediff(s,s.removegrain(11,-1)).removegrain(20,-1) # Larger radius for VHS
ssDD = ssD.repair(ssD.repair(allD,1),1) # Larger radius for VHS
ssDD = ssDD.mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?")
[...]


BTW, did anyone note yet that this thread started about HD. Now we've reached VHS.

[Terranigma]

Y'all are discussing wardrobe while the Titanic is sinking.

FixBlendIVTC(dclip=last.medianblur(2,2,2))
tdecimate(0,1,5)
setcache(8) # AvsTimer
temporaldegrain(degrain=3) # (with contrasharp extended to radius=2)

Linkydinky (http://www.mediafire.com/?wxx7myxdhs9)


BTW, did anyone note yet that this thread started about HD. Now we've reached VHS.

lol, I needed a good laugh, and yes, I agree we veered way off topic. :p
I think fixblendivtc is what I may need to use for an ntsc source I have that seems to stay blended with every other method of ivtc i've tried. It's similar to the sample clip that was posted.

[thetoof]

Well, I don't think we were completely off topic... The script g-force is developping could be applied to HD and the approach he's using could still be described as some kind of "Temporal Degraining", so by giving more elements to compare with, it can help the development of his function. He's trying to get similar results than TD, but faster, which could help on HD sources. So, it's still about "suggestion for temporal degraining for HD contents"... imo ;)

And.. thanks for the deblender suggestion, but I already knew about it and it is in my final script for the source I posted. However, I'm very interested by the setcache function you used... I read the documentation, but I can't quite understand how you ended up with the value "8" for TemporalDegrain()... care to explain it to me? I just ran some tests and it really is faster!

@ Terranigma
I asked a few people around me which image they thought was visually better (blind test) and they all placed TD in first place. But, instead of having a "yes, no, yes, no, yes, no, my dad is stronger than yours" discussion, I suggest that we just leave it at that and we'll let those viewing this thread judge by themselves:)

[Didée]

Well, I got :rolleyes: when I saw that everybody happily threw the blended source at the various mocomp-denoisers. Blends are ill poison for motion compensation. When there is motion, a blended frame can not be successfully motion-compensated to a clean reference frame, and vice versa. Those frames are basically useless for mocomp-denoising. In practice, the motion engine either will manage to discard the blendings (resulting in less denoising), or it will manage to make some matches nonetheless, resulting in detail loss. In the case of blended telecine, for each frame to denoise there are *more* "bad neighbors" than "good" ones.
It's obvious that you first have to fix the blendIVTC, before doing (mo-comp'ed) denoising. Otherwise, everything is screwed from the start.

The caching thingy was added for safety reasons, it's not clear whether it's really needed or not. The combination of (conditional environment)+(decimating) can become unstable when "demanding" processing is done afterwards. So the aim in first place wasn't speed, but the hope to avoid strange problems. RequestLinear() would be the more natural choice for this issue, but it crashes on my PC at home ...
The point is that the downstream filter(s) should not make out-of-order frame requests from the FixBlendIVTC-TDecimate combo. Since TemporalDegrain(degrain=3) has to access 13 input frames for each frame to process (current, 3+3 backward, 3+3 forward), the cache should hold at least 13 frames, and my SetCache setting even was too small ...

---
P.S.: added the used diff for TemporalDegrain to my previous post.

[thetoof]

Thanks for that info 'bout mocomp... when you think about it, it's pure logic that blends will have a negative impact on the denoising efficiency.

[g-force]

Well, I got :rolleyes: when I saw that everybody happily threw the blended source at the various mocomp-denoisers.

Well, I can't say I was "Happy" about it, but since thetoof had 3 good frames in a row, and since GTDF2 uses only current, previous and next frame, there is no problem if you make sure to only look at the middle good frame. I figured thetoof would have it in his code to at least mitigate the blend.

Didée, maybe you can help me. GTDF2 could do more with more frames if I could obtain the median of 5 frames. I'd also be happy with the average of the 3 remaining pixels after the min and max have been thrown away.
I've written a median of 3 filter before using several combinations of MT_logic, MT_Average's and MT_lutxy's but it was nowhere near as fast as Clense().

-G

[Didée]

The MedianBlur plugin (by tsp) also has a temporal mode. To get the pure temporal median of a pixel over 5 frames, use spatial radii of 0 (i.e. no spatial operation) and a temporal radius of 2:

MedianBlurT(0,0,0,2)

Downside is that it works slooow ... with temp.radius=2, a "naked" MedianBlurT runs roughly at the same speed as MVDegrain2 does (including all the motion search etc.) ...

In general, using median filtering is a good idea for grain removal ... but it needs more safety checks and such, which slows things down even more ... with the only available temporal-median filter being so slow as it is, this isn't exactly the route to high-speed filtering of HD content. ;)

[Terranigma]

@ Terranigma
I asked a few people around me which image they thought was visually better (blind test) and they all placed TD in first place. But, instead of having a "yes, no, yes, no, yes, no, my dad is stronger than yours" discussion, I suggest that we just leave it at that and we'll let those viewing this thread judge by themselves:)

thetoof, it's pointless in trying to convince me to believe that temporaldegrain is better than mc_spuds. I haven't even begun to use mc_spuds to it's maximum potential. All I did was changed probably, 2-3 settings out of 20+.

Anyways, temporaldegrain is a faster version of this (http://forum.doom9.org/showthread.php?p=908950#post908950) filter chain by Didée with some tweaks here and there.
Maybe you can call it, TemporalUltimateStabilization :P

[g-force]

MedianBlurT(0,0,0,2)

Downside is that it works slooow ...

Thanks for the info Didée!
Wow, that is slow. Even with a temporal radius of 1, it's still WAY slower than Clense() (which is also a median filter with radius 1).

Noticed in the link that Terranigma posted that you have already tried this approach. I should have known. Oh well, maybe one day I'll get brave and learn to write plugins.

So, some questions about TD. I noticed that I was using an old version when I was looking at the idx values and saw that the second MV stage was wrong. I removed it since it wasn't doing anything anyway, and I then saw in the wiki that you fixed this. But, I was getting pretty decent results without it. So my first question is, how much does the second MV stage really help (I haven't tried it on very many different sources)?

Second question. The FFT filter is pretty quality, and it seems to me that if it was actually used for filtering the clip instead of just for obtaining the motion vectors, you could 1. offload some of the work from the MV stage (would require less temporal radius and hence speed up the script), 2. use the same idx values for both MVAnalyse and MVDegrain (again a speedup), 3. POSSIBLY get better motion estimate due to the fact that you would be forced to not over-blur the input to MVAnalyse. Do you have any thoughts as to the downsides of this approach? Just curious.

thanks,

-G

[Didée]

Even with a temporal radius of 1, it's still WAY slower than Clense() (which is also a median filter with radius 1).
Yay ... it's a pity that there's currently no "full-speed" implementation for such a *generic* filter operation.
With Clense, one has to keep in mind that with default settings it is not a generic median filter. Per default, it doesn't use the original input pixel of [previous_frame], instead it works recursively and uses the already processed [previous_frame] to get neighbor pixels from. That's okay for the static sections when Clense is applied directly; in the context of mo'comp/interleave/filter/selectevery it is not.
In older versions of clense (up to v0.9, for v1.0pre I'm not sure ATM) the recursion can be switched off by using Clense(reduceflicker=false). To switch it off also in the latest versions of the RemoveGrainT package, one has to explicitely assign the correct clip(s) to the recursion slots via Clense(previous=?,[next=?]), since the 'ReduceFlicker' parameter doesn't exist anymore.

So my first question is, how much does the second MV stage really help (I haven't tried it on very many different sources)?
The 1st stage does much more than the 2nd one, sure. The 1st removes the majority of grain an flicker, the 2nd one calms down most of the still remaining flicker. And of course, you have to see that with respect to how strong the source's grain is. When the grain is rather light, there's no need at all to use scripts that do so much effort ... that's shooting the butterfly on a wheel. On sources with light grain a basic MVDegrain is fully sufficient most times. Heck, often something simple like FluxSmooth does a good job there. It's the cases with strong grain where the big-effort scripts shine the most.


Second question. The FFT filter is pretty quality, and it seems to me that if it was actually used for filtering the clip instead of just for obtaining the motion vectors, you could 1. offload some of the work from the MV stage (would require less temporal radius and hence speed up the script), 2. use the same idx values for both MVAnalyse and MVDegrain (again a speedup), 3. POSSIBLY get better motion estimate due to the fact that you would be forced to not over-blur the input to MVAnalyse. Do you have any thoughts as to the downsides of this approach?
You mean, like in your script somewhere above where you did 80%-90% of the overall filter effect with an initial FFT3DFilter, and afterwards used a little appendix to do a little bit of mocomp-clensing? :D

The FFT3DFilter used in TD isn't what I'd call pretty quality. It's more like a sledgehammer to calm the source in order to get stable motion vectors. Generally, settings for FFT3DFilter that are strong enough to get the grainy source calm enough for motion search are not suited to be used for the actual output. OTOH, settings that are weak enough to be used for the actual output are not strong enough to make things calm enough for the motion search.

Specifically, I don't like FFT§DFilter too much for quality filtering of strong grain. See these examples (http://forum.doom9.org/showthread.php?p=1076275#post1076275) (and more obviously these zooms (http://forum.doom9.org/showthread.php?p=1076276#post1076276)) as for why. The left examples are from a filter chain that has FFT3DFilter as the central filter. There's much ringing, ugly texture echoes, but not enough detail retention. FFT3DFilter surely is a very good allround performer. But usually, a good allrounder isn't as good as a specialist for a certain task.

And most importantly, first of all I'm interested in getting a clean and straightforward method to get the most out of purely temporal motion compensated filtering. Tweaks and additional twists may come later. The first thing is to get the quirks out of the basic method (and there are many quirks left over, it would fill pages to mention them all), instead of just hitting with more and more filters before and more and evenmore filters after...
I've rather little interest in developping more of these

FunkyMixOfTwentyDifferentNoiseFilters( reason_for_combining = "it.just.happens.to.work.by.accident" )

functions ... anyone feel free to build such ones, each to his/her own. Some care about symptoms and how to mak'em go away; some care about the reasons for why there are symptoms.
(The formers are plenty, the latters are minor ... that's why we've almost no effectual antibiotics left over. We know better, but don't do so.)

[moviefan]

So after a wide discussion that exceeded a little bit the topic of this thread, I would like to come back to it and ask for a conclusion how I could degrain my 1080p source to in the first place make it as much compressible as possible with as little OBVIOUS quality/detail loss as possible. I don't care about grain that has some special effect for a movie like in e.g. 300 since in whatever movie I find it disturbing. So my primary target is to achieve the said degraining combined with my second (less important, but if possible) target to do this in a reasonable amount of time.

[Terranigma]

how I could degrain my 1080p source to in the first place make it as much compressible as possible with as little OBVIOUS quality/detail loss as possible. I don't care about grain that has some special effect for a movie like in e.g. 300 since in whatever movie I find it disturbing. So my primary target is to achieve the said degraining combined with my second (less important, but if possible) target to do this in a reasonable amount of time.

motion-compensated ttempsmooth, and for stabilizing the grain, use hqdn3d as a prefilter. No need for fft3d tbh with you.

[moviefan]

What do you exactly mean? Motion compensation using MVTools and between the interleave and the SelectEvery part the ttempsmooth?

[Terranigma]

What do you exactly mean? Motion compensation using MVTools and between the interleave and the SelectEvery part the ttempsmooth?

Yes, and the way you'd use prefilter this way, is by assigning different idx values for mvanalyse and mvcompensate.

[Didée]

The first thing to try is plain MVDegrain as shown in the MVTools documentation. Starting from there you've to decide whether that's enough, or if you need stronger filtering. (... and how much more time you're willing to give it if stronger filtering is needed.)

[g-force]

Didée

I am finally convinced of the benefits of not using the pre-filter as part of the filter chain. I had an idea though. What if the temporal radius 1 was mo-comped and temporal radius 2 wasn't, and then you threw away the min and max and averaged the rest? This should provide the same amount of degraining as a radius 2 mo-comp at least on low-motion areas.

Just a thought. Here is basically what the script would look like. Sorry for all the stuff that I have stolen from TD.

sigma = 16
source = last
filter = source.FFT3DFilter(sigma=sigma,sigma2=sigma*0.625,sigma3=sigma*0.375,sigma4=sigma*0.250,bt=5,bw=16,bh=16,ow=8,oh=8)
filterd = mt_makediff(source,filter)

b1vec = filter.MVAnalyse(isb=true, delta=1,pel=2,overlap=4,blksize=8,idx=1)
f1vec = filter.MVAnalyse(isb=false,delta=1,pel=2,overlap=4,blksize=8,idx=1)

bw2 = blankclip(source).Trim(0,1)+source #just use raw -2 frame
bw1 = source.MVCompensate(b1vec,thSAD=400,idx=2)
fw1 = source.MVCompensate(f1vec,thSAD=400,idx=2)
fw2 = source.Trim(2,0) #just use raw +2 frame

pmax_t = source.MT_Logic(bw1,"max").MT_Logic(fw1,"max").MT_Logic(bw2,"max").MT_Logic(fw2,"max")
pmin_t = source.MT_Logic(bw1,"min").MT_Logic(fw1,"min").MT_Logic(bw2,"min").MT_Logic(fw2,"min")
pavg = MT_Average(pmax_t,pmin_t)

inter = Interleave(bw2,bw1,source,fw1,fw2)
avg = inter.TTempSmoothF(maxr=2,lthresh=255,cthresh=255,strength=8,scthresh=255).SelectEvery(5,2)
nr1 = MT_lutxy(avg,pavg,"x 5 * y 2 * - 3 /") #subtract off min and max and average the rest

#don't do more than filter did
nr1d = MT_MakeDiff(source,nr1)
dd = MT_lutxy(filterd,nr1d,"x 128 - abs y 128 - abs < x y ?")

source.MT_MakeDiff(dd,u=2,v=2)

last.MVDegrain1(b1vec,f1vec,thSAD=400,idx=3) #second MV stage

# contra-sharpening by Didée
s = last.MinBlur(1,1) #requires Didée's minblur script
alld = MT_MakeDiff(source,last)
ssd = MT_MakeDiff(s,s.RemoveGrain(12,-1))
ssdd = ssd.Repair(alld,1)
ssdd = ssdd.MT_lutxy(ssd,"x 128 - abs y 128 - abs < x y ?")

last.MT_AddDiff(ssdd,U=2,V=2)

What do you think?

thetoof... wanna give this a try and report back?

-G

[thetoof]

RAM usage = 200 MB less than TD
Speed = about 9-12 times faster than TD

Results (http://www.megaupload.com/fr/?d=Q6QY4HXC) (15 frames with high motion + 45 frames with no motion and a scene change)

Too busy to post screenshots... will do later.

Good work!

[g-force]

RAM usage = 200 MB less than TD
Speed = about 9-12 times faster than TD

Results (http://www.megaupload.com/fr/?d=Q6QY4HXC)

Too busy to post screenshots... will do later.

thetoof,

thanks, I will download and look at the results.

might want to run those comparisons again. I noticed that the sharpening wasn't as good as what is in TD, so I updated my previous post.

-G

[Didée]

There's an error in the script: "trim(0,2)+..." needs to be "trim(0,1)+..." to get the wished 2-frame offset. (or more elegant: use "SelectEvery(0,-2)" & "SelectEvery(0,2)" for this kind of offsetting). The wrong offset throws the median-like correction a bit out of balance...

Correct that bug, and it's a good one!

[g-force]

There's an error in the script: "trim(0,2)+..." needs to be "trim(0,1)+..."

Correct that bug, and it's a good one!

Oops, that was a stupid mistake. Thanks for catching that!

Anyone have any suggestions for a name?

-G

[SPiKA]

Where can I find TemporalDegrain script?

[moviefan]

http://avisynth.org/mediawiki/Temporal_Degrain

[SPiKA]

Doh! xD
Thanks...

[thetoof]

You could call it SpeedyGonzalesDegrain :p

Here's a sample with the updated script. (http://www.mediafire.com/?21j212bb1mi)

[sidewinder711]

How do I use the last version of g-force's script to get the speed improvement thetoof has mentioned ? I tried SetMTMode(2), but the speed stays at 2-3 fps.


Greetz

[thetoof]

I get this kind of speed too (well, at single core with many programs running in background)... so I'll let you imagine how slow TD is :p

edit: You should also know that I'm using search=3 and degrain=3 with TD, as well as my own prefiltered reference clip with 2 FFT3DFilter calls (1 for chroma and 1 for luma with a noise pattern), so that's why "my" TD is very very slow. I used the same reference clip with g-force's script, but left all the other settings untouched, so maybe you won't get a speed boost as big as the one I got. Still, the results are very good and it's undoubtedly faster.

[sidewinder711]

thetoof,
thanks for your feedback. I'm only using the same script posted by g-force, no prog running in the background ... on a DualCore (2.2 Ghz) and get 2-3 fps. The same goes with the last version of TemporalDegrain for me (2-3 fps), which means that I don't see any speed advantage for g-force's version.

BTW: Thanks to Didée for his work... for me TD is the best solution to "tame" noisy vids. Yesterday i discovered that it even works perfectly on a blocked footage I tried to improve. :thanks:

Greetz

[g-force]

thetoof,
thanks for your feedback. I'm only using the same script posted by g-force, no prog running in the background ... on a DualCore (2.2 Ghz) and get 2-3 fps. The same goes with the last version of TemporalDegrain for me (2-3 fps), which means that I don't see any speed advantage for g-force's version.


That's pretty impressive speed for TD. Are you sure that you're comparing using TD with "degrain" set to 2? If so, maybe I just need a dual core!

Didée,
that would you think of moving the "don't do more than filter..." part to after the second MV stage, in the hopes of preventing more artifacting. Seems to work pretty well on the stuff I'm throwing at it, but wanted your feedback as to why you chose to do it before the second stage.

-G

[sidewinder711]

g-force
yep, default values for TD with degrain=2. If the afore mentioned speed is impressive for you, you should definitely go with a dual core! ;) I just made a test with a new E8400 dual core processor from intel (3.0 Ghz) and the speed of FastDegrain (i used it for a rip from a dvd) increased by more than 40% in comparison to my E6420.

Greetz

[thetoof]

That function is in bad shape, short of saying it were b0rked.

You mentioned of FastDegrain... imo you shouldn't use it.

[sidewinder711]

Thanks thetoof for your hint, but if I remember correctly, that this is an earlier quote made by Didée. He re-defined the FastDegrain function and posted it together with the latest TemporalDegrain to the mediawiki. Here (http://avisynth.org/mediawiki/Temporal_Degrain) you can find the versions I'm using.

Greetz

[Didée]

He re-defined the FastDegrain function and posted it together with the latest TemporalDegrain to the mediawiki
You are spreading false rumours. I fixed a bug in TemporalDegrain, but I did not touch FastDegrain in any way.

FastDegrain still is a technically correct script with frouzy defaults and too little parameters. Don't really care about it, since it offers nothing new anyway. It's just a wrapper-script for MVDegrainX().LimitedSharpen().HQDN3D() ... and at least for the MVDegrain part, there're morce configurable scripts out there.


@ g-force
that would you think of moving the "don't do more than filter..." part to after the second MV stage, in the hopes of preventing more artifacting. Seems to work pretty well on the stuff I'm throwing at it, but wanted your feedback as to why you chose to do it before the second stage
Put it wherever you want.:)
I'm sure that it works out with some sources ... most probably with those that would be sufficiently treated with just MVDegrain. For the more heavy sources, it won't. The pitfall is that FFT3DFilter is not generally sufficient, and it's not generally failsafe on high motion (which also depends heavily on blocksize and temporal depth and and and ...). From the viewpoint of a general procedure, I definetly wouldn't recommend to use it for limiting at the final stage.

Maybe later I'll show some cases where your script produces artifacts, and eventually expand a bit about the causes.

[g-force]

So, over the weekend I was playing around with some of the filters that Didée suggested might be suitable alternatives for the FFT prefilter in TD. I have to say they are quite good, but I was getting bad motion vectors in a few spots. I found an alternative that seems to work quite well (at least on sources with medium amounts of grain)

instead of the FFT filter call, use a filter like this:

source = last
temp=source.TemporalSoften(x,255,255,25,2) #x from 1 to 7
Repair(temp,source,mode=9)

so in TD, you would replace the second "filter" line with this:

filter = defined(denoise) ? denoise : filter.TemporalSoften(sigma,255,255,25,2).Repair(o,mode=9)


It's much faster and seems to produce less artifacts than the FFT when used in TD.

-G

[Spuds]

What kind of artifacts are you seeing with fft3dfilter? In the script the denoised clip is used to search for vectors and then limit the allowed change to the original, its output is not really directly applied to the final image, just curious what you are seeing.

Perhaps changing the script so spat allowed less change to the original would prevent the artifacts, or over denoising, of the clip. Be nice to allow passing a spat clip and not having it tied directly to the filtered clip.


filter = defined(denoise) ? denoise : filter.TemporalSoften(sigma,255,255,25,2).Repair(o,mode=9)

I don't think you want to use sigma there, or if you do you need to pass a new value as the default is 16. You could make it a child of the degrain variable like =2*degrain.

Of course since the script does allow you to pass a denoised clip you could just call it as below instead of having YATDmod.

filtered = last.TemporalSoften(4,255,255,25,2).Repair(last,mode=9)
TemporalDegrain(last,denoise=filtered)

[Nikos]

@g-force
Me too, i was playing around with some of the Didée's filters :)
The FFT sometimes with high sigmas produce ugly artifacts in the prefilter clip.

For prefiltering try this old Didée's function:

# >> LimitedTemporalSoften <<
# A function by Didée
#
# Soften bigger noise by TemporalSoften(),
# while still minimize ghosting.
#
#
# Full de-noising is only achieved with weak noise, but it IHMO does a pretty
# good job in "calming down" any sort of noise, before handing the clip to any final denoiser(s).
#
# Stronger pre-blurring. The pre-blurring is only a rough spatial cleaner to get a
# better measurement from the full, unrestricted temporal blur. For strong noise,
# replace that "blur(preblur)" with something like
# "bicubicresize([33%~~66%],.33,.33).bicubicresize([100%],1.0,0.0)".
# Finding the "sweetspot of preblurring" is the main trick.
#
#
# blur() -> removegrain(11) is no problem.
# fade does a funky kind of soft thresholding.
# It's value is not intuitive, and even is inverted (smaller 'fade' -> stronger filtering).
#

function LimitedTemporalSoften(clip clp, int "frames", int "limit",
\ float "preblur", float "fade", bool "color")
{
frames = default(frames, 1)
limit = default(limit, 2)
preblur = default(preblur,1.0)
fade = default(fade, 4.0)
color = default(color, true)
limstr = string(limit)
fadestr = string(fade)
uv = color ? 3 : 2

spat = (preblur==0.0) ? clp : clp.blur(preblur)
temp_spat = spat.TemporalSoften(frames, 255, 255, 255, 2)
temp = clp .TemporalSoften(frames, 48, 48, 32, 2)

Diff_SpatTemp = mt_lutxy(spat, temp_spat, Yexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Uexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Vexpr="255 "+limstr+" * x y - abs "+fadestr+" * /",U=uv,V=uv)

out = mt_merge(clp, temp, Diff_SpatTemp, Y=3, U=uv, V=uv)

return(out )
}

Try:
LimitedTemporalSoften(frames=2, limit=4, fade=3)
and you will have more speed and less blurriness than your TemporalSoften and Repair combo.

If your video have flicker, or "dancing" noise try this Didée's script before LimitedTemporalSoften:

o = last
f = o.MinBlur(1,2).MinBlur(2,2).RemoveGrain(11,-1)
f.FluxSmoothT(7).mt_AddDiff(mt_MakeDiff(o,f,U=2,V=2),U=4,V=4)

# eventually, limit the maximum pixel change to +/- 2
mt_LutXY(o,last,"x 2 + y < x 2 + x 2 - y > x 2 - y ? ?",U=2,V=2)

# to compare:
# interleave(o,last)

return(last)

The result is great.

[g-force]

What kind of artifacts are you seeing with fft3dfilter? In the script the denoised clip is used to search for vectors and then limit the allowed change to the original, its output is not really directly applied to the final image, just curious what you are seeing.

Perhaps changing the script so spat allowed less change to the original would prevent the artifacts, or over denoising, of the clip. Be nice to allow passing a spat clip and not having it tied directly to the filtered clip.



I don't think you want to use sigma there, or if you do you need to pass a new value as the default is 16. You could make it a child of the degrain variable like =2*degrain.

Of course since the script does allow you to pass a denoised clip you could just call it as below instead of having YATDmod.

filtered = last.TemporalSoften(4,255,255,25,2).Repair(last,mode=9)
TemporalDegrain(last,denoise=filtered)


Spuds,

I agree with everything you are saying. I just left it "sigma" so people could pop it in without changing any of the variables to TD, but both of your methods are more elegant.

As for the artifacts I was getting, there are a couple scenes where large objects pass in front of the camera, and the background gets distorted (warped) on the frame where they are still half in the picture. I'll try to post some screenshots when I get home. If I look at the direct output of the FFT filter, I can see how the MV analysis might get confused. Anyway, just posing yet another possibility of a replacement for the slow FFT filter. Maybe someone will find it useful.

-G

[g-force]

@g-force
Me too, i was playing around with some of the Didée's filters :)
The FFT sometimes with high sigmas produce ugly artifacts in the prefilter clip.

For prefiltering try this old Didée's function:

# >> LimitedTemporalSoften <<
# A function by Didée
#
# Soften bigger noise by TemporalSoften(),
# while still minimize ghosting.
#
#
# Full de-noising is only achieved with weak noise, but it IHMO does a pretty
# good job in "calming down" any sort of noise, before handing the clip to any final denoiser(s).
#
# Stronger pre-blurring. The pre-blurring is only a rough spatial cleaner to get a
# better measurement from the full, unrestricted temporal blur. For strong noise,
# replace that "blur(preblur)" with something like
# "bicubicresize([33%~~66%],.33,.33).bicubicresize([100%],1.0,0.0)".
# Finding the "sweetspot of preblurring" is the main trick.
#
#
# blur() -> removegrain(11) is no problem.
# fade does a funky kind of soft thresholding.
# It's value is not intuitive, and even is inverted (smaller 'fade' -> stronger filtering).
#

function LimitedTemporalSoften(clip clp, int "frames", int "limit",
\ float "preblur", float "fade", bool "color")
{
frames = default(frames, 1)
limit = default(limit, 2)
preblur = default(preblur,1.0)
fade = default(fade, 4.0)
color = default(color, true)
limstr = string(limit)
fadestr = string(fade)
uv = color ? 3 : 2

spat = (preblur==0.0) ? clp : clp.blur(preblur)
temp_spat = spat.TemporalSoften(frames, 255, 255, 255, 2)
temp = clp .TemporalSoften(frames, 48, 48, 32, 2)

Diff_SpatTemp = mt_lutxy(spat, temp_spat, Yexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Uexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Vexpr="255 "+limstr+" * x y - abs "+fadestr+" * /",U=uv,V=uv)

out = mt_merge(clp, temp, Diff_SpatTemp, Y=3, U=uv, V=uv)

return(out )
}

Try:
LimitedTemporalSoften(frames=2, limit=4, fade=3)
and you will have more speed and less blurriness than your TemporalSoften and Repair combo.

If your video have flicker, or "dancing" noise try this Didée's script before LimitedTemporalSoften:

o = last
f = o.MinBlur(1,2).MinBlur(2,2).RemoveGrain(11,-1)
f.FluxSmoothT(7).mt_AddDiff(mt_MakeDiff(o,f,U=2,V=2),U=4,V=4)

# eventually, limit the maximum pixel change to +/- 2
mt_LutXY(o,last,"x 2 + y < x 2 + x 2 - y > x 2 - y ? ?",U=2,V=2)

# to compare:
# interleave(o,last)

return(last)

The result is great.

Nikos,

cool! I'll try those out.

-G

[Spuds]

LimitedTemporalSoften(frames=2, limit=4, fade=3)
and you will have more speed and less blurriness than your TemporalSoften and Repair combo.

Speed I'll agree with (along with less noise removal), but less blurriness? I hardly think so. You can't expect a filter that cleans only by smoothing to have less blurriness (softness), it selectively smooths and does so very well.

Look into his eye's, they are loosing life with limitedtemporalsoften

http://img266.imageshack.us/img266/952/ltsce5.th.jpg (http://img266.imageshack.us/my.php?image=ltsce5.jpg)http://img266.imageshack.us/img266/728/tdwdntk1.th.jpg (http://img266.imageshack.us/my.php?image=tdwdntk1.jpg)http://img266.imageshack.us/img266/8093/tdpl6.th.jpg (http://img266.imageshack.us/my.php?image=tdpl6.jpg)http://img236.imageshack.us/img236/6302/tdtsip2.th.jpg (http://img236.imageshack.us/my.php?image=tdtsip2.jpg)

[g-force]

Speed I'll agree with (along with less noise removal), but less blurriness? I hardly think so. You can't expect a filter that cleans only by smoothing to have less blurriness (softness), it selectively smooths and does so very well.

Look into his eye's, they are loosing life with limitedtemporalsoften



Nikos,

are you advocating using LTS instead of TD, or just as the replacement for the FFT portion of TD?

Spuds,

judging by your response, I take it that you think he is looking to replace all of TD with LTS, which is how I read it too, but just wanted to make sure. Also, did you use my suggested filter as the input to TD in the middle picture?

-G

[Nikos]

@Spuds and g-force i recommended limitedtemporalsoften only for prefiltering stage in contrast the temporalsoften+repair combo :)

From my previous post:

For prefiltering try this old Didée's function:
......

[Spuds]

As for the artifacts I was getting, there are a couple scenes where large objects pass in front of the camera, and the background gets distorted (warped) on the frame where they are still half in the picture. I'll try to post some screenshots when I get home. If I look at the direct output of the FFT filter, I can see how the MV analysis might get confused. Anyway, just posing yet another possibility of a replacement for the slow FFT filter. Maybe someone will find it useful.

Cool, just wanted to get an example of what type of scene would cause the erroneous vectors and how those would then manifest on the output. Given that sigma=16 as a default I'd expect some prefilter distortion. Yup FFT3dfilter is slow especially when you lower the blocksize and increase bt as is done in temporaldegrain. You can gain a lot of speed back with bt=1 and blocksize=32 for less noisy sources.

i recommended limitedtemporalsoften only for prefiltering stage Thanks, it was the LimitedTemporalSoften(frames=2, limit=4, fade=3)
and you will have more speed and less blurriness than your TemporalSoften and Repair combo. that made it sound like it was a full replacement. Now if it is intended as the prefilter then I'll let you off on the less blurry part (maybe), but now how are two calls to temporalsoften, a lutxy plus a merge faster than a single temporalsoften + repair :). Fast yes, faster?

Also, did you use my suggested filter as the input to TD in the middle picture The third one was your prefilter passed via denoise. For completeness I added a 4th view using what Nikos suggested.

[g-force]

The third one was your prefilter passed via denoise. For completeness I added a 4th view using what Nikos suggested.

Nice! Results look pretty similar, although it appears that the "don't do more than" part is keeping a little more noise in Nikos' suggestion than in my suggestion. I think these are both pretty decent alternatives. Maybe worth updating the TD Wiki entry to include these as well as Didée's suggestions from page one of this thread as faster alternatives to the internal FFT filter.

-G

[Nikos]

From a quick test with VirtualDub's "Run Video analysis pass" with this script:

SetMTMode(5,4)

SetMemoryMax(256)
DirectShowSource("BSG.GRF", audio=false, seek=true, fps=23.976023976023976023976023976024, framecount=136152)

trim(45295, 46265)

SetMTMode(2)

Spline36Resize(1280,720,4,4,1912,1072)

my.prefilter()

a. fft3dfilter(sigma=4), 15 fps

b. TemporalSoften(2, 255, 255, 25, 2) + Repair, 23.25 fps

c. LimitedTemporalSoften(frames=2, limit=4, fade=3), 29.5 fps

CPU: Intel Q6600, 2400 MHz

I think that we need a Didee's suggestion for an alternative fast prefilter in Temporal Degrain :)
The LimitedTemporalSoften alone is week for very grainy sources.

Unfortunately, in my opinion every source need a different prefilter.

This is my prefilter chain for noisy sources: [Thanks Didee for the teach :)]


source=last
# "calming down" any sort of noise
pre1=source.LimitedTemporalSoften(frames=2, limit=3, fade=3)

# FFT with moderate sigma values
SetMTMode(5)
pre2=pre1.FFT3Dgpu(sigma=..., sigma2=..., sigma3=..., sigma4=..., plane=0, mode=1, precision=2, bt=3, wintype=1, bw=48, bh=48, ow=16, oh=16)
SetMTMode(2)

# Eventualy limit the FFT strength to fit my needs
NRL= some.value
pre=mt_lutxy(pre1, pre2, "x "+NRL+" + y < x "+NRL+" + x "+NRL+" - y > x "+NRL+" - y ? ?", chroma="copy first")
# or
# pre=mt_lutxy(pre1, pre2, "x "+NRL+" + y < x "+NRL+" + x "+NRL+" - y > x "+NRL+" - x 51 * y 49 * + 100 / ? ?", chroma="copy first")


It's fast because i utilize the GPU, and effective with rights sigma values.

[Spuds]

@Nikos,

Those are some non-intuitive results, not doubting what you got its just they don't, on the surface, make sense. For kicks I ran a short pass as well on my laptop with a very basic script to take other variables out of the loop. Also unloaded vdub and avisynth after each pass to avoid favorable cache hits.

avisource("test.avi") # progressive source in yv12 720X480
trim(0,1000) # lets not spend all day on this
test filter a,b,c

a = fft3dfilter(sigma=4) # 17 fps
b = LimitedTemporalSoften(frames=2, limit=4, fade=3) # 60 fps
c = TemporalSoften(4,255,255,25,2).Repair(last,mode=9) # 77 fps

[moviefan]

@Spuds: With reference to the latest development of this thread: Wouldn't the improved prefilter chain also be beneficial to your mcspuds script?

[Spuds]

@moviefan

Oh yes, I'm always on the lookout for a clean, fast, detail saving, noise removing prefilter :) The 'standard' one in mc_spuds is relatively slow but effective, its based on fft3dfilter.

The prefilter that prefast=true invokes in the latest beta is about 57fps, just a tad slower then limitedtemporalsoften and works well on not so noisy sources. Of course there is always room for improvement !

[Nikos]

Hi Spuds, i try for prefiltering the RemoveGrain and FluxSmoothT combo with limiter in each stage.

clp=last

s = clp.RemoveGrain(mode=xx, modeU=2)
spat = mt_lutxy(clp, s, "x "+NLS+" + y < x "+NLS+" + x "+NLS+" - y > x "+NLS+" - y ? ?", Y=3, U=3, V=3)

st=spat.FluxSmoothT(y)
prefilter=mt_lutxy(spat, st, "x "+NLT+" + y < x "+NLT+" + x "+NLT+" - y > x "+NLT+" - y ? ?", Y=3, U=3, V=3)

It's fast and effective.

My question is:
The logical order is spatial first temporal then
or temporal first spatial then?

In my eyes the results are identical.