Temporal or spatial? Which is more important cleaning anime?

[moita]

Hi everyone

I'm capturing anime from a noisy analog signal. In the recent days I have read a lot through this forum, but it is impossible to read everything, so this may very well have been discussed elsewhere in the forum.

For anime what type of cleaners does the job better? Temporal or spatial? I want to use both (or spatial-temporal), but I want a good starting point to start messing with it. Also, what is the best filter order (spatial, temporal and spatial-temporal).

Note that my sources are PAL interlaced and I want to keep them interlaced (I will reencode the captures to MPEG2 to wath on TV).

At the moment I have tried convolution3d(animeBQ) and the results are good compared to the source.

[ObiKenobi]

Originally posted by moita
For anime what type of cleaners does the job better? Temporal or spatial? I want to use both (or spatial-temporal), but I want a good starting point to start messing with it. Also, what is the best filter order (spatial, temporal and spatial-temporal).

Both are good to use. This (http://forum.doom9.org/showthread.php?s=&threadid=51181) thread has a pretty long list of denoisers and with a search you can find out about them.

[moita]

Thanks or your reply. I have reading along that thread. But it is too long and I think that it does not cover the topic I need.

From what I know (correct me if wrong), a temporal cleaner will compare the surrounding frames for similarities and use these similarities to clean the noise. A spatial cleaner will use only the information in the frame. I think that for anime temporal cleaners will do most of the job since there are many still frames or similar frames in an anime clip.

If that is true, then one would want to clean most of the noise with a temporal cleaner (hard settings) and then clean the remaining noise with a spatial cleaner (soft settings to prevent blurring). Following this reasoning, the filter order should be temporal -> spatial. Am I right?

By the way, meanwhile I have been playing around with overlay + convolution3d and the results are really amazing. However this cannot be my ultimate solution to clean the noise since I do not have two sources of all the materials.

[ObiKenobi]

I don't really know if there is any set order to use, but with Deen (the denoiser I use most) I usually use the 2d denoiser followed by the 3d denoiser and get good results that way. You might also look into DeGrainMedian which is good for grainy sources as in this (http://forum.doom9.org/showthread.php?s=&postid=570017#post570017) example.

[moita]

A temporal cleaner will clean the noise in moving areas of the picture, while a spatial cleaner will denoise in static areas of the picture.

If you are sure about that, then I have to review what I know about temporal vs spatial cleaners.

I will give a try to Deen and DeGrainMedian. But my source is by far noisier than the one of the link you provided.

[ObiKenobi]

Originally posted by moita
I will give a try to Deen and DeGrainMedian. But my source is by far noisier than the one of the link you provided.

Then just bump up the strength of Deen.

[moita]

Thanks ObiKenobi. The last link you provided has good scripts as a starting point.

[ObiKenobi]

You're welcome. If you have any more questions or needs of suggestions, just ask.

[stickboy]

Originally posted by ObiKenobi
A temporal cleaner will clean the noise in moving areas of the picture, while a spatial cleaner will denoise in static areas of the picture.That is almost backwards.

Purely temporal cleaners work by blending frames together. Clearly this works best for static frames which have no motion; then only noise gets averaged out.

Purely spatial cleaners have no concept of motion, because by definition they don't look at time. Therefore they affect both static and non-static areas.

[moita]

@stickboy
So my reasoning is (more or less) correct, isn't it?

[ObiKenobi]

Sorry moita I meant to agree with what you said, I just worded it badly and it came out completely wrong from what I meant. Yes a temporal cleaner works on the difference between the frames to determine what is noise, while the spatial does only look at the noise within a frame.

[stickboy]

Originally posted by moita
So my reasoning is (more or less) correct, isn't it?Sounds right to me.

I typically apply a temporal cleaner before a spatial one, especially on animation.

However, be careful about doing too much temporal smoothing; if the settings are too harsh, backgrounds with soft gradients will come out a smeared mess in pans.

[scharfis_brain]

if the settings are too harsh, backgrounds with soft gradients will come out a smeared mess in pans.

that's why I always use depan.dll's depaninterleave()
to do some kind of global motion compensation.

It helps a LOT on pans

[Didée]

Another little function that often can help much is LimitedTemporalSoften (http://mf.onthanet.com/forum/index.php/topic,161.msg1218.html#msg1218). Full de-noising is only achieved with weak noise, but it IHMO does a pretty good job in "calming down" any sort of noise, before handing the clip to any final denoiser(s).

[Dreassica]

Hmm i get this error when using the limitedtemporalsoften script snippet.

http://www.geocities.com/majinbubbles2001/error.JPG

[Didée]

That's because that board engine for some reasons removes the backslash line breakers. As exchange, scripts automatically get a nice colored dress.

# >> LimitedTemporalSoften <<
#
# Soften bigger noise by TemporalSoften(),
# while still minimize ghosting.
#
# A function by Didée.
#
# Needs MaskTools.dll >= v1.5.6


function LimitedTemporalSoften(clip clp, int "frames", int "limit",
\ float "preblur", float "fade", bool "color")
{
frames = default( frames, 1 )
limit = default( limit, 2 )
preblur = default( preblur,1.0 )
fade = default( fade, 4.0 )
color = default( color, true)
limstr = string(limit)
fadestr = string(fade)
uv = color ? 3 : 1

spat = (preblur==0.0) ? clp : clp.blur(preblur)
temp_spat = spat.TemporalSoften(frames,255,255,255,2)
temp = clp .TemporalSoften(frames,48,48,32,2)

Diff_SpatTemp = yv12lutXY(spat,temp_spat,Yexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Uexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Vexpr="255 "+limstr+" * x y - abs "+fadestr+" * /",U=uv,V=uv)

out = MaskedMerge(clp,temp,Diff_SpatTemp,Y=3,U=uv,V=uv)

return( out )
}


edit: updated for today released MaskTools v1.5.6 (http://manao4.free.fr/)

[Dreassica]

Ok taht did teh trick, thnx. Seems i got allot of those weird shit lately. :(

[moita]

It is offtopic but I do not want to abuse and crete 3 new threads in less than 24 hours. As I stated ealier, I've tried the overlay method of two different sources of the same clip followed by convolution3d. The results are realy amazing. Now I wonder if one overlay 2 sources and get good results, what about overlaying 3 or 4 different sources (rortunately, I'm able to get 4 different copies of the same clip). How can I do that?

[Soulhunter]

Should work like this...

A=AviSource("C:\Clip1.avi")
B=AviSource("C:\Clip2.avi")
C=AviSource("C:\Clip3.avi")
D=AviSource("C:\Clip4.avi")

Blah1=Overlay(A,B,mode="blend",opacity=0.5)
Blah2=Overlay(C,D,mode="blend",opacity=0.5)

Overlay(Blah1,Blah2,mode="blend",opacity=0.5)


EDIT: Fixed this "opacity" typo... ;)


Bye

[stickboy]

Not quite. Overlay by default gives the overlay clip 100% opacity. You need to use:Blah1 = Overlay(A, B, opacity=0.5) # mode="blend" by default
# ... etc. ...If the number of clips you want to blend together is a power of two, it should be obvious what to do. Blend pairs of clips, and then blend pairs of the pairs, and so on, using 50% opacity each time. Everything at the end will have equal weighting.

If the number of clips is not a power of two, then you need to adjust the opacities to avoid bias. Think about what weights each clip will have in the end and think inductively.

Suppose you have N clips, and you already have successfully blended clips 1..(N-1) with equal weights of 1/(N-1). How can you add clip N to this?

With N clips, each clip should end up with a weight of 1/N. Therefore we simply give clip N an opacity of 1/N.

(This means that clips 1..(N-1) collectively contribute a net weight of (1 - 1/N) = (N-1)/N. Since we assumed that clips 1..(N-1) had equal weights, each of them must end up with a weight of (N-1)/N * 1/(N-1) = 1/N, as desired.)

Okay, that probably was pretty confusing, so here's an example: if you have five clips, A, B, C, D, E:
AB = Overlay(A, B, opacity=0.5) # Each of {A, B} has 1/2 weight
ABC = Overlay(AB, C, opacity=(1.0 / 3)) # Each of {A, B, C } now has 1/3 weight
ABCD = Overlay(ABC, D, opacity=0.25) # ... 1/4 weight
ABCDE = Overlay(ABCD, E, opacity=0.2) # ... 1/5 weight

[Soulhunter]

@ stickboy

Ooops, totally forgotten to add the opacity params... :D


Bye

[Nikos]

Is this the correct way to convert the LimitedTemporalSoften for use with MaskTools ver. 2.0?

# >> LimitedTemporalSoften <<
#
# Soften bigger noise by TemporalSoften(),
# while still minimize ghosting.
#
# A function by Didée.
#
# Needs MaskTools.dll >= v1.5.6


function LimitedTemporalSoften(clip clp, int "frames", int "limit",
\ float "preblur", float "fade", bool "color")
{
frames = default( frames, 1 )
limit = default( limit, 2 )
preblur = default( preblur,1.0 )
fade = default( fade, 4.0 )
color = default( color, true)
limstr = string(limit)
fadestr = string(fade)
uv = color ? 3 : 1

spat = (preblur==0.0) ? clp : clp.blur(preblur)
temp_spat = spat.TemporalSoften(frames,255,255,255,2)
temp = clp .TemporalSoften(frames,48,48,32,2)

Diff_SpatTemp = mt_lutxy(spat,temp_spat,Yexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Uexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Vexpr="255 "+limstr+" * x y - abs "+fadestr+" * /",U=uv,V=uv)

out = mt_merge(clp,temp,Diff_SpatTemp,Y=3,U=uv,V=uv)

return( out )
}

What's exactly doing the fadestr?
Also may i replace the TemporalSoften with TTempSmooth and the blur(1.0) with RemoveGrain(mode=11) or mode=12?

[Didée]

I'd rather suggest to completely forget about LimitedTemporalSoften, and instead just use ttempsmooth with an approprietly filtered clip for ttemp's pfclip parameter ...

[Nikos]

Didée i know your scripts with ttemp and pfclip but i have a friend who want a very quick script for moderate quality tv-capture and the above script it's quick, simple and good.

Is the above script correct for MaskTools ver 2.0? :)

Also may i replace the blur(1.0) with RemoveGrain(mode=11) or mode=12?

[Didée]

Well, then.

The script is correct. What could go wrong by replacing yv12lutxy -> mt_lutxy ... ?
blur() -> removegrain(11) is no problem.
temporalsoften -> ttempsmooth is not good since speed is your argument.
'fade' does a funky kind of soft thresholding. It's value is not intuitive, and even is inverted (smaller 'fade' -> stronger filtering).

That uv line better should read "uv = color ? 3 : 2", don't know why I once had set it to 1 ...

[Zanejin]

temporalsoften -> ttempsmooth is not good since speed is your argument.

Might TTempSmoothF bring the filter up to the speed of TemporalSoften?

[Nikos]

Thanks Didée for the quick answer and the explain.
In my opinion this function does a good job in calming down any sort of noise with minimal blurriness and is fast :)

I like to study your old and new functions .
I don't like the chains of dumb filters :)