How to denoise this?

[SPiKA]

I'm currently using trim + dfttest(sigma=5) to denoise just the heavily noised scenes... but it kills way to many detail.
Any suggestions?

Don't worry about the "clean" scenes... since I'm using light filtering on them.

Source:
http://img107.imageshack.us/img107/6141/84640386kk1.png

http://img107.imageshack.us/img107/3214/91199879um8.png

http://img89.imageshack.us/img89/4416/64240147js8.png

Chopped footage:
http://www.mediafire.com/?n06a2c1n4o3

[Terranigma]

tnlmeans? That would be my number 1 choice for anime, but if that's too slow, you can try mvdegrain with dct 3 and maybe a search radius of 30+.

[SPiKA]

tnlmeans? That would be my number 1 choice for anime, but if that's too slow, you can try mvdegrain with dct 3 and maybe a search radius of 30+.

Speed isn't a problem. Quality is... :p
Can you recommend me settings for it?

[martino]

Deen and BlindPP! No, I'm not joking. :)

BlindPP(14,6)
Deen()
Not tuned, etc...

It's what I used last time on similarly looking artefacts and worked for me, but there may be a better, "newer" solution.

Oh yeah... by the way, is there a technical name for that blocking and a reason why it happens since we're on this subject already?

[Terranigma]

Speed isn't a problem. Quality is... :p
Can you recommend me settings for it?

Default settings?
By default though, there's no temporal operation going on, so if you want to do temporal filtering, then configure the AZ parameter. The higher, the better the results & slower the processing (much slower actually :p). AZ=5 would be equaled to that of bt=5 in fft3dfilter. Also, since the source in question is indeed a toon/anime based source, set sse to false since sad usually works better with these types of sources. The "h" parameter would be the strength of the filtering, but you may want to start with 0.1 and adjust it as needed from there, since 0.1 in sad mode would be like, 0.45 in sse mode (which may prove to be too strong in some cases).

So all in all, try the default settings with the exception of configuring the "AZ" parameter and using SAD instead of SSE. :)

[SPiKA]

Something like this?

tnlmeans(ax=3,ay=3,az=5,sx=2,sy=2,bx=1,by=1,sse=false)

[Terranigma]

Something like this?

tnlmeans(ax=3,ay=3,az=5,sx=2,sy=2,bx=1,by=1,sse=false)

Yes. Now tune the strength (h) parameter to your liking
(az=5 wouldn't probably help that much unless you plan on using tnlmeans as a prefilter, or with 5 frames compensated, so you're probably better off using az=1.)

[SPiKA]

I though that MVDegrain3 was already ubber slow... but this (especially with az=5) is SLOOOW in a whole new dimension! :p

[cestfait]

first diagnose the problem. this doesn't need more denoising that mvdegrain3. what that needs is deblocking. try "aaa," my preferred anti aliaser, which has always served me better than a deblocker:

function AAA(clip clp, int "Xres", int "Yres", int "Xshrp", int "Yshrp",

\ int "US", int "DS", bool "chroma")

{

clp = clp.isYV12() ? clp : clp.ConvertToYV12()

ox = clp.width

oy = clp.height

Xres = default(Xres, ox)

Yres = default(Yres, oy)

us = default(us, 1)

ds = default(ds, 2)

Xshrp = default(Xshrp, 0)

Yshrp = default(Yshrp, 0)

chroma = default(chroma, false)



us==0 ? clp.PointResize(ox*2,oy*2) : clp.LanczosResize(ox*2,oy*2)



TurnLeft()

SangNom()



TurnRight()

SangNom()



ds==0 ? BilinearResize(Xres,Yres) :

\ ds==1 ? BicubicResize(Xres,Yres) :

\ LanczosResize(Xres,Yres)



Unfilter(Xshrp,Yshrp)



chroma ? MergeChroma(clp.Lanczosresize(Xres,Yres)) : last



}

I've modified the defaults a bit, so just aaa() should work fine. you'll need sangnom and unfilter.

i'm sure I can trust you to sharpen up afterwards!

[Manao]

You can try somthing like at_denoise(sstep = 0.4, edge = 3.0, iterations = 10, center = 1, tstep = 0) - it's in anisotools, in my signature.

[SPiKA]

tnlmeans(ax=3,ay=3,az=5,sx=2,sy=2,bx=1,by=1,h=2,sse=false)
http://img137.imageshack.us/img137/4508/tnlwl5.png

aaa()
http://img377.imageshack.us/img377/8815/aaaue0.png

at_denoise(sstep = 0.4, edge = 3.0, iterations = 10, center = 1, tstep = 0)
http://img89.imageshack.us/img89/6811/atmq8.png

dfttest(sigma=5)
http://img377.imageshack.us/img377/3726/dftrx2.png

[cestfait]

pick the one you think looks best MOVING. You could also just try normal x264 deblocking settings.

[SPiKA]

TNLMeans worked the best... thanks to everybody for your help.

[thetoof]

To my eyes, what gives the best result on your clip is aaa() + some light denoiser (like dfttest with a low sigma), because the main problem seems to be, as cestfait said, blocking.

However, "The best is what works best for you!". :p

[SPiKA]

In fact, I ended up using TNLMeans (with lower strength) + aaa().

[cestfait]

good! It's always satisfying to be decisive. I do, however agree with thetoof. your tnlmeans in a scene like that is not going to help compressibility much (still high contrast edges shifting every frame), and x264's deblocker/quantization in general should smooth away the worst of the artifacts left over.

there can be no TRUE perfectionism here-- don't waste your time filtering every blocky scene to kingdom come-- you are starting with a flawed source.

[cestfait]

on one other note: another sort of cheater's solution to blocking is just spline resizing the frame to double-size and then back down again. it takes advantage of spline-interpolation's idea of what the frame is supposed to look like. a sort of crude method of supersampling.

doesn't always work perfectly, of course!