Going crazy with this anime denoise

[mp3dom]

Hi to all!
I'm going crazy in denoising the anime "Beck". The source of this anime is pretty ugly because there are a lot of scenes where the noise isn't "regular" and constant... so for example in the same scene there's zone with a lot of noise and zone with no (or very low) noise.
I've tried various denoiser (DegrainMedian, RemoveDust, FFT3DFilter, Convolution etc) and in "general" the best results I've obtained is with Convolution+FFT3DFilter but with this combination there are some scenes that makes color banding more visible or strange chroma "weaves" or artifacts. The following is the script that I've used:


MPEG2Source("beck.d2v") # YUY2 Colorspace
Convolution3D(0, 6, 10, 6, 8, 1.8, 0)
FFT3dFilter(bh=48, bw=48, oh=16, ow=16, sigma=1.5, plane=4, bt=4, beta=1, sharpen=0.4, wintype=1)


Here's 2 raw images:

IMG01 S (http://img440.imageshack.us/img440/4089/s01fd.png)
IMG02 S (http://img233.imageshack.us/img233/8871/s17ni.png)

and the same images filtered:

IMG01 F (http://img305.imageshack.us/img305/1962/f09vo.png)
IMG02 F (http://img241.imageshack.us/img241/7875/f19fk.png)

[Isochroma]

I took your raw #2 and put it thru Neat Image, and obtained far better denoising... here's a link to the denoised image:

s17ni-nr-1.png (http://isochroma.com/Testfiles/Comparisons/Denoise/s17ni-nr-1.png)

Neat Image supports the Auto Fine Tune feature, which will adjust the level of noise reduction depending on how much noise is in the image. My program Neat Batch also supports feeding NI that switch.

Because it only operates in the spacial dimension, it will not produce weird artifacts like fft3d does.

You can use Neat Batch to generate a .bat file to process large numbers of auto-demuxed avi frames. Also, it supports parameters that the new Neat Video does not.

Neat Batch (http://www.isochroma.com/Testfiles/NB/neatbatch.htm)

[Dark Alchemist]

I see they finally released neat video which means we can frame serve to it.

http://www.neatvideo.com/

[Isochroma]

Unfortunately, Neat Video doesn't take all the parameters that Neat Image does, particularly Auto Fine Tune. However, it may make use of this feature automatically. I'd recommend trying both on a still and a short clip including that still; this way you'll know which if any is superior.

[Dark Alchemist]

I noticed that but it is worth a shot to try both.

[mp3dom]

Thanks for reply.
I've tried Neat Image times ago (on this anime) but I've decided to go with AviSynth due to the fact that some parts were very blurried or "out of focus". Could be that I've skip some settings... :confused:
Thanks!

[Dark Alchemist]

On the neat image I still think it demands a sharpening filter right after it.

[Didée]

@ mp3dom

The "banding" artefacts you're getting are perfectly normal. Basically, there's nothing at all wrong with that - except for that it looks ugly. :)

Long story in short: YV12 colorspace is badly suited for smooth gradients without an noise. (The "step size" at those banding boundaries often is +/- 1 of pixel value per color plane and cannot get any more smooth.) That's why some minor noise is necessary to give the impression of smooth gradients.
If you look very closely (400% zoom) at Isochroma's screenshot, you see why it looks "better": it is because there has been less denoising happening on the gradient parts.

For a quick'n'ugly test through avisynth, we try the following: we take the source and the denoised source, and reduce the "denoising effect" by 1 for every pixel: source = whatever
denoised = source .YourDenoisingChain()
# denoised = denoised.deen("w2d",rad=5,thrY=3,thrUV=3,min=0.5) # try activating this, too
reducedNR = yv12lutxy(source,denoised, "x y > x 1 - x y < x 1 + y ? ?",
\ "x y > x 1 - x y < x 1 + y ? ?",
\ "x y > x 1 - x y < x 1 + y ? ?", U=3,V=3)

interleave(source,denoised,reducedNR,source) # for visual testing

It could be done even better with some sort of clever noise dithering. Suggested I had this already months (or years?) ago ... perhaps I'll script it one day ;)

[Isochroma]

I'm not quite sure what you mean, but the NI denoised image looks far superior and even under magnification does not exhibit the ugly blocking found in the fft3d version.

[Didée]

Sure. That's because you're sort of "faking" with RGB filtering. But in the end, you've to convert back to YV12 , and that's where your pleasently smooth gradients start breaking up again. :)

[mp3dom]

@ Isochroma:
Which settings do you have used? (I'm referring about the amount of noise to "kill", luma and sharp settings etc.) I've tried to reproduce the exact Picture Quality using the wall as noise pattern but I've obtained a less quality picture (NI says that noise pattern is at 87% quality)

@ Didéè:
I've understand the problem about YV12... but the raw/filtered pictures in my 1st post are referring to a YUY2 source (is it a problem also for YUY2?). I mean.. it's not an MPEG 4:2:0 Upconverted to YUY2... but a natively YUY2 4:2:2 which I have filtered keeping the colorspace as is.

[Isochroma]

s17ni-nr-1.yv12.png (http://isochroma.com/Testfiles/Comparisons/Denoise/s17ni-nr-1.yv12.png)

Still loaded into Vdub, saved to AVI with Helix YV12 codec, avi loaded back into vdub, source image captured and saved as png. No evidence of that blocking...

Using Neat Image 5.25 in RGB mode, here's the noise profile & filter settings:

Noise Profile (http://isochroma.com/Testfiles/Comparisons/Denoise/s17ni.dnp)
Filter Settings (http://isochroma.com/Testfiles/Comparisons/Denoise/s17ni.nfp)

[mp3dom]

I've re-tried NeatImage setting a profile for the anime but happends that too blur or out-of-focus occour so there are unusable images. I've enabled "Auto Fine Tune", "Auto Match Filter Presets" in Neat Batch. How are you doing to avoid these problems?! Also... is there a way to obtain a good results with actual AviSynth filters? I'm quite hopeless...
Anyway, thanks very much for the noise profile and filter presets.

[Isochroma]

It really depends on the nature of your source material. I use fft3d for temporal pre-processing before processing thru NI.

I wrote an extensive description of my workflow here (http://forum.doom9.org/showthread.php?t=104577), so definitely check it over and see if it can be useful...

Note that I use LimitedSharpen() between fft3d and NI, and that I use fft3d before NI. Careful tests determined that this order of precedence is superior to the reverse, in that more noise was removed with less blurring.