How to denoise clip with large grain size? (sample included)

[ph4nt0m]

I'm trying to find a suitable filter to reduce very heavy noise. Please have a look at this sample: http://www28.zippyshare.com/v/6M3siEiN/file.html

What makes this type of noise different from other ones I had to tackle until now is that the "grain size" seems to be very large. In other words, it looks like there are blocks of about 2x2 pixels instead of single pixels which make up the noise pattern. Many of the well known filters seem to work on a smaller scope, and therefore they are not very effective against this specific type of coarse noise.

While FastDegrain() does a reasonable job, it seems to miss the noise completely in one scene (might actually be a no-op, haven't verified):

https://picload.org/thumbnail/rracaoai/source_1.jpg (https://picload.org/image/rracaoai/source_1.png) https://picload.org/thumbnail/rracaodw/fd_1.jpg (https://picload.org/image/rracaodw/fd_1.png) https://picload.org/thumbnail/rracaoor/td_1.jpg (https://picload.org/image/rracaoor/td_1.png) https://picload.org/thumbnail/rracaoaa/rg_fd_1.jpg (https://picload.org/image/rracaoaa/rg_fd_1.png)

TemporalDegrain() works better in this case, but still a lot of grain remains. Moreover, in the first part of the sample, TemporalDegrain() leads to an unnatural, washed-out result (look at the leaves in the upper right part):

https://picload.org/thumbnail/rracaoaw/source_2.jpg (https://picload.org/image/rracaoaw/source_2.png) https://picload.org/thumbnail/rracaoar/fd_2.jpg (https://picload.org/image/rracaoar/fd_2.png) https://picload.org/thumbnail/rracaooa/td_2.jpg (https://picload.org/image/rracaooa/td_2.png) https://picload.org/thumbnail/rracaoal/rg_fd_2.jpg (https://picload.org/image/rracaoal/rg_fd_2.png)

Are there any filters (or combinations of filters) that might be more effective? Usually, I'm not the one who tries to remove every bit of noise, but this coarse grain is really unsightly in my opinion.

Thanks in advance!

[Groucho2004]

I tried a simple MDegrain2 with your clip:
LoadPlugin("E:\Apps\VideoTools\DGDec\DGDecode.dll")
MPEG2Source("F:\Test\sample.d2v", idct = 4)

RemoveNoise2(threshold = 800)

function RemoveNoise2(clip video, int "threshold")
{
last = video
threshold = default(threshold, 300)

sc = MSuper(hpad = 16, vpad = 16)
backward_vector2 = MAnalyse(sc, isb = true, delta = 2, blksize = 16, overlap = 4, truemotion = false, sadx264 = 4)
backward_vector1 = MAnalyse(sc, isb = true, delta = 1, blksize = 16, overlap = 4, truemotion = false, sadx264 = 4)
forward_vector1 = MAnalyse(sc, isb = false, delta = 1, blksize = 16, overlap = 4, truemotion = false, sadx264 = 4)
forward_vector2 = MAnalyse(sc, isb = false, delta = 2, blksize = 16, overlap = 4, truemotion = false, sadx264 = 4)
MDegrain2(sc, backward_vector1, forward_vector1, backward_vector2, forward_vector2, thSAD = threshold)
f3kdb(sample_mode = 2, keep_tv_range = true, grainY = 32, grainC = 32)

return last
}


Result (https://www.dropbox.com/s/0r7xdovyf6m8yno/sample.mkv?dl=0)

You can lower the value for "threshold" in order to get more detail but also less grain removal.

[ph4nt0m]

Thanks, that's not bad, although the result is pretty similar to FastDegrain().

Particularly, your function unfortunately does not remove any noise in the very last part of the sample, either, and I don't know why. Something about this particular scene seems to be special, making MDegrain-based approaches fail completely. :(

[feisty2]

V-BM3D
That's your man

[ph4nt0m]

V-BM3D seems to be available for VapourSynth only. So I downloaded the portable version, grabbed V-BM3D from here (http://forum.doom9.org/showthread.php?t=172172) as well as the mvsfunc script mentioned there. Still, denoising didn't seem too impressive. Could you please share your settings?

[feisty2]

import vapoursynth as vs
import Vine

core = vs.get_core()
clp = core.lsmas.LWLibavSource ("sample.mpg")
clp = core.fmtc.bitdepth(clp,bits=32,fulls=False,fulld=True)
clp = core.std.ShufflePlanes(clp,0,vs.GRAY)

lowf = Vine.helpers.gauss (clp, 4)
hif = core.std.MakeDiff(clp,Vine.helpers.gauss (clp, 16))
ref = core.std.MergeDiff(lowf,hif)
blr = Vine.helpers.gauss (ref,8)

ref = core.bm3d.VBasic(clp,blr,profile="vn",radius=3,th_mse=4000,sigma=64,hard_thr=5.2)
ref = core.bm3d.VAggregate(ref,3,1)
ref = core.knlm.KNLMeansCL(clp, 3, 8, 4, 5.6, rclip=ref)
clp = core.bm3d.VFinal(clp,ref,profile="vn",radius=3,th_mse=2000,sigma=64)
clp = core.bm3d.VAggregate(clp,3,1)

clp.set_output()

http://imgur.com/QAXGzqT.png

[Reel.Deel]

Eww, that's gross, you should name that function Plasticizer(). :)

Particularly, your function unfortunately does not remove any noise in the very last part of the sample, either, and I don't know why. Something about this particular scene seems to be special, making MDegrain-based approaches fail completely. :(

On heavy noise clips you'll get much better result if use a prefiltered clip to analyze the motion. The prefiltered clip should be "calm" but should not look anything like feisty's picture above.

[feisty2]

Eww, that's gross, you should name that function Plasticizer(). :)

that's just what's gonna happen when trying to remove insane amount of noise completely.
OP said he could not remove like any grain from that footage, and I simply showed him the opposite extreme, removing grain completely from the footage, he can always lower the strength down and make that much less aggressive


On heavy noise clips you'll get much better result if use a prefiltered clip to analyze the motion. The prefiltered clip should be "calm" but should not look anything like feisty's picture above.

that's simple minded, you simply cannot average the noise out like mdegrain when macroblocks go almost random noise block alike, the problem is the filtering part, you need something other than weighted averaging, pre-filtering helps with the estimating part, and not gonna do a damn thing there

[feisty2]

import vapoursynth as vs
import Vine
import mvmulti

core = vs.get_core()
clp = core.lsmas.LWLibavSource ("sample.mpg")
clp = core.fmtc.bitdepth(clp,bits=32,fulls=False,fulld=True)
clp = core.std.ShufflePlanes(clp,0,vs.GRAY)

lowf = Vine.helpers.gauss (clp, 4)
hif = core.std.MakeDiff(clp,Vine.helpers.gauss (clp, 16))
ref = core.std.MergeDiff(lowf,hif)
ref = Vine.helpers.gauss (ref,8)
coarse = core.knlm.KNLMeansCL(clp, 3, 8, 4, 9.6, rclip=ref)
sup = core.mvsf.Super(coarse)
supd = core.mvsf.Super(core.std.MakeDiff(clp,coarse))
vec = mvmulti.Analyze(sup, tr=6,blksize=32,overlap=16,search=3,badrange=-24)
vec = mvmulti.Recalculate(sup,vec,tr=6,blksize=16,overlap=8,thsad=400,search=3)
vec = mvmulti.Recalculate(sup,vec,tr=6,blksize=8,overlap=4,thsad=400,search=3)
vec = mvmulti.Recalculate(sup,vec,tr=6,blksize=4,overlap=2,thsad=400,search=3)
dif = mvmulti.DegrainN(core.std.Expr(clp,"0.5"), supd, vec, tr=6, thscd1=1600, thsad=10000)
clp = core.std.MergeDiff(coarse,dif)
clp.set_output()

http://i.imgur.com/FC5m4Ec.png
alternative approach, do some "stabilizing" to the grain instead of removing it

[feisty2]

that pic overhead looks too grainy to me still, I grew up watching digital stuff, no cheesy analog crap.
but that's far as you can go with that approach
and the idea is simple, since mdegrain doesn't work here, go apply something working first, remove the noise completely
and then mdegrain the noise layer and stabilize it and apply the stabilized layer back to make stuff look less crappy

[ph4nt0m]

Personally, I don't think that your last example looks too grainy and I actually like the result quite a lot (but that is highly subjective, of course). Unfortunately, I'm having difficulties getting Vapour Synth to work. Well, I think VS itself works, but the VS Editor crashes when I open the preview. MeGUI does not seem to support VS, either.

For the moment, I tried to modify FastDegrain() to implement @Reel.Deel's suggestion of using a prefiltered clip for motion analysis:

function FastDegrainPreFiltered(clip src, int "degrain", int "blksize")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)

# Create pre filtered clip for motion analysis
preNR = src.FFT3DFilter(sigma=20)
sup1 = src.MSuper(pel=2)
sup2 = preNR.MSuper(pel=2)

bvec3 = (degrain>=3) ? sup2.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=blksize/2) : NOP()
bvec2 = (degrain>=2) ? sup2.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=blksize/2) : NOP()
bvec1 = (degrain>=1) ? sup2.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=blksize/2) : NOP()
fvec1 = (degrain>=1) ? sup2.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=blksize/2) : NOP()
fvec2 = (degrain>=2) ? sup2.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=blksize/2) : NOP()
fvec3 = (degrain>=3) ? sup2.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=blksize/2) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
src = (degrain==1) ? src.MDegrain1(sup1, bvec1, fvec1) : src
src = (degrain==2) ? src.MDegrain2(sup1, bvec1, fvec1, bvec2, fvec2) : src
src = (degrain>=3) ? src.MDegrain3(sup1, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3) : src

return src
}

I think that's basically the same technique as in your latest VS example, although you seem to perform some more advanced steps there.

[feisty2]

Nah, it's fairly different
But it's okay long as it works

pre-filtering or sometimes called basic estimation before the actual estimation is, well, a pretty common step amongst many denoisers
Stabilizing the grain is, umm, that's a more alternative approach