Removing low-light noise/grain from DV

[nhope]

I have quite a lot of underwater footage shot in low light with my Sony VX2000 3-CD DV camera with a lot of gain and so lots of noise/grain.

Here is a typical sample DV .avi file.

Any suggestions for dealing with this? I tried a few things such as MCTemporalDenoise(settings="medium", interlaced=true) and the result was promising but I'm wondering if I'm on the wrong track, or if this degree of noise is a bit of a lost cause.

[poisondeathray]

I think zip file is corrupt , CRC errors. And there is no recovery record

[johnmeyer]

Yup, zip is corrupt.

[nhope]

Really sorry about that. I've posted the .avi file without zipping and the link in the first post now works.

[Taurus]

@nhope:
Heavy fights need heavy guns
In a ten minutes exercise in degraining.
Temporaldegrain was the winner...
Here is the simple sample script:

avisource("LIBbatfish-orbicular1a.avi")
tweak(sat=1.25)
temporaldegrain()
LSFMod()

Here is a mpeg sample:
http://www.mediafire.com/?cquz8qcf4sppdxv
Sorry, could'nt resist in a little colour tweaking.
Decoder: Cedocida
Encoder: HCEnc

Just a suggestion.

[Didée]

@Taurus: you forgot that the source is interlaced.

[johnmeyer]

Nick,

I have had pretty good luck with MDegrain2, using settings right out of the "cookbook."

I played around a little, trying all the block size, overlap, and dct settings. After I tweaked those, I changed the thSAD setting. If you go too high, you begin to get an artifact where the residual noise in the flat shadow areas (under the coral in the bottom center of the video) begins to act coherently, which is really distracting. I often prefer a little residual noise, as long as it looks "natural," if that description makes any sense.

I also tried to do some things with fft3Dfilter, both on its own, and within a motion compensated setting. I played around with the separate sigma settings, but never could get anything that looked good. I also tried to use it as a post filter.

Here is a link to the result:

Desnoised Fish

I rather like the way the script maintained the details on the dorsel fin of the fish in the foreground.

I thought the result looked a little soft. Rather than sharpen in AVISynth, I put it in Vegas and used the Unsharp Mask fX with an Amount of 0.2 and a radius of 0.02.

And here is the script I used. This is pretty much straight from the documentation, except for my choices of block size, overlap, and thSAD. If I were a masking guru, like Didée, I would be tempted to try to figure out a way to apply heavier denoising in the shadows, especially if I could only apply the mask to shadows that are large, and mostly "flat" (without much high frequency detail).

#Denoiser script for interlaced video using MDegrain3
#July 1, 2011

loadPlugin("c:\Program Files\AviSynth 2.5\plugins\fft3dfilter.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

SetMTMode(5,0)
#Modify this line to point to your video file
source=AVISource("E:\frameserver.avi").killaudio().AssumeBFF()
SetMTMode(2)

denoised=MDegrain2i2(source,8,2,0)

#Possible post-filtering and sharpening
#denoised=fft3dfilter(denoised, sigma=1, sigma2=1, sigma3=1, sigma4=1, sharpen=0, interlaced=true)

#stackhorizontal(source,denoised)
return denoised


#-------------------------------

function MDegrain2i2(clip source, int "blksize", int "overlap", int "dct")
{
  overlap=default(overlap,0)                # overlap value (0 to 4 for blksize=8)
  dct=default(dct,0)                        # use dct=1 for clip with light flicker

  fields=source.SeparateFields()            # separate by fields

  super = fields.MSuper(pel=2, sharp=1)

  backward_vec2 = super.MAnalyse(isb = true, delta = 2, blksize=blksize, overlap=overlap, dct=dct)
  forward_vec2 = super.MAnalyse(isb = false, delta = 2, blksize=blksize, overlap=overlap, dct=dct)
  backward_vec4 = super.MAnalyse(isb = true, delta = 4, blksize=blksize, overlap=overlap, dct=dct)
  forward_vec4 = super.MAnalyse(isb = false, delta = 4, blksize=blksize, overlap=overlap, dct=dct)

# MDegrain2(fields,super, backward_vec2,forward_vec2,backward_vec4,forward_vec4,thSAD=400) 

# Eliminate next three lines and uncomment above line to use MDegrain2 instead of MDegrain3
  backward_vec6 = super.MAnalyse(isb = true, delta = 6, blksize=blksize, overlap=overlap, dct=dct)
  forward_vec6 = super.MAnalyse(isb = false, delta = 6, blksize=blksize, overlap=overlap, dct=dct)

# Added thSCD1=550 parameter after posts below pointed out script doesn't work when reading video file directly (I was frameserving)
  MDegrain3(fields,super, backward_vec2,forward_vec2,backward_vec4,forward_vec4,backward_vec6,forward_vec6,thSCD1=550, thSAD=600)

  Weave()
}

[Taurus]

Originally Posted by Didée

@Taurus: you forgot that the source is interlaced.

Da..it, I always forget to think about it

[Didée]

@johnmeyer

Something can not be right. The script you posted does ZERO denoising on this clip. You must've used another script, or vastly different settings.

(The noise triggers MVTools' scenechange reckognition, and consequentially MDegrain does *nothing*at*all*.)

[johnmeyer]

Originally Posted by Didée

@johnmeyer

Something can not be right. The script you posted does ZERO denoising on this clip. You must've used another script, or vastly different settings.

Well that is disturbing ...

To "check my work," I just copied the text from the code section of my post above, put it in Notepad and saved it, and then opened the result in VirtualDub. The script is most definitely working on my computer.

I looked at the MVTools2.dll and got this version info: 2.5.11.2. It is dated March 20, 2011. I made a version call and got "AviSynth 2.60, build: Aug 13 2009 [15:07:40] (c) 2000-2007 Ben Rudiak-Gould, et al."

Those are the only two dependencies I can think of that would be different between your computer and mine.

[johnmeyer]

I just did a little more work on this, and explicitly included both the thSCD1 and thSCD2 parameters. The default for thSCD1 is 400. I experimented and found that around 300, the denoising stopped, as it apparently is doing on your computer.

I then looked at the MVTools2 documentation and found this under the change log for version 1.4.1:

"Chanded thSCD1 default from 300 to 400."

So, if you are using a really old version of MVTools2, that would explain it, or if you have some set of defaults stored somewhere in order to maintain consistency with the older version, that would also explain it.

As far as the OP is concerned, it might make sense to include this parameter and raise its value to 500.

[Taurus]

Originally Posted by Didée

@johnmeyer

Something can not be right. The script you posted does ZERO denoising on this clip. You must've used another script, or vastly different settings.

(The noise triggers MVTools' scenechange reckognition, and consequentially MDegrain does *nothing*at*all*.)

I can confirm Didée's error observation.
There is no call for a denoiser.
Tested with mvtools2/2.5.11.2

[Didée]

@johnmeyer: No. I am using exactly that version of MVTools2, and of Avisynth 2.6 too. "Change of Defaults" would require a re-compiling of the plugin. Which I surely did not do.

And when I run *literally* your original script, then the final difference between "source" and "denoised" is absolutely zero. The difference screen is flat 128.

I rather assume you applied some pre-denoising before using that script. This would explain that the MDegrain worked (because the predenoise made the SADs in MVTools small enough). And it would explain why you used post-sharpening (because the predenoise did some blurring, which would not be the case [or only very little] with only MDegrain.)

[johnmeyer]

Well, this is curious.

You said "I rather assume you applied some pre-denoising before using that script." I didn't do anything like that, but I did frameserve the footage out of Sony Vegas and into the AVISynth script rather than reading the AVI file itself. So, just to make sure I was doing the same thing you were doing, I instead opened the OP's original file directly in AVISynth.

This time I got the same result as you.

I am totally clueless as to why this is happening. The frameserver doesn't alter the video in any way: I can do an A/B comparison with video frameserved into AVISynth scripts and then rendered or served back into Vegas, and they always look identical (in terms of resolution, levels, sharpness, and color). There is definitely no filtering of any kind going on.

It is a mystery.

As for helping the OP, I found that the threshold for thSCD1 when reading the file directly was somewhere around 450. So, if you add this parameter and set it to something between 500 and 550, I think the script should work. I have edited the script in my post above to add this parameter, and have noted in that post that I made the alteration.

[nhope]

Thanks folks.

Originally Posted by Didée

@Taurus: you forgot that the source is interlaced.

How should such a script be modified for interlaced source?

Actually all my projects with PAL interlaced source are going to get bobbed with QTGMC. DVD projects will then be re-timed to 60p, downscaled to 640x480, then reinterlaced. YouTube projects will be upscaled (with pillarboxing) to 1280x720. So perhaps denoising of noisy clips could be done further down the chain? Or better to do it straight away on the interlaced PAL footage?

[johnmeyer]

Originally Posted by nhope

How should such a script be modified for interlaced source?

You've spent a lot of time on this with your re-sizing "quests," so you probably know most of the answers, and many of the issues. The method you choose to use depends a lot on the nature of the functions your script runs.

I grabbed the following code from somewhere in this forum (from years ago). It is a quick & dirty way to make many functions, which were originally designed only for progressive footage, work well for interlaced footage, without having to do a lot of new coding:

function ApplyInterlacedFilter(clip v1, string filter)
{
   v2 = separatefields(v1)
   selecteven(v2)
   even = Eval(filter)
   selectodd(v2)
   odd = Eval(filter)
   interleave(even,odd)
   return weave()
}

I used this earlier today when I was trying out various approaches to your noise problem. Here is one script that I used. It was based on code I got from somewhere else, and didn't want to re-write. This shows how the above function can be used.

source=AVISource("E:\frameserver.avi")
ApplyInterlacedFilter(source, "MotionCompDenoise")

function ApplyInterlacedFilter(clip v1, string filter)
{
   v2 = separatefields(v1)
   selecteven(v2)
   even = Eval(filter)
   selectodd(v2)
   odd = Eval(filter)
   interleave(even,odd)
   return weave()
}

function MotionCompDenoise(clip thisclip)
{
  super = MSuper(thisclip)
  backward_vectors = MAnalyse(super, isb = true)
  forward_vectors = MAnalyse(super, isb = false)
  forward_compensation = MFlow(thisclip, super, forward_vectors, thSCD1=400) # or use MCompensate
  backward_compensation = MFlow(thisclip, super, backward_vectors, thSCD1=400)
  interleave(forward_compensation, thisclip, backward_compensation)
  fft3dfilter(sigma=2, sigma2=10, sigma3=15, sigma4=8)
  selectevery(3,1) # return filtered central (not-compensated) frames only
}

As for when to do denoising, I always do it before everything else, because just about any operation on noisy video is not going to do good things to it.

[nhope]

Originally Posted by johnmeyer

...The frameserver doesn't alter the video in any way...

The DV clip is YUV. Vegas converts to RGB, which it uses internally. Then the Frameserver converts to YUY2 (assuming you have it set to that, and not RGB; I assume so as there is no conversion to YUV going on in your script). So there are a couple of conversions going on there.

[johnmeyer]

Yeah, I didn't want to burden this thread with all of that. It is surprising, since the levels aren't detectably changing that it would make that much difference.

Did you download the PAL DV AVI file that I posted which shows the result? Did you like it, or did you find it lacking? I'm always open to suggestions or criticisms (I guess that's why I post here )

[TheSkiller]

Originally Posted by nhope

YouTube projects will be upscaled (with pillarboxing) to 1280x720.

Pillarboxing to 1280 width is not necessary for YouTube. You should simply omit the pillarbox which leaves you with 960x720 in case of 4:3 video. YouTube will perfectly accept this as 720p HD like it does with 1280x720 video. Now you might say it doesn't really matter, well, if you pillarbox 4:3 content for YouTube then it'll make the video appear in a small window in fullscreen mode on non-16:9 displays, in other words windowboxing on all non-16:9 displays in fullscreen mode, not nice at all.

Just curious, why would you convert the framerate of a PAL source to 60p instead of leaving at 50p or reinterlacing it back to 25i (50i as some would say).


Edit: @johnmeyer I had a look at the source and your filtering. Honestly, but that's just me, I found it to be slightly overfiltered/blurry. I would rather leave some more faint grain as it is giving the illusion of detail. The stones in the background have noticeably lost their sharpness.

[johnmeyer]

Originally Posted by TheSkiller

@johnmeyer I had a look at the source and your filtering. Honestly, but that's just me, I found it to be slightly overfiltered/blurry. I would rather leave some more faint grain as it is giving the illusion of detail. The stones in the background have noticeably lost their sharpness.

{sigh} ... yes, looking at it now, I think you are probably correct, although as I noted in my original post, I did sharpen the result later, in Vegas. The result I posted was the result of the AVISynth script, before the sharpening in Vegas.

Thinking back to this morning, I think the reason I dulled it that much is that when I used a less aggressive de-noising setting which retained more detail, the noise looked less "normal." What I mean by this is that noise, while annoying, is something that you sort of get used to, but only if it is uniform and invariant over both space and time. However, when it comes and goes, or "clumps" and then un-clumps, or does something else to call attention to itself by not being uniform, then the de-noised version is sometimes worse than the original.