MC noise removal script

[Spuds]

Well in an effort to first learn more about avisynth scripting capabilities and secondly to learn more about what you can do with mvtools I created a MC noise reduction script. Nothing really new here but I figured I'd post it since others may find it useful.

To use it you import it just like any other script and call it with its various parameters. To make it easy I have presets which set lots of things so you don't have to. You can use a preset plus override part of it by specifying the individual parameter as well.

Those more familiar with scripting and mvtools and some of other functions that I used, please let me know what mistakes I'm making since I'm trying to learn.

The basic calling syntax is:

mc_spuds(frames=(1-4),strength=(1-6)) 

The higher the strength and frames the stronger the denoising effect. As you increase strength you change block size and overlap in mvanalyse and fft3dfilter, plus turn on pre and post processing. The settings are in the script.

A good starting place for a noisy capture is mc_spuds(frames=2,strength=3or4or5). You have to be insane to use frames=4,strength=6 but its there if you have the time. Use debug=1-7 to see some of the settings and masks being used. I use mc_spuds(frames=2,strength=3) on some pretty bad vhs clips with very good results.

The full syntax is a bit more complex

mc+spuds(clip clp, int "frames", int "strength", int "blocksize", int "overlap", int "thsad", int "dct", int "ml", bool "chro", bool "postprocess", bool "preprocess", bool "aggressive",int "debug",bool "edgeclean")

What the script does:
- optional pre process for motion vector detection (supersample and denoise for search)
- Gets the motion vectors
- For frames = 1 or 2 use the motion vectors to mvdegrain, for frames = 3 to 4 use the vectors to mvflow.
- Optional for frames 3-4 is to first mvdegrain as if frames = 1-2 then do an mvflow.
- create a motion mask for the new degrained frame
- apply removegrain to the motion mask
- apply limitedsharpenfaster to the inverse (static) mask
- compute an edgemask and dering
- optional post process

The pre process does some simple cleaning and up sizing of the clip to help with the motion vector calculation. The post process is a bit of a jumble of remove dirt and dust scripts which needs improvement and/or simplification but seems to work. With post I was really trying to limit what changed since at that point the clip been worked on quite a bit.

There are also various debug views (debug=1-5) I like 4 and 5 the most.

Script will be in the next post, give it a try, tell me whats not working and what would work better.

[Spuds]

Old script removed, get the latest here

[Chipstra]

*Spuds:
- mt_masktools ------>> version ??
- removegrain ------>> version ??
- mvtools ------>> version ??

and where can I find the correct version?

error: mvanalyse does not have the named argument "sharp" - line 116

[Spuds]

I'll update the header in the script in a bit, but for now here is what you need to have installed to have this run.

required filters:

- mt_masktools -- version: 2.0.32 or later
code: http://manao4.free.fr/masktools-v2.0a32.zip
discussion: http://forum.doom9.org/showthread.php?t=98985

- removegrain -- version: 0.9 or later
code: http://www.removegrain.de.tf/
discussion: http://forum.doom9.org/showthread.php?s=&threadid=79751

- removedirt -- version 0.9 or later
code: http://www.removedirt.de.tf/
discussion: http://forum.doom9.org/showthread.php?s=&threadid=70856

- mvtools -- version: 1.8.4 or later
code: http://avisynth.org.ru/fizick.html
discussion: http://forum.doom9.org/showthread.php?t=84770

- limitedsharpenfaster -- in mt_masktools

- ff3dfilter -- version 2.1.1 or later
code: http://avisynth.org.ru/fizick.html
discussion: http://forum.doom9.org/showthread.php?t=85790

- ttempsmooth -- version: 0.8.4 or later
code: http://bengal.missouri.edu/~kes25c/
discussion: http://forum.doom9.org/showthread.php?t=77856

- deblock -- version 1.2 or later
code http://avisynth.org.ru/fizick.html
discussion: http://forum.doom9.org/showthread.php?t=110352

[Didée]

This definetly qualifies for the "complex script" category, congratulations.

Small bug found in line 78:

  # make a mask based on the new motion vectors
  idx = idx + 1

should be: global idx = idx + 1


Apart of that, I looked close if there are any "idx" mistakes in the script - could happen easily, with the multitude of different options and possible combinations. For a minute, I thought you trapped in with the "preprocess" thingy, then I saw you escaped with "supersample". Cheeese.

AFAICS, all idx usage seem to be correct.


But one last thing:

global idx = rand()

Funny idea, but not really good, IMO. Do not randomize that. Even if it's unlikely, it might give idx collisions if someone combines your script with other MVTools-based functions. (Often a silent(!) collision without error message, only a sub-optimal endresult, that is.)

[EuropeanMan]

^ any ideas on the speeds for this?

i'll assume this simplifies things? -scriptwise...for example:

setmemmax...
dgdecode...
colormatrix...
deinterlacing...
cropping...
resizing...
mc_spuds(....)

that'll do the denoising plus sharpening? sort of like integrated imaging p?

[Didée]

Bug for frames=3|4:

To my understanding, the implementation for frames=3 and 4 is flawed. In a nutshell, it does not do temporal processing over +/- 3|4 frames, but only over +/-1 (when frames=3) or over +/-2 (when frames=4). Also, maskmotion/maskstatic cannot yield to the wished result in this case.

IF frames > 2:

1) clp2 is firstly created as a long interleaved sequence of MVFlow compensations. (For frames=1|2, at this stage MVDegrain1|2 is applied. For frames=3|4, it is not. Only the interleave is done.)

2) Next, f34_phase2 is called on the long interleaved clip. Here, new vectors are calculated on the long interleaved clip. This means that the already-compensated (by MVFlow) wing frames are searched once more against the original center frame. Then, MVDegrain1|2 is applied to the long interleaved clip.

3) Now, a new vector search is done to build maskmotion. Since the working clip is still the long interleaved clip (with already compensated, and now even denoised wing frames), there should be virtually no motion to be detected around the center frame: maskmotion becomes useless.

4) Lastly, the center frame is Select()'ed out of the long interleaved clip again.


Suggestions: (for frames=3|4)

- the steps 3) and 4) are swapped. You first have to Select() the center out of the interleave, and then build maskmotion.

- Step 1 would have to be split in step 1a and step 1b: First do a MVDegrain over 2 frames, then call flow() on that degrained clip, then MVDegrain once more.
(Or something similar ... doing such "long distance" processing by means of MVDegrain is tricky.)


However, the denoising power of this function is really good, already with frames=2. Good work!

Originally Posted by EuropeanMan

^ any ideas on the speeds for this?

Pretty slow. That's impossible to avoid when doing advanced MVTools stuff. If you're interested in speed, rather stay with more simple filters.

[Fizick]

about frames=3.
It is quite possible that you will see MVdegrain3 soon...

[Didée]

OT - if you're at it, maybe consider to give MVDegrain two thresholds: thSAD_low and thSAD_high. thSAD_high the same as thSAD is now, and thSAD_low defining a lower thresh, allowing for full averaging strength if block SAD is below that. (I.e. the way like ttempsmooth is working.)
Just an idea, if it's not too difficult to do.

[Spuds]

This definetly qualifies for the "complex script" category, congratulations

I was quite impressed with what you could do with the scripting language in avisynth, pretty powerful. I did a couple of things that were not good "practice" but was trying to test out what I could do.

Small bug found in line 78:

ah yes, thanks for catching that, i'll make the update. I'll also change the idx=rand initialization and use some number that should be safe like 25. That way its at least deterministic.

On your analysis of the frames 3-4, I think I understand what you are saying. I was looking for a way to integrate mvdegrain with mvflow and went a little off track. Its working but I'm not getting all of the benefit out of the cpu time ... let me work with your suggestions and see what I can come up with. Greatly appreciate the analysis.

However, the denoising power of this function is really good, already with frames=2. Good work!

[Spuds]

I've made some changes over the last couple of days, v0.2b has been posted in post number 2.

The changes are:

1. a couple of bug fixes with global idx as discussed above.
2. change with idx in the main function to increment in case mc_spuds was called more than once in the same script to ensure no collisions.
3. changed the pre filter noise function from remove grain to fft3dfilter to allow the noise vector prefilter to respect the chroma search flag.
4. changed the blocksize of smoothed mask noise filter (increased) so it was not as aggressive. Makes increasing of strength more linear (less of a jump between 3 and 4).
5. changed frames = 3 or 4 function to first do mvdegrain then mvflow and hold that flowed clip through mask and post functions.
6. fixed dct not being used on user input, only default was being used. Improved default selection values.

[TheRyuu]

I get an error when trying to use strength=4-6 (4 through 6).

I forgot the exact wording.

[Spuds]

Thats not much of a hint to go work on

Are you saying all other modes worked or those are the only ones you tested and you received an error?

The most obvious error that could get thrown is if the video is not in yv12 colorspace or if you are missing one of the required plugins. If you could post the error I can help a lot more.

For a vhs clip you should have something like this:

load_stdcall_plugin(plugin + "yadif.dll")
Import("mc_spuds.avs") # MC noise removal
AVISource("somevideo.avi")
ConvertToYV12(interlaced=true)
AssumebFF()
yadif(mode=1,order=-1)
mc_spuds(frames=2,strength=4,debug=0)
AssumebFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

[Spuds]

I updated post number 2 with rev 0.3 of the script. I had to change the code a bit to get proper results out of frames=3 or 4. Didee first pointed out the error in my logic and it took me some time to wrap my head around the basic issue.

Anyway I had to add an option to the frames =3 or =4 call and its called aggressive. MC_spuds(frames=3,strength=2,aggressive=true)

With aggressive off the script will first mvflow a clip, then apply fft3d and temporalsoften to the now long flowed clip, then pull out the denoised frame and use that frame back in the original clip sequence to generate the masks.

If you turn aggressive on it will first get motion vectors and process the clip with mvdegrain2, then get new motion vectors and do a mvflow on the mv denoised frame. That will then be temporalsoftened and then the center frame pulled back out and used in the original sequence and mv masks generated.

With aggressive on you can use a low strength such as 2 and get quite good denoising. I'll post some example clips with various calls in a bit.

[Spuds]

Some examples of what the script can do:

Original image vs mc_spuds(frames=1,strength=4)


Original image vs mc_spuds(frames=2,strength=4)


Original image vs mc_spuds(frames=3,strength=4)


Original image vs mc_spuds(frames=4,strength=4)


Not 'clean' enough, then try aggressive=true with frames= 3 or 4

Original image vs mc_spuds(frames=3,strength=3,aggressive=true)


insane processing with full frames and strength and aggressive.
Original image vs mc_spuds(frames=4,strength=6,aggressive=true)


Example overrides, normally post process is set to false with strength=3, override like this

original vs mc_spuds(frames=3,strength=3,aggressive=true,postprocess=true)


Or like this
original vs mc_spuds(frames=2,strength=3,postprocess=true)

[Adub]

I have to say, I am impressed. That is a really crappy source, not just because of noise, but it looks like aliasing has some affect. Take a look at the white horse ring poles.

[foxyshadis]

Spuds, this is a cool filter =D But I was wondering, is there a way to work with the filter without the global blankclip? Because that seems to disable setmtmode when mc_spuds is used as an avsi. (Loaded before the script.)

[tsp]

foxyshadis: try moving it inside FUNCTION MC_Spuds.
As it is global it should work inside the other functions but are only evaluated when MC_spuds is actually used

[Spuds]

@foxyshadis: I updated the function to rev 0.3b, its in the second post of this thread. I removed the global nullclp and now set nullclp in the first function and pass it as need to the other sub functions. Let me know if this solves the issue you were having using it as an avsi and setmt.

Granted doing it this way there is a tiny overhead of setting the nullclp on each call to mc_spuds vs just once with global, but I don't think that will be noticed given all the calls to mvtools

@Merlin7777 Thanks. Hard to believe, or maybe not, but that was a commercially produced video. Goes to show what low light, poor optics and wrong film choice can do for you.

[salehin]

Thanks a lot for your hard work and for sharing it with us

Seeing those screens, I'm motivated to try mc_spuds.avsi with some really old films (now on DVD) i got