script function RemoveNoiseMC

[Fizick]

Small comment. Generally, I prefer do not use too many (>2) denoisers in one chain of script. Every denoiser kills some details, and deforms noise spectrum - some denoisers are developed to deal with un-denoised video.
And whole noise removal is not optimal (at least for non-anime video).

[Alain2]

I agree for the lower qual filters were deen is involved for instance, but nethertheless I am still impressed by what can be achieved with this motion denoising script; I used the hq filter on a grainy film I had and wanted to denoise with the least detail loss, and this filter chain gave me the best compromise And on animation the low settings are rather good as well

I've avoided so far diving into the mvtools, I mostly used the masktools for playing with my scripts, I think I'll start having a look to the mvtools as well

[Disabled]

After trying a lot more settings I think that any rdlimit over 2 kills small objects just flashing in, like the reflections in my second set of screens.
You wont notice it until you know what is missing, but that is too much for me.
rdlimit=2,rgrain=1,denoise=4 still kills detail in the rocks in set 2 & 3 but its a fast action scene and you wont notice anyway. And it kills some major noise in still scenes. I think I'll encode some more with these settings!
*edit*
Oh and with 2-3 fps its kinda fast for me

[Heini011]

@Disabled

pleae notice my small paramester selection guide, added in the first posting.

rgrain=3 is aimed only for analog capture or really poor quality sources. for high quality sources you could trying to set cpn=75*ccf in the script.

flashing in objects are in conflict with the aim of dirt dection/removing. this script is not aimed for high quality action movies. use the mvdenoise part of this script instead.

@Fizick:
good denoising is not only a matter of taste but means high compressibility too. therefore we need temporal and spatial filtering. a single temporal filter is hardly good enough for very strong noise. deen with weak parameters is a good extension for vaguedenoiser, cause it removes some dirt on bigger homogeneous surfaces.

[Disabled]

Yeah thanks Heini, I already noticed your parameter guide, but As I said I notice major object deletion with settings as low as rdlimit=3,rgrain=1,denoise=3. With rdlimit=3,rgrain=0,denoise=0 still some objects get removed.
Oh and with rdlimit=0 objects get removed too. I guess thats some bug, as its as strong as with rdlimit>=3
I experimented with the rest of the script too, but I think I still have a long way to go, but results are quite promising.
Thanks a lot for the script!

[hartford]

I tried this two ways; progressive works, interlaced gives exception. Perhaps an avisyth problem?

I'm using Avisynth v2.5.6.0

This causes an Avisynth exception:

LoadPlugin("d:\plugins\mt_masktools_2.0a28.dll")
LoadPlugin("d:\plugins\MaskTools-1.5.8.dll")
LoadPlugin("d:\plugins\Deen.dll")
LoadPlugin("d:\plugins\FluxSmooth-1.1.dll")
LoadPlugin("d:\plugins\mvtools-1.2.2.dll")
LoadPlugin("d:\plugins\RemoveDirt-0.9.dll")
LoadPlugin("d:\plugins\RemoveGrainS-09.dll")
LoadPlugin("d:\plugins\Repair-09.dll")
LoadPlugin("d:\plugins\VagueDenoiser-0.35.1.dll")
LoadPlugin("d:\plugins\deblock-1.2.dll")
LoadPlugin("d:\plugins\RemoveGrainS-1a.dll")

import("d:\plugins\import\RemoveNoiseMC.avsi")
import("d:\plugins\import\SeeSaw.avsi")

avisource("d:\ALL_This.avi").convertToYV12(interlaced=true)

separatefields()
Selectodd().RemoveNoiseMC(rdlimit=6,rgrain=1,denoise=6)
Selecteven().RemoveNoiseMC(rdlimit=6,rgrain=1,denoise=6)
Weave()

==

If I remove the "separatefields()" stuff, it works.

I'm not saying that there is anything "wrong" with your script.

What version of Avisynth are you using?

[Mr. Brown]

Originally Posted by Disabled

Yeah thanks Heini, I already noticed your parameter guide, but As I said I notice major object deletion with settings as low as rdlimit=3,rgrain=1,denoise=3. With rdlimit=3,rgrain=0,denoise=0 still some objects get removed.
Oh and with rdlimit=0 objects get removed too. I guess thats some bug, as its as strong as with rdlimit>=3
I experimented with the rest of the script too, but I think I still have a long way to go, but results are quite promising.
Thanks a lot for the script!

The removed object's is not a bug it's a problem that affect all Dirt removers.
The big problem for the filter is to difference in middle - high motion what is Dirt and what is Object/Detail.
In the end you must live with a compromise between Dirt Remove and Object/Detail Lost.

[DarkNite]

I have to say this gives amazing results with animated sequences. Well, it is not so kind to CGI in high motion when certain effects are used, but that's to be expected. Thankfully there's always filtering by section as a scripting option.

The hq settings almost always play nicely with content imported from my camera, and that's my main concern. With some simple tweaking this is very versatile. Nicely done.

[Heini011]

Hi folks,

i wrote a simpler and less brutal script function for denoising modern high quality movies. it was aimed as a better replacement for mvdenoise.

you can modify the mvtools blocksize:
cbs = 4 sometimes better denoising caused by better block-matching, less blocking
cbs = 8 much faster and may preserve slightly more sharpness

function RemoveTempGrain(clip input, int _mode)
{
  rg = RemoveGrain(input, mode=_mode)
  return TemporalRepair(rg, input)
}

function RemoveDirt_HQ(clip input, int tlimit, int rgrain, bool "_grey")
{
  _dgr = 0.4+rgrain*0.25
  clensed = input.RemoveTempGrain(1).FluxSmoothST(tlimit,rgrain)
  restore = input.VagueDenoiser(threshold=_dgr, nsteps=6, chromaT=_dgr, percent=100).RemoveGrain(1)
  alt = restore
  return RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, pthreshold=3+2*rgrain, cthreshold=3+2*rgrain, gmthreshold=40, dist=1, dmode=1, debug=false, noise=tlimit+1, noisy=12, grey=false)
}

global idx_c = 4

function RemoveNoiseMC_HQ(clip,int "tlimit", int "rgrain", float "csharp")
{
  tlimit = default(tlimit,4)
  rgrain = default(rgrain,1)
  csharp = rgrain>1 ? default(csharp,0.14) : default(csharp,0.13)
  global idx_c = idx_c + 1
  cbs = 4
  ccf = cbs*cbs/64
  cpn = tlimit>6 ? 70*ccf : 75*ccf
  bvec1 = clip.MVAnalyse(isb=false, blksize=cbs, delta=1, pel=2, sharp=1, overlap=0, truemotion=true, pnew=cpn, idx=idx_c)
  fvec1 = clip.MVAnalyse(isb=true, blksize=cbs, delta=1, pel=2, sharp=1, overlap=0, truemotion=true, pnew=cpn, idx=idx_c)
  backw1 = clip.MVFlow(bvec1, idx=idx_c)
  forw1  = clip.MVFlow(fvec1, idx=idx_c)
  dnc = interleave(backw1,clip,forw1)
  dnc = dnc.RemoveDirt_HQ(tlimit,rgrain)
  dnc = dnc.SelectEvery(3,1)
  return csharp==0 ? dnc : \
    clip.SeeSaw(dnc, Sstr=csharp, Szp=12, SdampHi=20, bias=40)
}



function hqfilter(clip c)
{
  c=c.RemoveNoiseMC_HQ(tlimit=4,rgrain=1)
  c=c.LimitedSharpenFaster(Smode=4,strength=15,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)
  return c
}

27. may: very small update @ seesaw

[Alain2]

I will try

Question: Am I obliged to put global idx_c = 4 ? it was 25 in the previous functions you gave, and as I understand it this number is just a ref number for idx in mvtools, so it's starting point not very important ? (I ask because I want to keep all your functions in the same .avsi)

[Heini011]

Hi Alain2,

the only importance for idx_c is this: there shouldn't be another mvtools function call in your avisynth script with the same idx value.

[Clown shoes]

Hi there. Please excuse my scripting ignorance, but I am having a problem with the usage of the simple script. My .avs is reporting; There is no function named "TemporalRepair" I am imagining this is due to incorrect usage rather than incorrect .dlls, although I could be wrong. I am trying to use this in conjunction with LimitedSharpener. My script is as follows;

Import("C:\Program Files\AviSynth 2.5\plugins\LimitedSharpenFaster.avsi")
import("C:\Program Files\AviSynth 2.5\plugins\RemoveNoiseSimple.avsi")
import("C:\Program Files\AviSynth 2.5\plugins\SeeSaw.avsi")
avisource("E:\test.avi", audio=false)
#colorMatrix()
AssumeFPS(24,1,true)
#addborders(0,32,0,32)
Levels(0, 1.2, 255, 0, 255)
Tweak(bright=1.3, sat=1.2, cont=1.05 )
RemoveNoiseMC_HQ(tlimit=4,rgrain=1)
LimitedSharpen(dest_x=368,dest_y=208,ss_x=1.5,ss_y=1.5,Smode=3,strength=100,wide=true,special=true)


Would someone be good enough to point out where I am going wrong.

Thanks in advance

Rich

[Alain2]

Look at the first post of this thread, the needed plugins section

[Clown shoes]

I have them all in my plugins folder as far as I can tell. Is TemporalRepair refering to the lack or non functioning of a particular plugin?

[foxyshadis]

Repair.dll and its auxillaries are in the removegrain.rar linked in the first post. You may also need SSETools from the official 0.9.

[myck]

This is the best script for denoise I've ever used and with a good speed also. But I want an advice. I use this especialy on low light videos from my pal camera (the ones in day light doesn't have much noise). So please tell me which are the best settings for using with videos on low light and what ways are you using for bright the videos (I use HDRAGC for now with good results, but I want to know if is something better). Thank you.

[Heini011]

Hi Clown shoes,

foxyshadis answered already: you need the Repair.dll from RemoveGrain 1.0 pre package, linked in the first posting of the removenoisemc thread. if it still doesn't work, copy RepairS.dll in your avisynth plugin directory.

hi myck,

all filters used in my script have absolute thresholds. so there is little to tweak for low light scenes, sadly. but you could use didee's great ylevels script to enhance the brightness.

greetings.

[Clown shoes]

Thanks guys, yes that works. I also had to move the dlls out of the Avisynth plugin folder, as they were conflicting with other dlls and causing a crash each time I opened a script. I do have one other question though. Your script calls on the SeeSaw script. Does that then make my use of Limited Sharpen redundant, or am I getting myself a little confused.

I am using your simple action script with cbs = 8

and my script is now;

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\mt_masktools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\Deen.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\FluxSmooth.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\mvtools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\RemoveDirt.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\RemoveGrainS.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\Repair.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\VagueDenoiser.dll")


Import("C:\Program Files\AviSynth 2.5\plugins\LimitedSharpenFaster.avsi")
Import("C:\Program Files\AviSynth 2.5\plugins\RemoveNoiseSimple.avsi")
Import("C:\Program Files\AviSynth 2.5\plugins\SeeSaw.avsi")


avisource("E:\test.avi", audio=false)
#colorMatrix()
AssumeFPS(24,1,true)
#addborders(0,32,0,32)


Levels(0, 1.2, 255, 0, 255)
Tweak(bright=1.3, sat=1.2, cont=1.05 )


RemoveNoiseMC_HQ(tlimit=4,rgrain=1)
LimitedSharpen(dest_x=368,dest_y=208,ss_x=1.5,ss_y=1.5,Smode=3,strength=100,wide=true,special=true)

I am using your Remove noise script instead of; hqdn3d(3,2,4,4,5)

Could you see any purpose to using both denoisers, or is that just unnecesary overkill?


Sorry. I forgot one more stupid question. I am a little new to using longer complicated scripting, and I am confused by the addition at the end of your script of these lines;

c=c.RemoveNoiseMC_HQ(tlimit=4,rgrain=1)
c=c.LimitedSharpenFaster(Smode=4,strength=15,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)
return c

Are they active in the script? And does that mean I can just make my adjustments to Limited Sharpen within your Remove Noise script?

Apologies if these are very stupid questions. I am still quite new to this and there is quite a steep, initial learning curve.


Thanks again

Clown Shoes

[DarkNite]

I am using your Remove noise script instead of; hqdn3d(3,2,4,4,5)

Could you see any purpose to using both denoisers, or is that just unnecesary overkill?

I'm almost certain that would be overkill, but you could always try it and compare.

I am confused by the addition at the end of your script of these lines;

c=c.RemoveNoiseMC_HQ(tlimit=4,rgrain=1)
c=c.LimitedSharpenFaster(Smode=4,strength=15,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)
return c

Are they active in the script? And does that mean I can just make my adjustments to Limited Sharpen within your Remove Noise script?

Generally a function or plugin call that isn't commented out, and is followed by a return call is active.

Just comment out the original LimitedSharpenFaster line, and create a dupicate line in your script. Edit the duplicate line in the script to match your destination settings. If that doesn't work well enough for you then start adding your other settings in as well.

Example:

#c=c.LimitedSharpenFaster(Smode=4,strength=15,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)
c=c.LimitedSharpenFaster(dest_x=368,dest_y=208,Smode=4,strength=15,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)

That way you won't lose the original line if something goes wrong or you're not pleased with the results.

[Clown shoes]

Thank you for your help Darknite, but I am clearly still doing something wrong! If I comment out the function lines in my script and use the lines in the remove grain script, neither lines work and the file plays unfiltered! If I comment out the lines in the remove grain script and replace them in mine, it seems to work properly. I'm sure I should just be thankful I can get the filters working, but I want to understand what it is that's going wrong. If I post up the two scripts that are not working, maybe someone could point out where I am going wrong?

My Script;

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\mt_masktools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\Deen.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\FluxSmooth.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\mvtools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\RemoveDirt.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\RemoveGrainS.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\Repair.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Remove Noise Simple\VagueDenoiser.dll")


Import("C:\Program Files\AviSynth 2.5\plugins\LimitedSharpenFaster.avsi")
Import("C:\Program Files\AviSynth 2.5\plugins\RemoveNoiseMC.avsi")
Import("C:\Program Files\AviSynth 2.5\plugins\SeeSaw.avsi")


avisource("E:\test.avi", audio=false)
#colorMatrix()
AssumeFPS(24,1,true)
#addborders(0,32,0,32)


Levels(0, 1.2, 255, 0, 255)
Tweak(bright=1.3, sat=1.2, cont=1.05 )


#RemoveNoiseMC_HQ(tlimit=4,rgrain=1)
#LimitedSharpenFaster(dest_x=368,dest_y=208,ss_x=1.3,ss_y=1.3,Smode=3,strength=100,wide=true,special=true)


Remove Noise Script

function RemoveTempGrain(clip input, int _mode)
{
rg = RemoveGrain(input, mode=_mode)
return TemporalRepair(rg, input)
}

function RemoveDirt_HQ(clip input, int tlimit, int rgrain, bool "_grey")
{
_dgr = 0.35+rgrain*0.2
clensed = input.RemoveTempGrain(1).FluxSmoothST(tlimit,rgrain)
restore = input.VagueDenoiser(threshold=_dgr, nsteps=6, chromaT=_dgr, percent=100).RemoveGrain(1)
alt = restore
return RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, pthreshold=3+2*rgrain, cthreshold=3+2*rgrain, gmthreshold=40, dist=1, dmode=1, debug=false, noise=tlimit+1, noisy=12, grey=false)
}

global idx_c = 4

function RemoveNoiseMC_HQ(clip,int "tlimit", int "rgrain", float "csharp")
{
tlimit = default(tlimit,4)
rgrain = default(rgrain,1)
csharp = (rgrain>1) ? default(csharp,0.28) : default(csharp,0.26)
global idx_c = idx_c + 1
cbs = 8
ccf = cbs*cbs/64
cpn = (tlimit>6) ? 72*ccf : 76*ccf
bvec1 = clip.MVAnalyse(isb=false, blksize=cbs, delta=1, pel=2, truemotion=true, pnew=cpn, idx=idx_c)
fvec1 = clip.MVAnalyse(isb=true, blksize=cbs, delta=1, pel=2, truemotion=true, pnew=cpn, idx=idx_c)
backw1 = clip.MVFlow(bvec1, idx=idx_c)
forw1 = clip.MVFlow(fvec1, idx=idx_c)
dnc = interleave(backw1,clip,forw1)
dnc = dnc.RemoveDirt_HQ(tlimit,rgrain)
dnc = dnc.SelectEvery(3,1)
return csharp == 0 ? dnc : \
dnc.SeeSaw(dnc, Sstr=csharp, Szp=12, SdampHi=20, bias=40)
}



function hqfilter(clip c)
{
c=c.RemoveNoiseMC_HQ(tlimit=4,rgrain=1)
#c=c.LimitedSharpenFaster(Smode=4,strength=15,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)
c=c.LimitedSharpenFaster(dest_x=368,dest_y=208,ss_x=1.3,ss_y=1.3,Smode=3,strength=100,wide=true,special=true)
return c
}

I'm sure, I'm doing something silly wrong here, but I would like to understand what that is, so as I can learn more about quality scripting.

The only other thing I am confused about is the use of SeeSaw in the Remove Noise Script. Is it actually being called upon?