Help me to clean a very noisy (snowy) clip with a script

[medp7060]

I have a cip, which is TV recording. It is very nosy. I put a sample http://rapidshare.com/files/141925672/TV.mpg.

I tried several filters to clean it, but the results were not good.

Thanks in advance guys.

[Jawed]

Try the following script:

global MeGUI_darx = 4
global MeGUI_dary = 3
DGDecode_mpeg2source("TV.d2v")

vinverse()
crop(14,4,-8,-4)
spline36resize(528,432)
FizzKillerMulti(blksize=16, sharp=2, thsad=800, refframes=6)
return last

function FizzKillerMulti(clip source, int "blksize", int "overlap", int "sharp", int "thSAD", int "refframes")
{ # Motion compensated denoiser for progressive source clip with prefiltering optimised to reduce high-contrast noise
# Uses MVTools, ReduceFlicker and RemoveGrain

blksize = default(blksize,8) # blksize value (4, 8 or 16)
overlap = default(overlap,blksize/2) # overlap value (0 to half blksize)
sharp = default(sharp,1) # 0=bilinear softest, 1=bicubic, 2=Wiener sharpest
thSAD = default(thSAD,200) # higher risks motion ghosting and swimming, lower risks blotchy denoising
refframes = default(refframes,5) # temporal radius for MVTools using the "Multi" modification by Josey Wells

idx = 20 # "safe" idx for use within this function only

# Prefilter the clip
calm = source.reduceflicker(strength=3,aggressive=true)
calm = calm.greyscale()
calm = calm.spline36resize(source.width*2,source.height*2)
calm = calm.invert("Y").levels(0, 0.5, 255, 0, 255, coring = false)
calm = calm.removegrain(mode = 17)
calm = calm.removegrain(mode = 17)
calm = calm.removegrain(mode = 17)
calm = calm.spline36resize(source.width,source.height)

vectors = calm.MVAnalyseMulti(refframes = refframes, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx)
source.MVDegrainMulti(vectors, refframes = refframes, thSAD = thSAD, idx = idx + 1)
}

You need to get ReduceFlicker, RemoveGrain and MVTools. You actually need Josey Wells's version of MVTools. I am using version 1.9.7.0:

http://forum.doom9.org/showthread.php?p=1173626#post1173626

but you can use later versions I guess. If his version of MVTools doesn't work for you, I can give you a version of the script that uses the standard version of MVTools.

If the noise reduction is too strong then you can reduce the effect by reducing the thSAD and/or refframes parameters. You should experiment with these settings anyway, because they're so strong that there's bound to be problems with movement that's blurred too much or with unwanted movement.

Also notice that I cropped and resized the video. The resize is there to reduce a "pumping" noise that appears every half-second or every second. This noise is much finer in size than most of the noise in this clip, so by making the video frame smaller I'm blurring the noise slightly and making it less different from other frames. If I resize any smaller then fine detail in the animation at the start is lost.

This script function, FizzKillerMulti, is work in progress that I've been playing with for a while now, and I've been writing about controlling high contrast noise here:

http://forum.doom9.org/showthread.php?t=133977

Your clip has been useful as it's given me a couple of good ideas and I've learnt some new things too.

Jawed

[Nightshiver]

Excellent script Jawed. I'm becoming more interested in fizzkiller....
Udegrain works good aswell.

UDegrain(radius=7,settings="heavy")

[Jawed]

I just tried UDegrain as you suggested but it's not really working as well as FK.

See attachment, FK on the left and UD on the right.

Jawed

[Didée]

Are you aware that the sample is interlaced? The shown script forces simple field blending...

[Jawed]

I did ponder this question for a while. AvsP reports the clip as frame based not field based, and between TDeint() and VInverse() I was seeing little difference (different kinds of blur), plus the "interlacing" artefacts look a lot like "badly de-interlaced" video, i.e. it seemed to me that the original poster has provided a clip that has been "de-interlaced". So in the end I decided to go with the simplest.

Jawed

[Didée]

Just bob() it, and you'll see that there are unique motion states in all of the fields (look at the scenes with the dancers).

On a technical note, any particular reason for Reduceflicker(3,true) in the prefiltering chain? It's a filter to remove regular oscillations, not general noise.

Reduceflicker's characteristics are denoising=little, artifacts=plenty. See:

http://img149.imageshack.us/img149/1319/reduceflickeriv4.th.jpg (http://img149.imageshack.us/my.php?image=reduceflickeriv4.jpg)

That's not a good base for doing motion search on.

[Nightshiver]

Or you could use yadif.

[Jawed]

Just bob() it, and you'll see that there are unique motion states in all of the fields (look at the scenes with the dancers).
Thanks - that definitely clears that up. Oh well, I'll not refer to AvsP's field/frame info again.

On a technical note, any particular reason for Reduceflicker(3,true) in the prefiltering chain? It's a filter to remove regular oscillations, not general noise.

Reduceflicker's characteristics are denoising=little, artifacts=plenty.
That's not a good base for doing motion search on.
Thanks for the tip off. I found a section of one of my test clips with a high contrast pattern moving through the frame that generates a similar artefact. This definitely screws up FK.

EDIT: it turns out ReduceFlicker is key to the effectiveness of FK on this clip :eek:

Jawed

[medp7060]

Thank you, guys. I will test it with the script and will let you know the results.

I incorparated Didée's suggestions into the script by Jawed as below:


global MeGUI_darx = 4
global MeGUI_dary = 3
mpeg2source("e:\TV.d2v")

TDeint() # replaces Vinverse() for better de-interlacing as said by Jawed
crop(14,4,-8,-4)
spline36resize(528,432)
FizzKillerMulti(blksize=16, sharp=2, thsad=800, refframes=6)
return last

function FizzKillerMulti(clip source, int "blksize", int "overlap", int "sharp", int "thSAD", int "refframes")
{ # Motion compensated denoiser for progressive source clip with prefiltering optimised to reduce high-contrast noise
# Uses MVTools, ReduceFlicker and RemoveGrain

blksize = default(blksize,8) # blksize value (4, 8 or 16)
overlap = default(overlap,blksize/2) # overlap value (0 to half blksize)
sharp = default(sharp,1) # 0=bilinear softest, 1=bicubic, 2=Wiener sharpest
thSAD = default(thSAD,200) # higher risks motion ghosting and swimming, lower risks blotchy denoising
refframes = default(refframes,5) # temporal radius for MVTools using the "Multi" modification by Josey Wells

idx = 20 # "safe" idx for use within this function only

# Prefilter the clip
calm = source #.reduceflicker(strength=3,aggressive=true) #removed as suggested by Jawed
calm = calm.greyscale()
calm = calm.spline36resize(source.width*2,source.height*2)
calm = calm.invert("Y").levels(0, 0.5, 255, 0, 255, coring = false)
calm = calm.removegrain(mode = 17)
calm = calm.removegrain(mode = 17)
calm = calm.removegrain(mode = 17)
calm = calm.spline36resize(source.width,source.height)

vectors = calm.MVAnalyseMulti(refframes = refframes, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx)
source.MVDegrainMulti(vectors, refframes = refframes, thSAD = thSAD, idx = idx + 1)
}

[Sagekilla]

You're just joking around right? ReduceFlicker(denoising=little, artifacts=plenty) isn't even valid syntax..

[thetoof]

LOL!! This isn't very constructive since Sagekilla already said something about it, but I just had to post :p
Didée only meant that reduceflicker introduces "plenty" of artifacts, while denoising only a "little"... It's not a suggestion to modify the call, but to totally replace that filter ;)

[Jawed]

medp7060, the ReduceFlicker function is, generally speaking, erroneous in this context. I will be removing it entirely from FizzKiller. Although it seems OK for this clip it will create visual problems in other clips. It may be causing visual problems in this clip as well, but it's hard to see amongst all the stuff happening!

I am currently experimenting with alternatives, so have removed it from the testing that I'm doing.

Also please note that you should replace Vinverse() with TDeint() for better de-interlacing.

Jawed

[Gavino]

Oh well, I'll not refer to AvsP's field/frame info again.
Knowing a clip is 'framebased' tells you nothing at all about whether it is interlaced or progressive. Within Avisynth, it just means (more or less) that it has not been subject to a SeparateFields operation (which turns it into fieldbased). Hence all sources will show as framebased initially.

See here (http://avisynth.org/mediawiki/Interlaced_fieldbased) for more info.

[medp7060]

Thanks, Jawed. I am now able to test your script. Of course, after correcting the incorrect syntax and removiong the ReduceFlicker.

Sagekilla and thetoof: Thanks for your constructive comments.

[Didée]

EDIT: it turns out ReduceFlicker is key to the effectiveness of FK on this clip :eek:
No. You didn't reckognize the keypoint here. This clip is very noisy. If motion search is done on the not-filtered (or only very lightly filtered) original clip, then the standard scenechange thresholds of MVTools are too low. Many frames exceed the scenechange threshold, and therefore the denoising efficiency drops seriously.
If you don't use ReduceFlicker in the "calm=..." section, and add e.g. "...,thSCD1=800,thSCD2=96)" to the MVDegrainMulti() line, then you get "full efficiency" again.

So ReduceFlicker is in no way "the key". It's only about the thresholds. ReduceFlicker just helped to get past a barrier that you didn't consider.


I just tried UDegrain as you suggested but it's not really working as well as FK.

See attachment, FK on the left and UD on the right.
UDegrain basically suffers from the same problem. Overall it doesn't seem to work very well with this source, agreed. But: if you raise thSCD1 to 800 in UDegrain, too, then UDegrain suddenly starts to denoise much more than with standard settings.

[Sagekilla]

On another note, how effective is Grayscale().Invert("Y").Levels(0,0.5,255,0,255,coring=false)? I tried testing it using a relatively simple script but there was little (if any) difference between using the inverted clip or the source clip as the search clip.

[Didée]

Dunno ... perhaps it's another "key" feature of FizzKiller().;) This inverted-gamma-thingy was discussed in the original FizzKiller thread (http://forum.doom9.org/showthread.php?t=133977).

a) Invert().Gamma(1/x)[.Invert()] is the same as Gamma(x). In practice it's slightly different, because Invert() flips the source around Y=128; however the standard YUV range is not symmetric to 128, but to (16+235)/2 = 125.5 (usually handled as 126). However, this small difference is pretty much irrelevant in the given context.
From the relevant part of what's happening, the inverting & inverse-gamma'ing is pointless. Plain Gamma() yields to the same result.

b) What happens due to the gamma adjustment is that local contrast gets expanded in dark areas, and squeezed in bright areas. Therefore, grain in bright areas will cause smaller SADs (since signal peaks got a bit smaller by the gamma operation), eventually resulting in a bit more grain removal in bright areas. But otoh, in dark areas the resulting SADs will get bigger (since pixel differences were blown up by the gamma operation), resulting in less denoising in dark areas.


It's a specific tweak that might or might not work out for the specific case it was made for. From a "general" point of view, it's a soufflé ... when you curiously start looking inside, it goes *poof*.

[Jawed]

No. You didn't reckognize the keypoint here. This clip is very noisy. If motion search is done on the not-filtered (or only very lightly filtered) original clip, then the standard scenechange thresholds of MVTools are too low. Many frames exceed the scenechange threshold, and therefore the denoising efficiency drops seriously.
If you don't use ReduceFlicker in the "calm=..." section, and add e.g. "...,thSCD1=800,thSCD2=96)" to the MVDegrainMulti() line, then you get "full efficiency" again.
Thanks. Those are parameters that I've not learnt how to use, so I shall go play :)

Jawed

[Jawed]

From the relevant part of what's happening, the inverting & inverse-gamma'ing is pointless. Plain Gamma() yields to the same result.
Since gamma(2) didn't produce the same result, as I noted in the FizzKiller thread, I left it alone.

b) What happens due to the gamma adjustment is that local contrast gets expanded in dark areas, and squeezed in bright areas. Therefore, grain in bright areas will cause smaller SADs (since signal peaks got a bit smaller by the gamma operation), eventually resulting in a bit more grain removal in bright areas. But otoh, in dark areas the resulting SADs will get bigger (since pixel differences were blown up by the gamma operation), resulting in less denoising in dark areas.
I deploy FizzKiller after doing TV->PC Levels (and follow up with gradfunkmirror) and I watch video using a carefully calibrated CRT, so maybe I'm just not seeing the "inferior dark area noise reduction". On the other hand I find the alternatives don't even touch the noise in bright areas, which is initially why I started mucking about with gamma-manipulation.

I'm simply tweaking FizzKiller with the aim of reducing the motion artefacts as much as possible while still benefitting from the power of MVDegrain. That's why it's "work in progress", something I'm still testing.

Jawed

[medp7060]

This clip is very noisy. If motion search is done on the not-filtered (or only very lightly filtered) original clip, then the standard scenechange thresholds of MVTools are too low.

I confirmed that. When I changed thiscalm = source .reduceflicker(strength=3,aggressive=true)

into to thiscalm = source #.reduceflicker(strength=3,aggressive=true)

the processing time for the first pass of two passes of Xvid was increased from 20 min into 19h on my machine.

If you don't use ReduceFlicker in the "calm=..." section, and add e.g. "...,thSCD1=800,thSCD2=96)" to the MVDegrainMulti() line, then you get "full efficiency" again.

Is this what you meant? MVDegrainMulti(vectors, refframes = refframes, thSAD = thSAD, idx = idx + 1,thSCD1=800,thSCD2=96)

Jawed:

Your original clip already worked efficiently. The quality improves a lot. Thanks.


I am not good at using these advanced filters as your guy do. I will wait and see if there are any further modifcations of the script. If not, I will use it to clean my original whole clip, which is 4 h long.

[Jawed]

Is this what you meant? MVDegrainMulti(vectors, refframes = refframes, thSAD = thSAD, idx = idx + 1,thSCD1=800,thSCD2=96)
That appears to work very well - just don't forget to remove the ReduceFlicker function :)

Jawed:

Your original clip already worked efficiently. The quality improves a lot. Thanks.
Hopefully you are using something better than VInverse() to de-interlace, as I was wrong to use that. If you don't want to de-interlace, well that's a whole other question!

I am not good at using these advanced filters as your guy do. I will wait and see if there are any further modifcations of the script. If not, I will use it to clean my original whole clip, which is 4 h long.
For the sake of performance you should try lowering the value of refframes in the line:

FizzKillerMulti(blksize=16, sharp=2, thsad=800, refframes=6)

As you lower the number there'll be more noise left over. It's up to you to decide what you like. If you are happy with setting refframes=3 (or less) then you should consider using the "official" version of MVTools, since it is faster. That requires rewriting the script though, because the "Multi" functions are not available. I can post that if you like.

I also tried UDegrain, as follows:

UDegrain(radius=7,settings="heavy",thSCD1=800)

The movement of the dancers is not smeared as it is with FizzKiller - it is doing less noise reduction in areas that are moving. It's much slower though. But you should try it as there's a good chance you'll prefer it.

Jawed

[Sagekilla]

@Didee: Except he doesn't actually do a plain Gamma conversion (as if it were a Invert.Invert) since he inverts it once and leaves it at that -- I don't see what difference having an inverted output would make though, or even the gamma adjustment. I've seen very, very little change in denoising capabilities with that in place, on the order of "If I use Overlay(a,b,mode="difference") I see nothing"

[Didée]

The 2nd Invert() doesn't matter. If you search vectors on [source], or on [source.invert], the result should be identical.

Well ... I was wrong with the statement that Invert().Gamma(1/x) would be the same as a direct Gamma(x); it's in fact different (think about mirroring around an axis, vs. a 180° turn around a point, there you have my mistake). However I was correct with the implications of such an operation, since that difference doesn't matter all too much. It can easily be shown, both theoretically and practically, that such an operation will actually worsen the overall denoising efficiency. Blown-up contrast leads to blown-up pixel differences leads to blown-up SADs leads to less denoising with MVDegrain (unless you compensate with higher SAD thresholds, which leads to more potential artifacts). At times it even makes a quite visible difference.

[Sagekilla]

So it's sort of like using x264 --placebo (--me tesa) ...

[LaTo]

Source
http://latoninf.free.fr/udegrain/SRC.png

FizzKiller
http://latoninf.free.fr/udegrain/FK.png

Udegrain_v1alpha(settings="insane")
http://latoninf.free.fr/udegrain/UD.png

:p

[Didée]

Keep your tongue. Using TemporalDegrain (which is a comparably *simplistic* script) :
TemporalDegrain(degrain=2,sigma=16,hq=0,SAD1=9999,SAD2=640) # no need for 50 temporal frames ... 2 suffice.
limitedsharpenfaster(ss_x=1.0,ss_y=1.0,strength=60,soft=40,overshoot=0)

http://img379.imageshack.us/img379/5198/temporaldegrainlsgq9.png (http://imageshack.us)

Not in the mood to tweak around ... seems it's competitive enough. :)

[Terranigma]

imho, the image by Didée is much more pleasing. :cool:

[Adub]

The one from LaTo seems a little calmer in my opinion. For instance, the brownish-redish back wall on the left.

[LaTo]

For me, the one by Didée is a bit "plastic"... It's look like a paint, I don't like this.
But with only 2 refs it's impressive :eek: (I have used 4 refs)

[Didée]

As said, I didn't tweak anything. Settings for TemporalDegrain were guesstimated, and left at that. Tried one shot with strength=80,soft=25 in LSF, then tried 60/40, and left at that.

Perhaps it's a little too harsh, yeah ... exactly the two-liner from above, with some additional weighting-in of a spatial smoother:
...#as above, plus
bb=removegrain(11).removegrain(20).removegrain(11)
merge(bb,0.2).mt_lutxy(bb,"x y > x 1 - x y < x 1 + y ? ?",U=3,V=3)

http://img384.imageshack.us/img384/7840/temporaldegrainlssmoothjq3.png (http://imageshack.us)

This, too, is a one-time shot without any "tweaking".

Fact is, with this very noisy and distorted source, there are no real miracles to expect. In the end it's mostly a matter of how much blurring you're going to accept ...

[corporalgator]

Have you tried mc_spuds?

http://forum.doom9.org/showthread.php?t=131279&highlight=mc_spuds

[Didée]

That, and quite some others. If you search around abit, you'll easily find ten or twenty different mocomp-denoisers. Take half an hour, or one or two hours for tweaking-around with each script, and for sure you'll be busy ...
... and in the end, it's like when you're going on a mushroom foray: it's pretty hard to tell the blueish redcap from the reddish bluecap ...

[Undead Sega]

I had a go, might not be perfect, but i can be happy with this:

http://rapidshare.com/files/144442476/TV_2.mpg.html

Your views please :D

[Delta2]

[QUOTE=Didée;1182172]
TemporalDegrain(degrain=2,sigma=16,hq=0,SAD1=9999,SAD2=640) # no need for 50 temporal frames ... 2 suffice.
limitedsharpenfaster(ss_x=1.0,ss_y=1.0,strength=60,soft=40,overshoot=0)

Hi, Didée

I'm interested to test that code, but I get this error :

CAVIStreamSynth: System exception - Acess Violation at 0x0, reading from 0x0

I have all the dependent filters, but maybe the wrong version of one of them

Can you please give me that filters in a package to download ?

[talen9]

@Delta2: try putting a line like this on the top of the script:

SetMemoryMax(512)

and see if it works better.

[Delta2]

@Delta2: try putting a line like this on the top of the script:

SetMemoryMax(512)

and see if it works better.

no, it doesn't

pherhaps is a buggy version of any of this dependent filters :

LoadPlugin("MVTools.dll")
LoadPlugin("RemoveGrain.dll")
LoadPlugin("hqdn3d.dll")
LoadPlugin("FFT3DFilter.dll")
LoadPlugin("mt_masktools.dll")
LoadPlugin("repair.dll")
LoadPlugin("LimitedSupport.dll")

Can you give me yours ?

[Didée]

LimitedSupport.dll isn't needed at all, those functionalities now are included in mt_masktools.dll.

For RemoveGrain and Repair, I use these (http://home.arcor.de/kassandro/RemoveGrain/RemoveGrain.rar) ones.

Remember that for FFT3DFilter() to work, you need to put fftw3.dll in your /system32 folder.

Please post your *complete* script.

[Delta2]

LimitedSupport.dll isn't needed at all, those functionalities now are included in mt_masktools.dll.

For RemoveGrain and Repair, I use these (http://home.arcor.de/kassandro/RemoveGrain/RemoveGrain.rar) ones.

Remember that for FFT3DFilter() to work, you need to put fftw3.dll in your /system32 folder.

Please post your *complete* script.

Thanks for the help

Here is the scripts :

http://rapidshare.com/files/144469387/temp.rar.html

I have AVISynth version 2.5.7, and I don't have fftw3.dll but only FFT3DFilter.dll

[Terranigma]

I don't have fftw3.dll

And that's the problem. Go here (http://www.fftw.org/install/windows.html) and download fftw 3.1.2., extract, rename "libfftw3f-3.dll" to fftw3.dll and move it to system32.

[Didée]

That's probably it. BTW, I can't DL that script from rapidshare, Undead Sega's clip is still downloading. (Rapidshare was fast in the past, but lately they throttle a lot, capping at 50K/s most of the time...)

Just post your script here ... copy/paste it into your message, and put / - tags around it (without the '_' underscore).

[Terranigma]

Delta2's temp archive.
It contains 2 scripts, one entitled "denoiser2", and the other entitled "temporaldegrain".


denoiser2

SetMemoryMax(512)
LoadPlugin("MVTools.dll")
LoadPlugin("RemoveGrain.dll")
LoadPlugin("hqdn3d.dll")
LoadPlugin("FFT3DFilter.dll")
LoadPlugin("mt_masktools.dll")
LoadPlugin("repair.dll")
LoadPlugin("LimitedSupport.dll")
Import("TemporalDegrain.avs")

DirectShowSource("TV.mpg")

TemporalDegrain(degrain=2,sigma=16,hq=0,SAD1=9999,SAD2=640) # no need for 50 temporal frames ... 2 suffice.
limitedsharpenfaster(ss_x=1.0,ss_y=1.0,strength=60,soft=40,overshoot=0)


& TemporalDegrain is the August 2nd version.

[Didée]

AAaaargh ... directshowsource'ing an MPEG! Nooo! Twenty strikes on the footsoles! :)

Get the DGMPGDec (http://www.neuron2.net/dgmpgdec/dgmpgdec.html) package, index the *.mpeg file with DGIndex, then import() the dgdecode.dll and use Mpeg2Source(path/to/*.d2v) to load it in your script.

[Delta2]

Sorry, I put in another server

I made a package with all dependent filters

http://avi2iso.planetaclix.pt/testing.rar

I also put that DLLs in System32

libfftw3l-3.dll
libfftw3f-3.dll
libfftw3-3.dll

But without sucess, does they need to be registered ?

I even restarted my computer

[Delta2]

AAaaargh ... directshowsource'ing an MPEG! Nooo! Twenty strikes on the footsoles! :)

Get the DGMPGDec (http://www.neuron2.net/dgmpgdec/dgmpgdec.html) package, index the *.mpeg file with DGIndex, then import() the dgdecode.dll and use Mpeg2Source(path/to/*.d2v) to load it in your script.


I thought it was a MPEG1 stream, but it is not

Will try that

[Didée]

That's the fftw3.dll from my system32: dll (http://home.arcor.de/dhanselmann/_stuff/fftw3.rar)

[Delta2]

damn, not even so, strange, could it be myt AVISynth version ?

here is the script code with the changes :


SetMemoryMax(512)
LoadPlugin("MVTools.dll")
LoadPlugin("RemoveGrain.dll")
LoadPlugin("hqdn3d.dll")
LoadPlugin("FFT3DFilter.dll")
LoadPlugin("mt_masktools.dll")
LoadPlugin("repair.dll")
LoadPlugin("LimitedSupport.dll")
LoadPlugin("DGDecode.dll")

Import("TemporalDegrain.avs")

Mpeg2Source("TV.d2v")

TemporalDegrain(degrain=2,sigma=16,hq=0,SAD1=9999,SAD2=640) # no need for 50 temporal frames ... 2 suffice.
limitedsharpenfaster(ss_x=1.0,ss_y=1.0,strength=60,soft=40,overshoot=0)

[Didée]

Oh dear.

1) Does this work for you?

.
.
Mpeg2Source("TV.d2v")
return( FFT3DFilter() )

2) Does this work for you?
.
.
Mpeg2Source("TV.d2v")
return( RemoveGrain(19) )

3) Does this work for you?
.
.
Mpeg2Source("TV.d2v")
o=last
v=o.MVAnalyse(pel=2,overlap=4)
Return( o.MVCompensate(v) )

[Delta2]

This code, works :


LoadPlugin("DGDecode.dll")
LoadPlugin("FFT3DFilter.dll")

Mpeg2Source("TV.d2v")
return( FFT3DFilter() )


Also run :


LoadPlugin("DGDecode.dll")
LoadPlugin("RemoveGrain.dll")

Mpeg2Source("TV.d2v")
return( RemoveGrain(19) )


This doesn't work :


LoadPlugin("DGDecode.dll")
LoadPlugin("MVTools.dll")


Mpeg2Source("TV.d2v")
o=last
v=o.MVAnalyze(pel=2,overlap=4)
Return( o.MVCompensate(v) )


There is no function MVAnalyse

[Didée]

Oh sorry, I misspelled "MVAnalyze()", correct is MVAnalyse(). Try that again.

[Delta2]

Oh sorry, I misspelled "MVAnalyze()", correct is MVAnalyse(). Try that again.

Well, with so many trying to replace filters I didn't notice the bad english word

It works, but Denoisers.avs that I gave another link to download gives the same error

CAVIStreamSynth: System exception - Access Violation at 0x20f2edc, writing to 0x48

I updated to AVISynth last version, also without sucess

I think it is useless, gosh

[Didée]

No not useless. Just a bit tricky, you clobbered together some whacky stuff.

First, your version of MVTools is causing problems. Too lazy to hunt hashsums to identify what your verion is ... go here (http://avisynth.org.ru/mvtools/mvtools.html), get the latest version (1.10.2.1), use that.

Also, you have an older version of LimitedSharpenFaster that indeed makes use of LimitedSupport.dll ... that should be actualised. ... Oops, it *must* be changed. I'm getting an exeption with LSF, too, when using your script+plugins.


But first, get MVTools v1.10.2.1, and see if the script with TemporalDegrain and without LimitedSharpenFaster does run.

[Delta2]

Finally it works, Didée

I changed your LimitedSharpenFaster to ( a modded version, 29 Oct 2005 )

And installed the suggested MVTools version ( 1.10.2.1 )

Et voilá, works even with DirectShowSource without a problem

What are the dependencies for LimitedSharpenFaster ?

many thanks

[Didée]

Hooray!

However, your script is not good, yet. The input must be deinterlaced before TemporalDegrain and LSF (or any other denoising/sharpening filters) do their work.

Do you know how to deinterlace? Options there are plenty ... from a simple blur(0,1) [or soft downsizing], over several adaptive deinterlacers like eg. FieldDeinterlace, TDeint, Yadif, etc.pp., up to the big weapons of mocomp-deinterlacers ...

[Delta2]

Hooray!

However, your script is not good, yet. The input must be deinterlaced before TemporalDegrain and LSF (or any other denoising/sharpening filters) do their work.


Applying to all ?! even a progressive generic movie ?

And does that apply to your LimitedSharpenFaster scripts also ?



Do you know how to deinterlace? Options there are plenty ... from a simple blur(0,1) [or soft downsizing], over several adaptive deinterlacers like eg. FieldDeinterlace, TDeint, Yadif, etc.pp., up to the big weapons of mocomp-deinterlacers ...


Yes, and there is plenty of them, which do you suggest ? one that is not so slow

Can you give a full code example to this case study ?

[Didée]

For progressive sources, of course you don't need to deinterlace.

In had in mind processing the clip "TV.mpeg" of medp7060, which is interlaced. It seems you are loading that very clip in your script, too, so I was referring to that one.

[medp7060]

I compared them from LaTo and Didée:

Udegrain_v1alpha(settings="insane")


Keep your tongue. Using TemporalDegrain (which is a comparably *simplistic* script) :
TemporalDegrain(degrain=2,sigma=16,hq=0,SAD1=9999,SAD2=640) # no need for 50 temporal frames ... 2 suffice.
limitedsharpenfaster(ss_x=1.0,ss_y=1.0,strength=60,soft=40,overshoot=0)


...#as above, plus
bb=removegrain(11).removegrain(20).removegrain(11)
merge(bb,0.2).mt_lutxy(bb,"x y > x 1 - x y < x 1 + y ? ?",U=3,V=3)



Source---------------------------------------------------------------------- LaTo (2nd)
http://latoninf.free.fr/udegrain/SRC.pnghttp://latoninf.free.fr/udegrain/UD.png

Didée (1st ------------------------------------------------------------------- 2nd):
http://img379.imageshack.us/img379/5198/temporaldegrainlsgq9.png (http://imageshack.us)http://img384.imageshack.us/img384/7840/temporaldegrainlssmoothjq3.png (http://imageshack.us)

I think they are comparable with Didée 2nd clearer, comapred with the sourrce.

Can you post the whole scripts here so I can have a go to tweak?

[Delta2]

For progressive sources, of course you don't need to deinterlace.


Misunderstandings, but it means that I must deinterlace before any sharpen or denoiser, right ?

I usually use Decomb/FieldDeinterlace for that purpose



In had in mind processing the clip "TV.mpeg" of medp7060, which is interlaced. It seems you are loading that very clip in your script, too, so I was referring to that one.


This is a case study, I'm trying to use a generic script to use in bad grainy sources...either progressive or interlaced

But I feel a little disappointed because there is DLLs dependencies that should be installed in System32 folder...which is a handicap for use this :(

[medp7060]

I had a go, might not be perfect, but i can be happy with this:

http://rapidshare.com/files/144442476/TV_2.mpg.html

Your views please :D

I just watched it and I must say that I am stunned at the result. You seemed to have restored the real colours with noises removed to the extreme!

Can you post your script for me? Thanks.

[Didée]

That's very denoised, and very soft. Smells like NeatVideo. (?)

[medp7060]

I will see if I can replicate the result from Undead Sega with his script for which I have requested.

Samples: source vs denoised by Undead Sega
http://img66.imageshack.us/img66/9413/tvsourcegu3.pnghttp://img76.imageshack.us/img76/1327/tvdenoisedud7.png

[Jeremy Duncan]

http://thumbnails5.imagebam.com/1313/812ad413120304.gif (http://www.imagebam.com/image/812ad413120304)

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\repairsse2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\dfttest.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\RemoveGrainSSE2.dll")

DirectShowSource("C:\Users\Donkey\Desktop\TV.mpg")

setmemorymax(1024)

converttoyv12()
DeHalo_alpha(brightstr=1.7, darkstr=1.7)
Spresso(limit=1, limitC=1, bias=85, biasc=85, rgmode=13, rgmodec=13)
Spresso(limit=1, limitC=1, bias=85, biasc=85, rgmode=16, rgmodec=16)
Spresso(limit=1, limitC=1, bias=85, biasc=85, rgmode=13, rgmodec=13)
dfttest(tbsize=1)
LimitedSharpenFaster(ss_x=1.0, ss_y=1.0, Smode=3, strength=175, overshoot=0, soft=60)

[Undead Sega]

It was Neat Video, and it was abit rushed and a tiny bit washed, but i did it in short time and probably can improve the results a tiny bit. Also what is not noticable after the high bitrate reencode, is that even though i cleaned all the noise, i did in Adobe Premiere Pro added noise, which did improve it visually.

i would say my example is very clean, but as i said, i tiny bit of tweaking would and more noticable added would be better. Also again, it is slightly sharpened.

[medp7060]

Thanks, Undead Sega. The visual quality was already very good now. Can you tell me what configuration you used in Neat Video? There are soo many options in the filter. I tried a few preset ones, but failed to get it improved as much as you did. There is also a plugin for Virtualdub in addition to Adobe Premiere. The option profiles are quite similar.


Didée: I followed you and used this


#LoadPlugin("MVTools.dll")
#LoadPlugin("RemoveGrain.dll")
#LoadPlugin("hqdn3d.dll")
#LoadPlugin("FFT3DFilter.dll")
#LoadPlugin("mt_masktools.dll")
#LoadPlugin("repair.dll")

mpeg2source("D:\TV.d2v")
TDeint()

TemporalDegrain(degrain=2,sigma=16,hq=0,SAD1=9999,SAD2=640) # no need for 50 temporal frames ... 2 suffice.
limitedsharpenfaster(ss_x=1.0,ss_y=1.0,strength=60,soft=40,overshoot=0)

bb=removegrain(11).removegrain(20).removegrain(11)
merge(bb,0.2).mt_lutxy(bb,"x y > x 1 - x y < x 1 + y ? ?",U=3,V=3)

I have managed to get it work, but I found there was a short "surging" effect, I will put a sample result here.

Edited: here is the sample http://rapidshare.com/files/147590474/TV_Didee.avi

New problem

The brightness is not not even, increased diagonally from bottom left corner to top right corner. This promlem is in the original clip, become more obvious after cleaning the noises.

I tried Level, or Tweek, all cannot get it equalized. Are there any filter can compensate the brightness diagonally?

[Jeremy Duncan]

try mine and see if you still get that.
how did it get so ugly, is it from a 15 year old vrc tape?

[medp7060]

Thanks, will try yours.