Suggestions for denoising *sample attached*

[Blue_MiSfit]

Hey folks,

I'm tackling my copy of Badlands today. It is VERY noisy and grainy. In fact, some scenes average over 13,000 kbit when I do a cq2 Xvid test using v3hr!

Anyway, here's the link to an m2v sample
http://rapidshare.de/files/21248726/badlands_clip.rar.html
My current script:

LoadPlugin("C:\program files\gordianknot\avisynthplugins\removegrainsse3.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\repairsse3.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\fft3dgpu.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\masktools.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\deen.dll")
import("C:\program files\gordianknot\avisynthplugins\seesaw.avs")

MPEG2Source("C:\Movies Work\Badlands\clip.D2V")

a=last
b=MPEG2Source("C:\Movies Work\Badlands\clip.D2V")

a=fft3dgpu(a, sigma=4, bt=4)
#a=deen(a)
a=removegrain(a,mode=2)

a=seesaw(a)

return(a)

#overlay(a,b,360,0)


As you can see I am currently using fft3dgpu and removegrain, and had experimented with deen as well. I am also using seesaw to recover lost detail from denoising. I also have an overly commented out for previewing filters.

Anway, I have managed to calm the image down a bit, but still lost too much detail IMO. The compression gain is about 23% with this script over an untouched version.

I must admit im pretty new to tuning fft3dgpu and have never done seesaw before. The defaults improve things, so I have left that alone for now.

Do any of you have ideas?

~MiSfit

[neuron2]

Your title asks about deinterlacing but the message talks only about denoising. Please correct it. Thank you.

[Daodan]

From my tries of SeeSaw it's a pretty powerfull sharpener with defaults. Not that recommended if the original was noisy (by the way I can't download that clip since rapidshare doesn't work for me). I would either remove the seesaw and use LSF with edgemode=1 (it doesn't recover lost detail but it enhances a bit the details but not the noise). Maybe you will like this effect. Also try a temporal smoother since usually noise is temporal, like ttempsmooth.

[Blue_MiSfit]

@Neuron2, how do I edit the title of the thread? I did some searching with no avail. I tried editing the original post but didnt see the title field.

Sorry, its such a simple task and I feel like a fool :)

@Daodan,
I will definately try those out and report back.

~MiSfit

edit mod: I changed the title for you :)

[Pookie]

Have you seen this thread ?

http://forum.doom9.org/showthread.php?t=110078

[neuron2]

I changed the title for you Thank you, Sir. May your stay at the forum be long and fruitful.

[Blue_MiSfit]

Thanks :)

@Pookie
That is one serious freaking denoiser script. It took me 20 minutes to resolve the plugin dependancies alone! I am currently running a quick series of tests using the suggested "medium" and "high" settings on that thread you gave me. It looks promising, if still pretty grainy. We will see.


The tests look very good, taking care of the noise without removing a noticeable amount of sharpness and detail. There is some other artifact in there now. Definately not grain or noise, I think it's a byproduct of the heavy denoising and sharpening.

Here is my current script (for the whole movie that is. I'm running that encode right now)

setmemorymax(384)

LoadPlugin("C:\program files\gordianknot\dgmpgdec\dgdecode.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\fluxsmooth.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\removegrainsse3.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\repairsse3.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\removedirtsse2.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\masktools.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\deen.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\mvtools.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\vaguedenoiser.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\deblock.dll")
LoadPlugin("C:\program files\gordianknot\avisynthplugins\mt_masktools.dll")

import("C:\program files\gordianknot\avisynthplugins\RemoveNoiseMC.avs")
import("C:\program files\gordianknot\avisynthplugins\SeeSaw.avs")
import("C:\program files\gordianknot\avisynthplugins\limitedsharpen.avs")

MPEG2Source("C:\Movies Work\Badlands\badlands.D2V")

RemoveNoiseMC(rdlimit=18,rgrain=3,denoise=14)
VagueDenoiser(threshold=0.6, nsteps=6, chromaT=0.6, percent=75)
deen("a3d",rad=3,thrY=3,thrUV=5,min=0.25,tthY=2,tthUV=3,scd=6)
LimitedSharpenFaster(Smode=4,strength=24,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)


Look at all those plugins!

I will upload the version of my previous sample compressed to cq2 xvid with the same settings as this script later, and report on the quality of the final encode.

I will also have to try out LSF and ttempsmooth, though I dont know if those can provide the sheer denoising potential that this really filthy movie needs :)

Thanks for the help


~MiSfit

[Pookie]

I tried a couple of tests on your source file. That's a really difficult one. What I did notice is that the grain removal worked much better if I supersampled before degraining. The drawbacks are slowwwwww fps performance and having to look out for haloing. Still, the difference is significant.


If you want to try it out, it would be

tdeint(mode=-1,type=3).turnleft().tdeint(mode=-1,type=3).turnright()

then
degraining / seesaw (with a strong sharpening level )/ resize to 720x480 again


I'll bet eedi2 would work as an alternate to tdeint, but I don't know if the performance would be any better.

[*.mp4 guy]

Thats a tough source, well heres my attempt.

LoadPlugin("C:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")
Import("D:\media\Plugins\SeeSaw.avs")
Import("D:\media\Plugins\BlindDeHalo3.avs")
Import("D:\media\Plugins\LimitedSharpen().avs")
LoadPlugin("D:\media\Plugins\fft3d\fft3dfilter.dll")

mpeg2source("C:\Captures\CLIP.DEMUXED.d2v").BlindDeHalo3(rx=2.5, ry=2.5, strength=120, sharpness=.2)

fft3dfilter(bw=24, bh=48, ow=12, oh=24, bt=4, sigma=2.33, sigma2=3, sigma3=2.33, sigma4=2, plane=0).FrFun7(1.2,7,2)

addgrain(2,.1,.1)
LimitedSharpen(SS_X=2, SS_Y=2, strength=100, overshoot=1)

crop(4,0,710,480)
LanczosResize(720,400)

Example Pic (http://img380.imageshack.us/img380/6341/example7ws.png)

Example Clip (http://rapidshare.de/files/21400193/Example.avi.html)

[foxyshadis]

EEDI2 performance would be worse, edge quality higher.

However, you could probably use spline16resize (spline is often best for large upsize but not useful for downsize) by ~150%-200% (1.25x-1.5x) and realize a major performance boost, instead of doing it at 400%. Plus, while you're up there, you can do some sharpening anyway, since supersampled sharpening leaves less edge aliasing.

Naturally this depends on the content, some will do better than others this way.

[donnyj]

WOW, I am really impressed, great job!!!

[Blue_MiSfit]

Very impressive *.MP4 guy! I can't believe that is the same source!

A bit too much detail is lost IMHO. Questions:
1) could fft3dgpu be substituted for fft3dfilter? or are you using features that arent implimented inboth

2) could the settings be adjusted to leave a bit more detail & noise?

3) Good point about using BlindDeHalo, there is lots of haloing in this source.

~MiSfit

[Pookie]

Great suggestions, foxyshadis. Speed improved significantly by using Spline16resize.

[tsp]

Blue_MiSfit: currently sigma2,sigma3 and sigma4 isn't implemented in fft3dgpu but try this setting and see if it is okay:
fft3dgpu(bw=32,bh=64,bt=4,precision=2,sigma=2)

[*.mp4 guy]

Very impressive *.MP4 guy! I can't believe that is the same source!

A bit too much detail is lost IMHO. Questions:
1) could fft3dgpu be substituted for fft3dfilter? or are you using features that arent implimented inboth
you could use fft3dgpu, but you would loose some quality.


2) could the settings be adjusted to leave a bit more detail & noise? Yes, you can tweak fft3d's sigma values and frfun's first and second values (the first is strength, the second is like a noise threshold for luma) to change the amount of detail/noise kept.

3) Good point about using BlindDeHalo, there is lots of haloing in this source.
you might wan't more or less halo removal (I didn't remove them completely because doing so murdered sharpness).

Imo the source has almost no detail, the noise is just so bad its hard to tell. Also the example clip was made with H.263 quantisization at Q2.5, so its a little softer then the actual avs output.

[edit] Forgot to mention that SeeSaw isn't suitable for this clip becuase all it does is a really good job of restoring the noise that just got killed and enhancing the halo's.

[Blue_MiSfit]

Yeah it's a really bad source. I saw the VHS version in one of my fillm classes, and it actually had better image quality IMO, same detail, less noise. It's an interesting movie though. Honestly, I am considering not even bothering with re-encoding it at this point. The source is a DVD5 flipper disc, so I think I will just back it up using DVD+R... Why not, you know? Still, its been very interesting trying to clean up this source. AviSynth is very cool.

~MiSfit

[myck]

Hello. I used the script from *.mp4 guy, but I got some weird vertical lines on final video. I attached a sample picture. Can someone tell me what is wrong? here is my script :


LoadPlugin("e:\captura\filtre\DGDecode.dll")
LoadPlugin("e:\captura\filtre\mt_masktools.dll")
LoadPlugin("e:\captura\filtre\RemoveGrain.dll")
LoadPlugin("e:\captura\filtre\frfun7.dll")
LoadPlugin("e:\captura\filtre\addgrain.dll")
LoadPlugin("e:\captura\filtre\fft3dfilter.dll")

Import("e:\captura\filtre\SeeSaw.avs")
Import("e:\captura\filtre\BlindDeHalo3_mt2.avs")
Import("e:\captura\filtre\LimitedSharpenFaster.avs")

DirectShowsource("e:\captura\windv\curte\test.avi").convertToYV12().BlindDeHalo3(rx=2.5, ry=2.5, strength=120, sharpness=.2)

fft3dfilter(bw=24, bh=48, ow=12, oh=24, bt=4, sigma=2.33, sigma2=3, sigma3=2.33, sigma4=2, plane=0).FrFun7(1.2,7,2)

addgrain(2,.1,.1)
LimitedSharpenFaster(SS_X=2, SS_Y=2, strength=100, overshoot=1)

Edit: Thanks Blue_MiSfit. Here is the link from imageshack:
dungi.jpg (http://img301.imageshack.us/img301/5848/dungi4wx.jpg)

[Blue_MiSfit]

I'd reccommend using ImageShack/PhotoBucket or your ISP's webspace to host images. Image attachments on the forum are somewhat frowned upon by the mods if memory serves.

[*.mp4 guy]

Replace LimitedSharpenFaster with LimitedSharpen, that should fix the problem.

[Didée]

What, no serious trial except for mp4guy's cartoon? Here's mine from two days ago:

sample on megaupload.com (http://www.megaupload.com/?d=XACFREZJ) (Technology research demo - too slow for real usage, yet.)

Wasn't tweaked to the max (ringing of halos' edges, tad of sharpness). It's just a sample on grain removal, and the source is still alive. Not destructed.

With SixOfNine or eqmv3hr @ q2, compression gain on such highly grainy sources is in range 35% to 70%, scene dependent. :)

[myck]

Replace LimitedSharpenFaster with LimitedSharpen, that should fix the problem.

I did that but that didn't resolve the problem. I have added a new line "LoadPlugin("e:\captura\filtre\masktools.dll")", and I replaced LimitedSharpenFaster with LimitedSharpen, but the stripes are still there. Any other suggestions?

[tsp]

myck: Try commenting out 1 filter at a time and see what causes it.

[myck]

Ok. The problem is in this line: "fft3dfilter(bw=24, bh=48, ow=12, oh=24, bt=4, sigma=2.33, sigma2=3, sigma3=2.33, sigma4=2, plane=0)", but I don't know where and I'm not able to find it.

Didee: What script did you use? It's a great quality.

[*.mp4 guy]

What, no serious trial except for mp4guy's cartoon? Here's mine from two days ago:

sample on megaupload.com (http://www.megaupload.com/?d=XACFREZJ) (Technology research demo - too slow for real usage, yet.)

Wasn't tweaked to the max (ringing of halos' edges, tad of sharpness). It's just a sample on grain removal, and the source is still alive. Not destructed.

With SixOfNine or eqmv3hr @ q2, compression gain on such highly grainy sources is in range 35% to 70%, scene dependent. :) To be honest The grain removal looks about the same for both scripts, your halo removal however is much more effective then... well your old halo removal ;) (which I probably didn't use to its best effect). Although I did have FrFun on a bit too aggressive settings, (1.1,4,2) works a little better.

Ok. The problem is in this line: "fft3dfilter(bw=24, bh=48, ow=12, oh=24, bt=4, sigma=2.33, sigma2=3, sigma3=2.33, sigma4=2, plane=0)", but I don't know where and I'm not able to find it.

Didee: What script did you use? It's a great quality.Make sure that the video is mod 16 throughout the processing steps, and If you still have the same problem make sure fft3dfilter is up to date, if that doesn;t work try "fft3dfilter(bw=48, bh=48, ow=12, oh=24, bt=4, sigma=2.33, sigma2=3, sigma3=2.33, sigma4=2, plane=0)".

[tsp]

myck: Does

DirectShowsource("e:\captura\windv\curte\test.avi").convertToYV12().BlindDeHalo3(rx=2.5, ry=2.5, strength=120, sharpness=.2)

fft3dfilter(bw=24, bh=48, ow=12, oh=24, bt=4, sigma=2.33, sigma2=3, sigma3=2.33, sigma4=2, plane=0)

produces the same problem?
If that is the case try uploading a 10 frame sample from the clip and send Fizick a PM about it.

[myck]

I resolved the problem. It was from the version of fft3dfilter. I got one from http://www.avisynth.org/warpenterprises/ which I believed to be the lastest. But I found one newer than that, and now it works. Thanks to all for your replies.

[Pookie]

Nice job on that one, Didee, especially if you got more than 2fps performance. Yeah, the guy's head against the river is a halo waiting to happen.