What denoiser is the worst ? Objective test attempt [Archive]

Fizick

11th August 2006, 00:05

What denoiser is the best? It is good, but not-allowed question here at the forum. :readrule:
Why? Becourse it is subjective, and dependent from your impression and the source type, and the noise type, codec, etc, etc.
Subjective rating is not well defined, and is possible probably by some poll only.
Legal question: what denoiser from the list is the currently better for some given video with given noise value and type, by some objective criterium?
But what compare, and how to compare?
In scentific publications it is not a new question.

Typical procedure:
1. Use some standard test sequence (clean, free of noise).
2. Add given noise amount, with normal distribution.
3. Clean this noisy sequence by denoiser.
4. Compare cleaned result with original sequence, calculate some metric (signal to noise ratio by PSNR, SSIM, VQM criterium)

(This method is also often used in image processing with test images "Lena", etc.)

I suggest use similar method here.
Lets make our own comparison!
(I often use it for my filter development).

Step 1.
There are many standard freely-redistributable test sequences (used in video codec development, etc).
I found and downloaded some of them.
Lets use so called "Flower" sequence. It has quite small file size and has some movement in it.
The original is at
http://bmrc.berkeley.edu/ftp/pub/multimedia/mpeg/EncodeData/
First part (frames 0 to 29) is in the file "flower.0-29.tar.z"
Direct link: http://bmrc.berkeley.edu/ftp/pub/multimedia/mpeg/EncodeData/flower.0-29.tar.z
This archive file can be unpacked by 7-zip, Winrar or some other archiver tools.
Archive contains image files sflowg.0.yuv to sflowg.29.yuv.
This raw YUV files has I420 format (planar 4:2:0 format similar to YV12, but with swapped U, V planes). Width=352, height=240

(not interlaced).
The file sequences can be open with ImageSequence plugin by WarpEnterprises
(Get it at http://www.avisynth.org/warpenterprises)
and we get normal YV12 video.

rawsequence("h:\flower\sflowg.%d.yuv",width=352,height=240,pixel_type="I420").trim(0,29)

Realy, video is not quite normal (it has colors range from 0 to 255), so lets convert it to recomended TV range (16-235):

source=coloryuv(levels="PC->TV")

Step 2.
Lets add some grain with AddGrain plugin by trbarry (or use modified version AddGrainC by Foxyshadis for YUY2, chroma).
(Get it at http://www.avisynth.org/warpenterprises)
Added noise amount is controlled by first parameter. It is really sigma*sigma (or noise variation),
where sigma is the noise standard deviation.
Let's use only luma (Y) channel for testing.
Lets use sigma=5 for test. It is rather big value, and the noise is visible.
(In scientific articles larger sigmas are usually used, about 10-20).
Noise variation parameter = 5*5=25.

noisy=source.addgrain(25,0,0)

For temporal denoisers, noise must not be the same for all frames, so we can not use seed option of AddgrainC.

Step 3.
Use your favorite denoiser here!
The null-transform denoiser:

denoised=noisy

Of course, I use FFT3DFilter :) as example.

denoised=noisy.FFT3DFilter(sigma=4.0, bt=4, bw=32, bh=32, ow=16, oh=16)

Step 4.
Lets use SSIM metric for camparison of denoised with original source images. (SSIM is better than PSNR).
Well known SSIM plugin by Lefungus
(Get it at http://www.avisynth.org/warpenterprises).
EDIT: It seems, that various versions of SSIM plugin exists, without source. :(

SSIM(source,denoised,"nul","nul")

I firstly used "nul" as dummy output files.
Now I use full info:

SSIM(source,denoised,"ssim.txt","ssimA.txt")

Open script in VirtualDubMod.
We get SSIM values typed on frame.
Skip some first frames 0,1,2,3 (for temporal denoisers), lets will consider frame 4.
The values for noisy video (null-transforn denoiser) is:
n=4
Y=0.9343 U=1.0000 V=1.0000
SSIM=0.9474 Wieght=1.0000

In this test, by my first suggestion,
we are interested not in summary SSIM, but in Y (luma) SSIM value (=0.9343) only! Chroma is not changed.
Reported last summary SSIM is average of channels (0.8*Yssim + 0.1*(Ussim + Vssim)).
But since this plugin write summary SSIM (not luma) to output file SSIM.TXT,
we now will use this summary per-frame SSIM for comparing.
So, play scipt in any player, and output files will be created.
File SSIMA.TXT will contain average clip SSIM.

--------------------------------------------------
Full test script:

rawsequence("h:\flower\sflowg.%d.yuv",width=352,height=240,pixel_type="I420").trim(0,29)
coloryuv(levels="PC->TV")
source=last
noisy=source.addgrain(source,25,0,0)
denoised=noisy#.DENOISER(parameters) # put your real denoiser script here
SSIM(source,denoised,"ssim.txt","ssimA.txt")

Here is result for FFT3DFilter v.1.85.
My best settings is:

denoised=noisy.FFT3DFilter(sigma=4.0, bt=4, bw=32, bh=32, ow=16, oh=16)

FFT3D result:
n=4
Y=0.9794 U=1.0000 V=1.0000
SSIM=0.9835 Wieght=1.0000

DeGrainMedian best script (almost default!):

denoised=noisy.degrainmedian(mode=1,limity=4,limitUV=0)

But result is not so great:
n=4
Y=0.9525
SSIM=0.9620

So, FFT3Dfilter SSIM score is above DegrainMedian.
FF3DFilter is better in this thread. :)
I will tune Vaguedenoiser, MVDegrain soon. :devil:

Please download the test clip "Flower" (link is above), tune your favorite denoiser plugin (or script) and post results!
:script:

You may also post speed results (but they are system-dependent).

tritical

11th August 2006, 02:27

With the way you are doing it (adding noise to the sequence with addgrain() during each run) wont the noise be different for each test? Sure, it will have the same variance, but it wont be exactly the same for each filter tested, which I think is important for comparison of the results (correct me on this if I'm wrong). It would be nice if we could generate a few lossless noisy versions of the sequence (maybe with noise std of 2.5, 5, and 10) along with a lossless clean version (just to make it a single clip that can be loaded with avisource()) and then everyone could download and use those for the tests... that way the noise would be the same for all. Since the flower sequence is only 30 frames the clips shouldn't be very large (a few MBs), and a fast lossless codec should be fine (lagarith would be my pick). I would be able to make and host the clips if needed.

Also, for fft3dfilter and degrainmedian you posted the result only for frame 4. Why look at only a single frame instead of results over the entire sequence?

Anyways, I definitely think this would be interesting as there have been a lot of new denoisers (both scripted and stand-alone filters) recently

Ferux

11th August 2006, 02:32

Why would one use FFT3DFilter if FFT3DGPU does the same, but faster?

Mug Funky

11th August 2006, 02:51

hmm. a standard test video is a good idea.

how about burning it to a rewritable DVD, playing it out through analog and recapturing, or dubbing to VHS and capturing, etc. could give a good idea of what people could expect from their various sources.

flood555

11th August 2006, 03:37

Why would one use FFT3DFilter if FFT3DGPU does the same, but faster?

One and only reason:Limitation of GPU RAM if script needs too many ...

neuron2

11th August 2006, 05:17

What denoiser is the best? It is good, but not-allowed question here at the forum. And since you know that, your violation is intentional. You had no need to mention "best" and could have simply referred to your objective test. Therefore, struck for rule 12.

Do not discuss this matter here, take it to PM or challenge it through proper channels. Violators will get a rule 3 strike.

Mug Funky

11th August 2006, 06:03

i believe this is an attempt at an objective test (says in the thread title and several times in the post), so at least in my own understanding "best" is applicable. if the situation is controlled enough, surely one can refer to the measurably better performing denoiser as the "best"?

i'm all for this test though - noise comes in all different forms, and it can't hurt to define what denoisers work best in what situations. this can really help the community, as there's a fair bit of confusion i've observed as to what denoiser to run and when.

one example i toy with often in my own work is the difference between removedirt and degrainmedian:

they both work in a similar manner and accomplish similar goals, but they still have their strengths and weaknesses, for instance removedirt will remove spots (often quite big ones) and stabilise shaky footage to a small extent, but on the flipside it'll spoil the picture if there's a regular shaking going on. degrainmedian doesn't do that, removes lots of grain, but will also leave in spots and not stabilise the motion as much. based on that we can't say what's "best", but if we define our goals rigidly toward spot removal or artefact prevention, suddenly we can validly say one is better than the other in this situation.

to reiterate: i think this thread is useful and should continue, perhaps with some re-wording to fit the forum rules better.

tritical

11th August 2006, 07:45

@Fizick
Just to give you an idea of what I was talking about earlier, I created/uploaded a rar file here (http://bengal.missouri.edu/~kes25c/denoise_test.rar) that includes:

the original source, 3 noisy versions of the source, ssim/yv12-psnr avs filters, and a test script

The original source clip was created using:

rawsequence("c:\flower\sflowg.%d.yuv",width=352,height=240,pixel_type="I420")
coloryuv(levels="PC->TV")

and compressed in vdub (fast recompress) using lagarith 1.3.9 in yv12 mode. The noisy clips were created using:

rawsequence("c:\flower\sflowg.%d.yuv",width=352,height=240,pixel_type="I420")
coloryuv(levels="PC->TV")
addgrainc(25,25,0,0)

where 25 was replaced with 9 and 100 in two of the instances. I used v1.3 of addgrain by foxyshadis, and, again, lagarith was used for lossless compression.

I know that ssim is usually preferred over psnr, but I think it would be interesting to see psnr results as well so I included psnr calculation in the provided script. Anyways, I think that having a single package that people can download, that contains the clips/needed comparison filters/test script would help quite a bit.

@Mug Funky
hmm. a standard test video is a good idea.

how about burning it to a rewritable DVD, playing it out through analog and recapturing, or dubbing to VHS and capturing, etc. could give a good idea of what people could expect from their various sources.
Sounds like a good idea, though from Fizick's post I was under the impression that he just wanted to use zero-mean gaussian white noise in this test (since he was using addgrain to add the noise).

tritical

11th August 2006, 10:07

Sorry for hijacking your thread Fizick, but I'm gonna be w/o internet for the next 4 days so have to get everything done quick :D. Some results using the noisy clip with noise variance of 25 from the rar file I posted:

tested filters:

sw-3ddct(NoiseParam=4) <= sw-3ddct (not an avisynth filter)

dctfun4b(12) <= dctfun4b

tnlmeans(h=3.5,bx=1,by=1,sx=3,sy=3,ax=6,ay=6,az=2) <= tnlmeans

frfun7(lambda=2.0,T=7.5) <= frfun7

frfun3d(t=7.0) <= frfun3d

fft3dfilter(sigma=3.8, bt=4, bw=16, bh=16, ow=8, oh=8, plane=0) <= fft3dfilter

dfttest(sigma=1.87,bsize=16,osize=12,tsr=1,ssr=0,max2dblocks=3) <= dfttest

vf=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,overlap=4,sharp=2,truemotion=true)
vb=noisy.mvanalyse(pel=2,blksize=8,isb=true,idx=1,overlap=4,sharp=2,truemotion=true)
vf2=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,delta=2,overlap=4,sharp=2,truemotion=true)
interleave(\
mvcompensate(noisy,vf2,idx=1,thSCD1=600)\
, mvcompensate(noisy,vf,idx=1,thSCD1=600)\
, noisy\
, mvcompensate(noisy,vb,idx=1,thSCD1=600))
FFT3DFilter(sigma=3.7, bt=4, bw=8, bh=8, ow=4, oh=4)
denoised=selectevery(4,2) <= fft3dfilter+emc

dfttest(sigma=1.87,bsize=8,osize=6,tsr=2,ssr=6,max2dblocks=4) <= dfttest+imc

vf1=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,overlap=4,sharp=2,truemotion=true)
vf2=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,delta=2,overlap=4,sharp=2,truemotion=true)
vb1=noisy.mvanalyse(pel=2,blksize=8,isb=true,idx=1,overlap=4,sharp=2,truemotion=true)
vb2=noisy.mvanalyse(pel=2,blksize=8,isb=true,idx=1,delta=2,overlap=4,sharp=2,truemotion=true)
interleave(\
mvcompensate(noisy,vf2,idx=1,thSCD1=600)\
, mvcompensate(noisy,vf1,idx=1,thSCD1=600)\
, noisy\
, mvcompensate(noisy,vb1,idx=1,thSCD1=600)\
, mvcompensate(noisy,vb2,idx=1,thSCD1=600))
dfttest(sigma=1.87,bsize=8,osize=6,ssr=0,tsr=2,max2dblocks=5)
denoised=selectevery(5,2) <= dfttest+emc

I tested only luma denoising (chroma from source was merged into the denoised clip before computing metrics). The bold writing above indicates the name I gave to each set of settings for making the graph and listing the results. (emc = external motion compenstation, imc = internal motion compensation... external as in a separate filter and internal as in part of the filter itself).

I optimized the settings for best ssim (or at least tried to), so read what you will into the psnr results.

PSNR:
name: Overall - (Y-min, Y-avg, Y-max)
dfttest+emc: 39.6374 - (36.9429, 37.8764, 38.4442)
dfttest+imc: 39.5612 - (36.8100, 37.8003, 38.2116)
fft3dfilter+emc: 39.5062 - (37.1861, 37.7453, 38.2638)
dfttest: 38.8855 - (36.1973, 37.1246, 37.3587)
fft3dfilter: 38.6796 - (35.8878, 36.9187, 37.2048)
sw-3ddct: 38.6577 - (36.5600, 36.8968, 37.1035)
dctfun4b: 38.0481 - (36.1463, 36.2872, 36.5296)
frfun7: 37.8178 - (35.9513, 36.0569, 36.2783)
frfun3d: 37.7291 - (35.8747, 35.9682, 36.1844)
tnlmeans: 37.2052 - (34.3804, 35.4443, 36.1713)
noisy clip: 36.6127 - (34.8308, 34.8518, 34.8684)

SSIM (frames 0-29):
name: Scaled SSIM Value
dfttest+emc: 91.836
dfttest+imc: 91.166
fft3dfilter+emc: 91.124
sw-3ddct: 89.606
dfttest: 89.263
fft3dfilter: 88.590
tnlmeans: 88.172
frfun7: 86.789
dctfun4b: 86.694
frfun3d: 86.640
noisy clip: 67.518

SSIM (frames 2-27):
name: Scaled SSIM Value
dfttest+emc: 92.052
fft3dfilter+emc: 91.374
dfttest+imc: 91.318
sw-3ddct: 89.766
dfttest: 89.435
fft3dfilter: 88.906
tnlmeans: 88.340
frfun7: 86.835
dctfun4b: 86.731
frfun3d: 86.684
noisy clip: 67.644

Here's a graph of the per-frame SSIM Values. I've scaled the values to the 0-100 range using the same equation that the ssim filter uses when it outputs the overall ssim value (scaled_value = 100*pow(original_value,8)).

http://bengal.missouri.edu/~kes25c/denoise_graph.png

After running the tests I noticed that lumimasking for ssim didn't really change anything on this clip (weights for all frames were > 0.999 with both lumimask=1 and 2)... so I just used the results for lumimask=0.

I'll try to add the execution times for each of the tested setting combinations...

Soulhunter

11th August 2006, 12:25

*cough* (http://soulhunter.chronocrossdev.com/index.html#022)

Didée

11th August 2006, 14:35

To not conflict with the rules, rename the thread to perhaps "what denoiser gives the best metrics" (or "the highest") ...
(Though quite I liked the joky intention of the original wording ;) )

About the noise - pure white gaussian noise surely is a typical means for such a comparison. But it is a highly artifical kind of noise in that sense that in practice, one is almost never dealing with such noise. In most cases, the original noise has been malformed by DCT compression, and therefore has another characteristic. In particular, withh AddGrain(x,0,0) there is no low-frequency noise present at all; it's all only high-frequency noise.

I think that either an additional step of compression after noise-adding should be done, like Mug Funky suggested, or the adding of noise should be slightly more complex, in order to cover a wider frequency range.
(quick idea: make a 2nd layer of noise, process it (only the noise layer) with DCTFilter, and apply that one, too. ?)

Backwoods

11th August 2006, 20:57

...In most cases, the original noise has been malformed by DCT compression, and therefore has another characteristic...

I've been running into this lately and it can make some scenes, in dark areas or flat one color surfaces, look worse degrained/noised than the original grain/noise.

Soulhunter

11th August 2006, 21:04

I've been running into this lately and it can make some scenes, in dark areas or flat one color surfaces, look worse degrained/noised than the original grain/noise.
You talk about... banding? :]

Bye

Backwoods

11th August 2006, 21:33

It's a jumbled mess spinning around. It's not like the banding I've seen on TV or some DVDs, but I guess you could consider it banding. I'll post a sample in a minute.

EDIT: Sample

http://img231.imageshack.us/img231/8749/messft0.png (http://www.flmpro.com/samples/mess.avi)

Right click Save As, 1sec HuffYUV

FRFun7(Lambda=3,T=2,Tuv=1)
aWarpSharp(depth=8,cm=4)
Tweak(bright=3.5,sat=1.185)
LimitedSharpenFaster(Lmode=2,Smode=3,strength=282,overshoot=1,wide=true,ss_x=2,ss_y=2)
FastLineDarken(80,200)

On this source I'm getting different degrees of that mess with different denoisers.

Didée

11th August 2006, 22:13

Fizick

11th August 2006, 22:33

tritical,
thanks for results!
O.K., lets use summary SSIM instead of Y.
Many frames info is important, but the more info, the more confusing.
For example, first 2 and last 1 frame is not full procesed in fft3dfilter for bt=4.
so, average sigma is more informative for trimmed frame range.

I used now for fft:

SSIM(source.trim(2,28),denoised.trim(2,28),"ssim.txt","ssimA.txt",false)

Here is updated per-frame (2 to 28) results for fft3dfilter for noise=25 (with Addgrain, not tritical's avi).

denoised=noisy.FFT3DFilter(sigma=4, bt=4, bw=32, bh=32,
ow=16, oh=16)

SSIM from file (not scaled)
0.982694
0.982774
0.983487
0.982951
0.983048
0.983332
0.983549
0.984140
0.984105
0.983672
0.983977
0.984027
0.984097
0.984429
0.984682
0.984191
0.984057
0.984143
0.984185
0.984713
0.984575
0.984306
0.984170
0.984299
0.984422
0.984390
0.984735

Average SSIM= 87.87 (it is scaled)

by the way, tritical, what version SSIM do you use? my version has lumimasking parameter as bool, not as integer.
i have got if from Lefungus site as SSIM-0.24src.rar, it is the same as at m.f.al site.

Here is results for vagedenoiser v0.35.1.
My best settings (without auxclip):
denoised=noisy.vaguedenoiser(threshold=9, method=3, chromat=0.01, wiener=true, wavelet=3)

0.976896
0.977495
0.977606
0.977716
0.978127
0.978130
0.978188
0.978439
0.978670
0.978279
0.978184
0.978304
0.977856
0.978276
0.978102
0.977974
0.978394
0.978593
0.978511
0.978867
0.978980
0.978637
0.978385
0.978741
0.978427
0.978425
0.978199

Average SSIM= 83.86 (scaled)

BTW, wavelet=3 good results (in wiener mode) was a little surprisingly for me :)
So, the better second stage wavelet, the better result is.

Fizick

11th August 2006, 22:52

May be Foxyshadis can modify AddgrainC (option) to provide same noise for given seed (but different for frames).

Backwoods

11th August 2006, 23:15

No surprise. That's what happens when somewhat stronger grain/noise is processed with just a spatial filter (like here, overpraised frfun.)
Treating such noise with just spatial filtering will kill out only HF noise, and leave you back with lots of LF noise. After encoding *that* with a codec using motion vectors, things will get worse, because ME most likely will find even more motion where should be none. LF noise is big poison for motion search, in particular in "flat" areas where you'll see the effect of erroneous motion vectors most easily.
Told this already several times, seems no-one listens.

You know I follow your posts well enough, but I hope that snide comment wasn't geared towards me. In my folder I am trying alot of different denoisers on this source and it seems they all do it. Even if you're not fond of FRFUN, it still does a good job on most of the anime sources I have tried it on. Just appartently not this one, which doesn't matter b.c others have done the same. I would guess this is from DCT compression. Unfortunately working from the original negatives or even a BETA tape isn't an option here.

Also I think this is getting off subject of the original thread, maybe this can be split or just forgotten about (or even PM). I wouldn't want to derail.

foxyshadis

11th August 2006, 23:32

Oddly enough I've had such a thing kicking around for a while, so I went ahead and uploaded it. seed=<random seed>; constant=true for the behavior seed used to have.

http://foxyshadis.slightlydark.com/random/AddGrainC.zip

I definitely agree that encoding the output with xvid or quenc is the way to go. For simulating dot crawl/rainbowing, try analog capturing a good sequence. (Even as simple as wiring a cable from one computers's output card to another's input.)

Fizick

12th August 2006, 11:42

foxyshadis,
thanks, will try.

Ferux,
We waiting fft3dgpu (different modes ?) SSIM results from you :)

Fizick

12th August 2006, 12:50

Global motion compensation can help a bit for flower clip.
Depan stabolize buildings position.

Script:
depaninterleave(noisy,data=depanestimate(noisy),prev=2,next=1,mirror=15,subpixel=2)
FFT3DFilter(sigma=4, bt=4, bw=32, bh=32, ow=16, oh=16)
denoised=selectevery(4,2)

SSIM:
0.983322
0.983469
0.983932
0.983700
0.983723
0.983960
0.984173
0.984921
0.984999
0.984656
0.984551
0.984761
0.984747
0.984959
0.985445
0.984766
0.984445
0.984882
0.985113
0.985433
0.985287
0.985040
0.984986
0.984930
0.985057
0.984778
0.984672

Average SSIM= 88.34

I used new AddGrainC:
noisy=source.addgrainC(25,0,seed=1)

Didée

12th August 2006, 17:21

Overlapped Blockbased motion compensation can help a bit more for flower clip.

Sandwiching FFT3DFilter like
merge(
\ interleave(bw3,bw2,clp,fw1).fft3dfilter(bt=4,sigma=4,sigma2=4,sigma3=4,sigma4=3,bw=8,bh=8,ow=4,oh=4).selectevery(4,2)
\ ,interleave(fw3,fw2,clp,bw1).fft3dfilter(bt=4,sigma=4,sigma2=4,sigma3=4,sigma4=3,bw=8,bh=8,ow=4,oh=4).selectevery(4,2)
\ ,0.5 )(where 'bwX' and 'fwX' are the compensations)

yielded these numbers:

SSIM:
0.986746
0.986410
0.986820
0.986603
0.986657
0.987276
0.987500
0.987570
0.986869
0.986059
0.985457
0.986690
0.987162
0.987491
0.987480
0.986716
0.985968
0.986855
0.986878
0.987247
0.987532
0.987435
0.987471
0.987146
0.987250

Average SSIM= 90.01119429

Ooops, cracked the 90's barrier. :)

Soulhunter

12th August 2006, 18:38

No surprise. That's what happens when somewhat stronger grain/noise is processed with just a spatial filter (like here, overpraised frfun.)
Treating such noise with just spatial filtering will kill out only HF noise, and leave you back with lots of LF noise. After encoding *that* with a codec using motion vectors, things will get worse, because ME most likely will find even more motion where should be none. LF noise is big poison for motion search, in particular in "flat" areas where you'll see the effect of erroneous motion vectors most easily.
Told this already several times, seems no-one listens.
Overpraised? Who? Where? When? Missed something? But seriously, its no secret that spatial denoisers work only spatial, if you want/need temporal stabilization, use temporal filtering! I often use something like HQDn3D(0,0,2,2) for sources with "pumping noise" before doing the real denoising, helps to calm the noise movement down! For the LF noise: So far I havent seen a filter which is able to remove LF noise [instead of just stabilizing it...] without removing a good part of the real details as well, you know one? ^^

Bye

Fizick

12th August 2006, 20:30

Soulhunter,
we wait optimal seetings for frfun, dctfun for flower from you :).

Soulhunter

12th August 2006, 22:20

Soulhunter,
we wait optimal seetings for frfun, dctfun for flower from you :).

Hey, I just did this comparison with more than 300 metric runs [~150x 2000 frames and ~150x 5000 frames] and Im also busy in RL, so plz gimme a break! ;] But... I have a plugin which helps to find the optimal params without much effort, tell me if ya interested! :]

Thanks n' Byea

Fizick

13th August 2006, 10:05

Lets compare our results with some advanced algorithm from modern scientific researches.
Consider this link: http://www.cs.tut.fi/sgn/lf3d/video/
3D DCT-based video-denoising algorithm, 3D Sliding Window DCT (3D-SWDCT).
It use some 3D sliding dct (8x8) window denoising method with full motion search and compensation (8 frames temporal diameter).
Good, that they provided EXE file for test. The program VD_SW3DDCT.exe can work with raw Y or YUV files. (it need in MSVCR80.DLL)

So, I make AVS file for noisy avi generation:

rawsequence("h:\flower\sflowg.%d.yuv",width=352,height=240,pixel_type="I420").trim(0,29)
coloryuv(levels="PC->TV")
addgrainC(25,0,seed=1)

I open this avs file with VirtualDubMod and save result to sflowg5.avi file with "direct stream copy" method (YV12 format).

To convert this AVI file to RAW YUV file, use avi2raw.exe utility from MPEG4IP tools (binary at http://celticdruid.no-ip.com/xvid/ and

mirrors).
I have got sflowg5.yuv file with 30 noisy frames.

Then I modify some parameters in Config_Y_SIF.txt file to:
FramesToProcess = 30 #Number frames to process
InputFile = "sflowg5.yuv" # Input data file
OutputFile = "sflowg5d.yuv" # Output data file, to be created
SourceColor = 1 # 0 - single channel (e.g. Y), 1- YUV 4:2:0
NoiseParam = 4 #Noise parameters, (e.g Gaussian Noise deviation)

Program runs very slow, about 9 sec per frame. Produced output file is Sflowg5d.yuv in YV12 raw format, we can read it with

RawSource plugin (by Warpenterprises):

denoised=rawsource("h:\flower\sflowg5d.yuv",width=352,height=240,pixel_type="YV12")

I remove first and last 4 frames (temporal radius) from SSIM calculation for most corect results.
SSIM results for frames 4 to 25:
0.982772
0.984977
0.985489
0.985731
0.985872
0.985582
0.985526
0.985523
0.986732
0.986914
0.987020
0.986785
0.986379
0.986544
0.986777
0.986767
0.986900
0.986846
0.986134
0.986324
0.986403
0.986773
0.986797
0.986894
0.987184
0.987104
0.987416
0.987119
0.986451
0.985280

Average SSIM= 89.55

Intersting, that the SSIM is better for used noiseParam=4 ( I got SSIM=89.46 for noiseParam=5).
(It is similar to FFT3Dfilter sigma.)

So, this VD_SW3DDCT denoiser is really good (but very slow). PSNR comparing in article says, that it is best from used in their

tests.
(Some strange, that they used almost the same "Flower" sequence, but shifted by 2 pixels to right,
and with YUV to RGB full range (PC) matrix.)

But as you can see from Didee's post, VD_SW3DDCT denoiser is already not the best in this thread :)

MfA

13th August 2006, 20:30

The flower garden sequence is a bit of a posterchild for global MC based methods.

Fizick

13th August 2006, 21:27

more sequences:
http://www.cipr.rpi.edu/resource/sequences/
http://media.xiph.org/

Fizick

14th August 2006, 20:10

Lets test some commercial denoisers.
One of the most advertised is probably NeatVideo from ABSoft. www. neatvideo.com

Neat Video provides the most accurate video noise reduction currently available...
QUALITY OF NOISE REDUCTION
provided by Neat Video is higher than that of all other methods because:
Neat Video incorporates the most advanced noise reduction algorithms in the industry...
...
Neat Video is not only the highest-quality noise reduction solution, it is the most cost-effective too.
...

Consider NeatVideo plugin for VirtualDub.
Demo version has some restrictions, but fine for our test.
I used latest version 1.5 (and v1.1 too).
To prepare noise profile I create pure grey cip and added noise to it.

blankclip(width=352,height=240,pixel_type="YV12",length=30,fps=25,color_yuv=$7e8080)
addgrainC(25,0,seed=1)

Open the file in VirtualDubMod, add filter NeatVideo, set profile in auto mode (for region about 200x200),
save this profile.
Now I open test noisy clip, and apply this profile.

rawsequence("h:\flower\sflowg.%d.yuv",width=352,height=240,pixel_type="I420").trim(0,29)
coloryuv(levels="PC->TV")
addgrainC(25,0,seed=1)
converttorgb()

Optimal (afrer some test) noise filter setting is:
Noise level Luminance channel =-10,
Noise reduction Luminance channel = 90,
(chrominance=0)
Temporal filter OFF.

I create output AVI file (save as... uncompressed RGB).

read it from avisynth script with:
denoised=avisource("h:\flower\neat90m10.avi").converttoyv12().mergechroma(source)

SSIM results:
0.980320
0.980413
0.980544
0.980890
0.980812
0.981072
0.981131
0.981437
0.981397
0.981490
0.981781
0.981825
0.981492
0.981501
0.981459
0.981363
0.981742
0.981675
0.981499
0.982043
0.982185
0.982230
0.982182
0.982112
0.981947
0.981856
0.982227
0.981715
0.981686
0.982052

Average SSIM= 86.15

I tried use optional temporar filter too, but SSIM results seems do not differ (?) for NeatVideov1.5
and are worse for v1.1. If somebody can correct me, please do it.

tritical

15th August 2006, 01:22

I used the ssim.dll linked to by sagittaire in the first post in this thread (http://forum.doom9.org/showthread.php?t=98633&highlight=ssim+lumimask%3D2). Afaik, it is the same as the one you used except that it has lumimask=2 (lumimask=0/1 are the same as false/true in the previous version), which was a modification by on2tech.

Anyways, I'm gonna update the results I posted earlier to include the new methods that have been posted.

EDIT1: Didée, could you post the lines you used to generate the motion compensated clips in your test? I've updated the rest of the results.

EDIT2: looks like Didée isn't the only one who can break 90 :).

EDIT3: looks like the 91 barrier is cracked as well :).

zambelli

15th August 2006, 02:11

I like the idea of measuring the success of denoisers with SSIM, but I think the setup is slightly flawed - it's only grain that's being added to the source. How about the other two most common noises: analog video noise and MPEG compression artifacts?
Analog video noise could be simulated by playing back the source video through VGA or component output, converting to S-video or composite and feeding it to an analog capture card for uncompressed capture.
MPEG compression noise is easy to generate: simply encode the source to MPEG-2 or MPEG-4 using various degrees of compression.

Fizick

15th August 2006, 06:10

zambelli,
current method is not perfect, bit it is reproducible.
Firstly i wanted use similar DVD-analod things too.
But you can make any comparison you like!

Fizick

17th August 2006, 22:55

tritical,
where is mythological "dfttest"? :)

tritical

17th August 2006, 23:59

It's coming, but I'm still working on it atm. Really, with the settings I used, it is not much different then Didée's fft3dfilter+mc method. The main differences being it uses 75% overlap instead of 50% on the 2d windows (bsize=8,osize=6), and instead of including the same block (in terms of spatial location) from the neighbor frames, it searches within a designated search area for the 2 blocks that best match the current block (in terms of sse distance). So it would be including 3, 8x8 windows in the 3d dft (that's with max2dblocks=3). With ssr=5/tsr=2, which is what I used, that search area is an 11x11 square covering the current frame + 2 previous frames + 2 next frames. Those settings are too slow for normal usage though. I am still working on other improvements.

Mug Funky

18th August 2006, 04:01

zambelli,
current method is not perfect, bit it is reproducible.
Firstly i wanted use similar DVD-analod things too.
But you can make any comparison you like!

this will give grain that's quite close to unprocessed 16mm film, with a slightly blurry transfer:
subtract(last,last.blankclip(color_yuv=$808080).addgrain(100,0,0).blur(.8))

of course you can swap out addgrain with anything else...

as for simulating analog noise, anyone with a DVD-RW and a capture card should be able to do this :) i might upload a sample if anyone's interested (composite only ATM, i don't have an s-video lead long enough).

DSP8000

18th August 2006, 05:39

Hi guys,

very good thread, I've got an idea for some of the coders out there.
There is free download of Imagenomic Noiseware standalone (http://www.imagenomic.com/) & IMHO works much better that NeatImage.
Is it somehow possible to load presets from Noiseware in AVISynth? Maybe similar to loading Photoshop or GIMP curves.

Here are some samples processed with Noiseware

http://img84.imageshack.us/img84/9959/testsu9.png

If someone can find a way how to use this in AVISynth, then we've got freeware solution similar or better to NeatVideo.

DSP8000

Fizick

18th August 2006, 05:47

DSP8000,
Please you make test for noiseware with suggested method.

EDIT:
Seems, it is photo-filter. (we works with video). And free version can save to JPG format only.

Backwoods

18th August 2006, 07:21

You can use the Photoshop plugin to automate original BMP image sequence to filtered BMP sequence. Would be time/HDD consuming . Noiseware is pretty nice though. They give you a decent amount of options.

Terka

18th August 2006, 10:05

look at http://www.michaelalmond.com/Articles/noise_conc.htm

Fizick

18th August 2006, 21:17

Better fft3d results:

denoised=noisy.FFT3DFilter(sigma=4, bt=4, bw=16, bh=16, ow=8, oh=8)

0.984368
0.984504
0.984450
0.984161
0.984429
0.984864
0.985257
0.985813
0.985353
0.985183
0.985257
0.985231
0.985192
0.985215
0.985583
0.985363
0.985316
0.985447
0.985649
0.986141
0.985947
0.985869
0.985750
0.985582
0.985845
0.985953

Average SSIM= 88.83
---------------------------------------------------------
Let's try motion compensation with MVTools 1.4.9.

bl=8
ov=4
sh=2
vf=noisy.mvanalyse(pel=2,blksize=bl,isb=false,idx=1,overlap=ov,sharp=sh,truemotion=true)
vb=noisy.mvanalyse(pel=2,blksize=bl,isb=true,idx=1,overlap=ov,sharp=sh,truemotion=true)
vf2=noisy.mvanalyse(pel=2,blksize=bl,isb=false,idx=1,delta=2,overlap=ov,sharp=sh,truemotion=true)
interleave(\
mvcompensate(noisy,vf2,idx=1,thSCD1=600)\
, mvcompensate(noisy,vf,idx=1,thSCD1=600)\
, noisy\
, mvcompensate(noisy,vb,idx=1,thSCD1=600))
FFT3DFilter(sigma=4, bt=4, bw=16, bh=16, ow=8, oh=8)
denoised=selectevery(4,2)

0.987434
0.987380
0.987763
0.987555
0.987279
0.988028
0.988260
0.988746
0.988331
0.987631
0.987086
0.987274
0.987520
0.988078
0.988639
0.987628
0.987099
0.987317
0.987432
0.988190
0.988408
0.988139
0.987922
0.987709
0.988231
0.988610

Average SSIM= 90.67

shaolin95

19th August 2006, 01:47

OK, so whos "winning" at the moment?

tritical

22nd August 2006, 05:01

Some results on the flower clip with noise std of 10 (var 100):

tested filters:

sw-3ddct(NoiseParam=9) <= sw-3ddct (not an avisynth filter)

dctfun4b(24.0) <= dctfun4b

frfun7(lambda=2.0,T=14.125) <= frfun7

frfun3d(t=13.3) <= frfun3d

fft3dfilter(sigma=7.9, bt=4, bw=16, bh=16, ow=8, oh=8, plane=0) <= fft3dfilter

dfttest(sigma=3.9,bsize=16,osize=12,tsr=1,ssr=0,max2dblocks=3) <= dfttest

vf=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,overlap=4,sharp=2,truemotion=true)
vb=noisy.mvanalyse(pel=2,blksize=8,isb=true,idx=1,overlap=4,sharp=2,truemotion=true)
vf2=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,delta=2,overlap=4,sharp=2,truemotion=true)
interleave(\
mvcompensate(noisy,vf2,idx=1,thSCD1=800)\
, mvcompensate(noisy,vf,idx=1,thSCD1=800)\
, noisy\
, mvcompensate(noisy,vb,idx=1,thSCD1=800))
FFT3DFilter(sigma=7.4, bt=4, bw=8, bh=8, ow=4, oh=4)
denoised=selectevery(4,2) <= fft3dfilter+emc

vf1=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,overlap=4,sharp=2,truemotion=true)
vf2=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,delta=2,overlap=4,sharp=2,truemotion=true)
vb1=noisy.mvanalyse(pel=2,blksize=8,isb=true,idx=1,overlap=4,sharp=2,truemotion=true)
vb2=noisy.mvanalyse(pel=2,blksize=8,isb=true,idx=1,delta=2,overlap=4,sharp=2,truemotion=true)
interleave(\
mvcompensate(noisy,vf2,idx=1,thSCD1=800)\
, mvcompensate(noisy,vf1,idx=1,thSCD1=800)\
, noisy\
, mvcompensate(noisy,vb1,idx=1,thSCD1=800)\
, mvcompensate(noisy,vb2,idx=1,thSCD1=800))
dfttest(sigma=3.6,bsize=8,osize=6,ssr=0,tsr=2,max2dblocks=5)
denoised=selectevery(5,2) <= dfttest+emc

I tested only luma denoising (chroma from source was merged into the denoised clip before computing metrics). The bold writing above indicates the name I gave to each set of settings for making the graph and listing the results. (emc = external motion compenstation, imc = internal motion compensation... external as in a separate filter and internal as in part of the filter itself).

I optimized the settings for best ssim (or at least tried to), so read what you will into the psnr results.

PSNR:
name: Overall - (Y-min, Y-avg, Y-max)
dfttest+emc: 35.2520 - (32.3217, 33.4911, 34.0736)
fft3dfilter+emc: 34.8975 - (32.1083, 33.1366, 33.7318)
sw-3ddct: 34.2174 - (31.9290, 32.4565, 32.7581)
dfttest: 33.9319 - (30.8815, 32.1710, 32.4758)
fft3dfilter: 33.7370 - (30.5839, 31.9761, 32.3819)
dctfun4b: 32.6397 - (30.6978, 30.8788, 31.1775)
frfun7: 32.2570 - (30.3373, 30.4960, 30.7684)
frfun3d: 32.0432 - (30.1331, 30.2823, 30.5516)
noisy clip: 30.2587 - (28.4802, 28.4978, 28.5098)

SSIM (frames 0-29):
name: Scaled SSIM Value
dfttest+emc: 79.876
fft3dfilter+emc: 77.437
sw-3ddct: 74.833
dfttest: 73.102
fft3dfilter: 71.763
frfun7: 65.280
frfun3d: 64.830
dctfun4b: 64.262
noisy clip: 30.906

SSIM (frames 2-27):
name: Scaled SSIM Value
dfttest+emc: 80.837
fft3dfilter+emc: 78.402
sw-3ddct: 75.162
dfttest: 73.580
fft3dfilter: 72.700
frfun7: 65.345
frfun3d: 64.885
dctfun4b: 64.319
noisy clip: 31.098

Here's a graph of the per-frame SSIM Values.

http://bengal.missouri.edu/~kes25c/denoise_graph2.png

And for those who prefer pictures to numbers, here's a pic of frame 21 (http://bengal.missouri.edu/~kes25c/frame21.png) showing the original source, the noisy source, and denoised versions.

foxyshadis

22nd August 2006, 05:59

Is dfttest so far only optimized for uniform gaussian noise, or does it work just as well if you toss your noisy subject video into an encoder at some middling bitrate first?

tritical

22nd August 2006, 07:17

I've only tested on gaussian noise so far, but I would expect it to perform similarly to fft3dfilter on other types of noise. I am definitely going to start testing on some other sources soon. Would it be more desirable to test on a noisy source generated by:

1.) sequence -> add gaussian noise -> mpeg2

2.) sequence -> mpeg2 (dvd) -> dvd player -> huffyuv capture (composite)

3.) sequence -> add gaussian noise -> mpeg2 (dvd) -> dvd player -> huffyuv capture (composite)

? I am also thinking about switching to a clip that isn't so perfect for block based motion compensation.

Richard Berg

22nd August 2006, 08:06

I think #2 would be most interesting.

I have a large collection of old (read: noisy) capture cards but no DVD player :( Best I could contribute would be the same program captured from both a good and bad card.

Mug Funky

22nd August 2006, 08:48

@ tritical: try old footage from archive.org.

the kind with strong luminance flicker... this'll make block-based motion compensation retch.

frednerk33

22nd August 2006, 14:04

How about Pookie's http://forum.doom9.org/showthread.php?p=865906#post865906 ?
backward_vec2 = last.MVAnalyse( isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1 )
backward_vec1 = last.MVAnalyse( isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1 )
forward_vec1 = last.MVAnalyse( isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1 )
forward_vec2 = last.MVAnalyse( isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1 )
a = last.MVDegrain2( backward_vec1, forward_vec1, backward_vec2, forward_vec2, thSAD=400, idx=1 )
b = a.DeGrainMedian( mode=1 )
SeeSaw( a, b, NRlimit=4, NRlimit2=5, Sstr=1.5, Slimit=9, Spower=9, Szp=16, sootheT=80, bias=40 )

Referred to in a visual test at http://forum.doom9.org/showthread.php?p=865972#post865972

Pretty results from noisy input: http://forum.doom9.org/showthread.php?p=865872#post865872

Didée

22nd August 2006, 14:17

"Visually pleasing" and "good metrics" are two different pairs of shoes.

I tried FFT3D+motioncompensation (top-notch combo in this thread) on a grainy DVD source, and the result looked kind of boring ("emptied-out"), while not being able to truly kill low-frequency flicker. OTOH, a custom-made MC'ed temporal averaging looked sharp & detailed, and highly stable: no more LF flicker ... though the same script gives a 2 points lower SSIM (!) on the source of this thread, heh.

Me, I dont care for high SSIMs or PSNRs, if the result is blurry and keeps flickering ...

tritical

23rd August 2006, 01:07

You're saying that your method looked better on the current test clip then the fft3d+mc method even though it scored 2 points lower on ssim? ... or just that it looked better on the grainy dvd source?

It's not surprising to me that fft3d+mc (and the like) has so far dominated this comparison (ssim/psnr wise) based on removing zero mean gaussian noise from a clip with pretty consistent global motion. It is also not surprising to me that it would not win in other noise removal tests (either different scoring system or different types of noise).

Start rant about metrics:

The thing to remember about ssim/psnr is that they are 'image' quality assessments, so for video they compare single frames and then pretty much just average (in the case of overall psnr it's slightly different) the results of all frames. In the original ssim paper there was a proposal for simple motion weighting (give less weight to frames in areas of large motion), but that doesn't come close to handling a lot of cases. For example, consider two 6 frame sequences with ssim scores of:

80.005 80.005 80.005 80.005 80.005 80.005
(.9725 .9725 .9725 .9725 .9725 .9725)

86.611 72.139 86.611 72.139 86.611 72.139
(.9850 .9600 .9850. 9600 .9825 .9600)

both would get an overall score of: 80.005 even though the first sequence would in all likelihood be ranked higher from a subjective quality perspective. ssim/psnr will also not account for things such as flickering/moving blocks in flat areas between frames, etc...

Even on the single image level, PSNR has been shown to not correspond well with subjective human tests as far as image quality is concerned. SSIM has been shown to be better, but is still not perfect. There are newer versions of ssim (cwssim, wcwssim) that come closer to human rankings of quality for certain types of artifacts (one is low frequency structural noise, such as block mean shift).

Another problem with using these types of metrics is that it assumes that the original source is the optimum. For example, assume you have a source that is noise free, you then add some noise to it, afterwhich you denoise the image using two different filters. One filter perfectly removes the noise and does nothing else. The other removes the noise, but also slightly sharpens, increases saturation, performs aa, or whatever... From a metric perspective the first denoiser is the best, but a person might prefer the look of the second denoiser (due to the fact that they would prefer a slightly modified version of the original vs the original itself).

end rant:

Anyways, I am sure almost everyone here already knows the limitations/issues of using psnr/ssim/etc... metrics. However, it is still interesting to perform these types of tests, if only to see how poorly the metrics compare to human perception (to me at least).

Soulhunter

24th August 2006, 13:40

Ok, let's compare temporal filters with temporal filters... ^^

Using tritical's settings for motion compensation on the flower sequence:

Noisy=Source.AddGrain(25,0,0)

Measured on frames 0-30

Denoised=MC.FFT3DFilter(Sigma=3.7,bt=4,bw=8,bh=8,ow=4,oh=4)
> Average SSIM = 91.19802121

Denoised=MC.DCTFun7(Ty=2.9)
> Average SSIM = 92.42250950
> Average SSIM = 92.68476356 [on frames 2-27]

The resul tritical published for DFTTest+EMC
> Average SSIM = 92.052

Noisy=Source.AddGrain(100,0,0)

Measured on frames 2-27

Denoised=MC.FFT3DFilter(Sigma=7.4,bt=4,bw=8,bh=8,ow=4,oh=4)
> Average SSIM = 78.28281343

Denoised=MC.DCTFun7(Ty=2.9*2)
> Average SSIM = 81.90503727

The result tritical published for DFTTest+EMC
> Average SSIM = 80.837

In short...

Some noise :

92.422 for DCTFun7
92.052 for DFTTest+EMC
91.198 for FFT3DFilter

Some more noise :

81.905 for DCTFun7
80.837 for DFTTest+EMC
78.282 for FFT3DFilter

Conclusion: MVTools wins! :]

Trivia about DCTFun7 [is a little bit less XD]

It's an hard thresholded TI 4x4x4 ICT using a multiplierless and branchless core!

Backwoods

25th August 2006, 08:52

Do you have a download link for DCTFun7?

Fizick

25th August 2006, 21:17

http://forum.doom9.org/showthread.php?p=864535#post864535

Soulhunter

26th August 2006, 01:18

It's coming...
Same goes for DCTFun7! ^^

Bye

frednerk33

26th August 2006, 08:31

Is it wrong to post a link to http://forum.doom9.org/showthread.php?p=868048#post868048 which has visual results but not objective per above ? (I am not smart enough to do the above, but someone else here might be.)

DryFire

26th August 2006, 08:37

What if you all used the lossless version of elphant's dream as a starting source? I'd suggest the 640x360 one, unless you just feel like playing with 20gb's worth of hd png's (the is the same source from saggatire's hd comparison test):
http://orange.blender.org/blog/original-lossless-source-available/

Or some other, clean predefined clip.

Fizick

8th September 2006, 20:32

O.K., now I have secret weapon (bt=5) too ;)

interleave(\
mvcompensate(noisy,vf2,idx=1,thSCD1=600)\
, mvcompensate(noisy,vf,idx=1,thSCD1=600)\
, noisy\
, mvcompensate(noisy,vb,idx=1,thSCD1=600)\
, mvcompensate(noisy,vb2,idx=1,thSCD1=600))
FFT3Dfilter(sigma=3.6, bt=5, bw=8, bh=8, ow=4, oh=4)
denoised=selectevery(5,2)

Average SSIM= 91.75

But fft3d is still the worst of secret filters.

cwk

8th September 2006, 23:21

O.K., now I have secret weapon (bt=5) too

Should we be expecting FFT3dFilter v 1.8.6??????

Didée

9th September 2006, 00:16

O.K., now I have secret weapon (bt=5) too ;)
Pfffft.:) There was already bt=7 (http://forum.doom9.org/showthread.php?p=862342#post862342) in use.

Well okay, [(bt=4)+(bt=4)]*0.5 is not exactly bt=7 ... just what a poor scripter can do. ;)

frednerk

9th September 2006, 03:23

O.K., now I have secret weapon (bt=5) too ;)

Er, bt=5 gives me an error message saying values must be -1,0,1,2,3,4 ... am using FFT3Dfilter Version 1.8.5 - 4 December 2005.

Am using
ConvertToYV12(interlaced=FALSE)
#http://forum.doom9.org/showthread.php?p=873307#post873307
#O.K., now I have secret weapon (bt=5) too
noisy = LAST
vf1=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,overlap=4,sharp=2,truemotion=true)
vf2=noisy.mvanalyse(pel=2,blksize=8,isb=false,idx=1,delta=2,overlap=4,sharp=2,truemotion=true)
vb1=noisy.mvanalyse(pel=2,blksize=8,isb=true,idx=1,overlap=4,sharp=2,truemotion=true)
vb2=noisy.mvanalyse(pel=2,blksize=8,isb=true,idx=1,delta=2,overlap=4,sharp=2,truemotion=true)
interleave(\
mvcompensate(noisy,vf2,idx=1,thSCD1=600)\
, mvcompensate(noisy,vf1,idx=1,thSCD1=600)\
, noisy\
, mvcompensate(noisy,vb1,idx=1,thSCD1=600)\
, mvcompensate(noisy,vb2,idx=1,thSCD1=600))
FFT3Dfilter(sigma=3.6, plane=4, bt=5, bw=8, bh=8, ow=4, oh=4)
selectevery(5,2)

Can you share the secret here ? I won't pass it on. :)

Fizick

9th September 2006, 08:44

v.1.9.2 is coming somedays ...

EDIT:
Released for public.

CruNcher

31st August 2007, 11:29

almost 1 year has passed now (1 year in dev time is like a century) so don't you think it's time again to take out the newest denoisers and show some results again ;)
*bump*

Soulhunter

31st August 2007, 16:39

Wow, almost 1 year already? Unfortunately there are no new denoisers from pruned... Afaik he mainly worked on his own programming language [and floods us with code nobody except him understands, lol] and some GUI stuff... Well, actually I think he spend most of his free time for watching anime, playing games n being lazy! :D

Bye

Terka

28th November 2007, 13:53

is there a link to DCTFun7?

tritical

12th April 2009, 00:40

Gonna dig up this old thread :). I wanted to do some tests using dfttest v1.5 to see how it compares.

Script:

noisy = source.addgrain(25,0,0)

sclip = noisy.msuper()
vf1=manalyse(sclip,isb=false,delta=1,overlap=4)
vf2=manalyse(sclip,isb=false,delta=2,overlap=4)
vb1=manalyse(sclip,isb=true,delta=1,overlap=4)
vb2=manalyse(sclip,isb=true,delta=2,overlap=4)

mcomp5 = interleave(\
mcompensate(noisy,sclip,vf2,thSCD1=600)\
, mcompensate(noisy,sclip,vf1,thSCD1=600)\
, noisy\
, mcompensate(noisy,sclip,vb1,thSCD1=600)\
, mcompensate(noisy,sclip,vb2,thSCD1=600))

denoised = mcomp5.dfttest_dfttest(tbsize=5,ftype=1,sbsize=8,sosize=7,sigma=110).selectevery(5,2).mergechroma(source)

c1 = SSIM(source.trim(2,27),denoised.trim(2,27),"ssim_per_frame_results.txt","ssim_overall_result.txt",lumimask=0).crop(0,0,16,16)
c2 = compareYV12(denoised.trim(2,27),source.trim(2,27),"YUV","psnr_results.txt").crop(0,0,16,16)
c3 = SSIM(source,denoised,"ssim_per_frame_results_f.txt","ssim_overall_result_f.txt",lumimask=0).crop(0,0,16,16)
c4 = compareYV12(denoised,source,"YUV","psnr_results_f.txt").crop(0,0,16,16)
stackhorizontal(c1,c2,c3,c4)

Results on flower with noise variance of 25.

SSIM PSNR: Y-min,Y-avg,Y-max
Frames [2-27]: 92.68554702 (37.8624,38.3406,38.8413)
Frames [0-29]: 92.39688068 (37.4715,38.2938,38.8413)

Results with variance of 25 but sosize=6 instead of 7 in dfttest (3-4 times faster):

SSIM PSNR: Y-min,Y-avg,Y-max
Frames [2-27]: 92.53761670 (37.7406,38.2574,38.7435)
Frames [0-29]: 92.23917376 (37.3694,38.2109,38.7435)

Results on flower with noise variance of 100.

SSIM PSNR: Y-min,Y-avg,Y-max
Frames [2-27]: 82.54926547 (33.1546,33.9058,34.4971)
Frames [0-29]: 81.47632380 (32.4489,33.7999,34.4971)

Those results really need to be averaged over multiple trials, but I didn't take the time. The variance of the ssim/psnr numbers from trial to trial is definitely much higher for noise variance 100 than 25.

MadRat

15th April 2009, 03:41

Suburb post tritical, it brings up questions for me: If I were to use dfttest+emc and fft3dfilter+emc as actual denoisers what would the script code be? How does the above dfttest+emc with the latest version of dfttest and optimized settings compare to fft3dfilter+emc with the latest version of fft3dfilter and optimized settings? How do the results from artificial noise compare to real world noise say from broadcast video? Does dfttest+emc and fft3dfilter+emc work equally as well on live action as they do for animation?

tritical

15th April 2009, 05:37

1.) The code would be the same as that in my previous post. Just replace the 'noisy' clip with your source, and dfttest with fft3dfilter.

What that script is doing is using mvtools to move neighboring frames forward or backward in time so that they line up with the current frame. With tbsize=5 and tmode=0 in dfttest, or bt=5 in fft3dfilter, you are centering a temporal window of length 5 on each frame. Thus, the window includes the two previous frames and the two next frames. To get the best result you want the previous and next frames to be aligned in time with the center frame. To do that, the script spaces the original frames out: [0,1,2,...] -> [2,7,12,...] and then fills in the two frames to either side with the motion compensated versions of the previous and next frames. It then calls dfttest on this padded sequence, and then the selectevery(5,2) call at the end picks out the original frames.

2.) The main differences (those contributing most to the metric difference) are that dfttest is using hard thresholding while fft3dfilter uses wiener filter (with spectral subtraction as signal estimate), and that dfttest is using 75+% overlap while fft3dfilter is only using 50% (which is the max that it supports).

3.) It is hard to say since broadcast video has many possible types of noise. What I can say is dfttest operates based on four assumptions:

a.) The noise is zero mean.
b.) The noise is additive (y(n) = s(n) + d(n))... so in the frequency domain the power spectrum of the noisy signal y(n) is equal to the sum of the power spectrum of the signal s(n) and the power spectrum of the noise d(n).
c.) The signal and noise are uncorrelated.
d.) The noise is stationary (power spectrum doesn't change based on frame number or x/y position in a frame).

Additive zero mean gaussian white noise (as added by addgrain(x,0,0)) meets all of these assumptions. Also, when you specify a single sigma value to dfttest(), you are basically telling it that the noise power spectrum is flat and equal to sigma everywhere. A flat power spectrum equal to sigma is the power spectrum of white noise with power=sigma. So removing the noise added by addgrain(x,0,0) is the best case scenario for dfttest.

dfttest is able to remove noise that doesn't have a flat spectrum by either using the nfile parameter to have it estimate the noise spectrum or by using sfile to enter it yourself. How well removing the noise will work depends on how closely it meets the four assumptions above. Of those four assumptions, c and d are the ones that real world noise usually doesn't conform to. Most of the time, it is correlated with the signal at least a little, and it is not stationary.

4.) It is more a matter of the type of noise than the type of video. I guess you could ask on which type does the filtering look better when some of the assumptions about signal/noise are not met, but I really don't know as I haven't tested it much on non anime.