Log in

View Full Version : Objective (as can be) Denoiser/Degrainer Comparisons

Nightshiver
21st September 2011, 23:01
Since there have been many comments, most recently in the "filter renewal" thread about the pro's and con's of each of the many denoiser/degrainer options out there today, I thought I'd try to make an objective test of the different filters and some of their options for all to see.

What I have done is taken some samples and used them (actually, just frames) and tested the many methods available today. If anyone would like certain options used, please post them here or if anyone would like a certain sourced to be used, please say so or, even better, supply the source yourself and I will do the testing on it.

Down below, you will see the many (42 pictures per sample) pic's and also the associated syntax used.

The filters used:

vaguedenoiser
ttempsmooth
tnlmeans
removegrain
mvdegrain
mdegrain
mctemporaldenoise
mc_spuds
fft3dfilter
dfttest
degrainmedian




Original: http://img546.imageshack.us/img546/374/originalo.png



http://img692.imageshack.us/img692/9504/degrainmedianmode0.png

degrainmedian(mode=0)

http://img21.imageshack.us/img21/6932/degrainmedianmode1.png

degrainmedian(mode=1)

http://img600.imageshack.us/img600/7113/degrainmedianmode2.png

degrainmedian(mode=2)

http://img4.imageshack.us/img4/1993/degrainmedianmode3.png

degrainmedian(mode=3)

http://img707.imageshack.us/img707/9264/degrainmedianmode4.png

degrainmedian(mode=4)

http://img8.imageshack.us/img8/4897/degrainmedianspecial.png

DeGrainMedian(limitY=5,limitUV=7,mode=0)

http://img41.imageshack.us/img41/4701/dfttest.png

dfttest()

http://img269.imageshack.us/img269/1039/dfttestsigma05.png

dfttest(sigma=0.5)

http://img713.imageshack.us/img713/8741/dfttestsigma10.png

dfttest(sigma=1.0)

http://img842.imageshack.us/img842/8983/dfttestsigma30.png

dfttest(sigma=3.0)

http://img195.imageshack.us/img195/4361/fft3dpic1.png

strength = 6
fft3dfilter(bw=6, bh=6, ow=3, oh=3, plane=0, bt=1, sigma=strength)
fft3dfilter(bw=216, bh=216, ow=108, oh=108, plane=0, bt=1, sigma=strength/8, sigma2=strength/4, sigma3=strength/2, sigma4=strength)

http://img843.imageshack.us/img843/5052/fft3dpic2.png

fft3dfilter()

http://img695.imageshack.us/img695/4894/fft3dpic3.png

fft3dfilter(bw=6, bh=6, ow=3, oh=3, plane=0, bt=1, sigma=6, sigma2=4, sigma3=2, sigma4=1)

http://img838.imageshack.us/img838/6892/fft3dpic4.png

fft3dfilter(plane=0, bt=1, sigma=6, sigma2=4, sigma3=2, sigma4=1)

http://img14.imageshack.us/img14/3663/mcspuds1.png

MC_Spuds(frames=2,strength=1,postprocess=true)

http://img853.imageshack.us/img853/3191/mcspuds2.png

MC_Spuds(frames=2,strength=3,postprocess=true)

http://img10.imageshack.us/img10/3719/mcspuds3.png

MC_Spuds(frames=2,strength=6,postprocess=true)

http://img11.imageshack.us/img11/2650/mcspuds4.png

MC_Spuds(frames=3,strength=3)

http://img202.imageshack.us/img202/929/mctemporaldenoiseverylo.png

MCTemporalDenoise(settings="very low")

http://img98.imageshack.us/img98/1345/mctemporaldenoiselow.png

MCTemporalDenoise(settings="low")

http://img689.imageshack.us/img689/9822/mctemporaldenoisemedium.png

MCTemporalDenoise(settings="medium")

http://img98.imageshack.us/img98/5886/mctemporaldenoisehigh.png

MCTemporalDenoise(settings="high")

http://img689.imageshack.us/img689/4739/mctemporaldenoiseveryhi.png

MCTemporalDenoise(settings="very high")

http://img69.imageshack.us/img69/2532/mdegrain2sad400.png

super = MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)

http://img850.imageshack.us/img850/9130/mdegrain3sad400.png

super = MSuper(pel=2, sharp=1)
backward_vec3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
forward_vec3 = MAnalyse(super, isb = false, delta = 3, overlap=4)
MDegrain3(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400)


http://img828.imageshack.us/img828/6307/mvdegrainmultiframes2sa.png

source = last
vectors = source.MVAnalyseMulti(refframes=2, pel=2, overlap=4, sharp=1, idx=1)
source.MVDegrainMulti(vectors,thSAD=400,idx=1)

http://img18.imageshack.us/img18/980/mvdegrainmultiframes3sa.png

source = last
vectors = source.MVAnalyseMulti(refframes=3, pel=2, overlap=4, sharp=1, idx=1)
source.MVDegrainMulti(vectors,thSAD=400,idx=1)

http://img638.imageshack.us/img638/9638/removegrain1.png

removegrain(1)

http://img198.imageshack.us/img198/3329/removegrain5.png

removegrain(5)

http://img26.imageshack.us/img26/9924/removegrain10.png

removegrain(10)

http://img833.imageshack.us/img833/6006/removegrain19.png

removegrain(19)

http://img189.imageshack.us/img189/8973/tnlmeans1.png

tnlmeans(h=2.2,ax=10,ay=10,bx=0,by=0,sx=3,sy=3,a=1.0)

http://img9.imageshack.us/img9/3018/tnlmeans2.png

tnlmeans(h=2.2,ax=10,ay=10,bx=0,by=0,sx=3,sy=3,a=100)

http://img716.imageshack.us/img716/7641/tnlmeans3.png

tnlmeans()

http://img31.imageshack.us/img31/2927/ttempsmooth1.png

ttempsmooth(maxr=7,strength=5,lthresh=3,cthresh=4)

http://img692.imageshack.us/img692/7099/ttempsmooth2.png

ttempsmooth()

http://img854.imageshack.us/img854/1825/ttempsmooth3.png

ttempsmooth(maxr=7,strength=5,lthresh=50,cthresh=50)

http://img820.imageshack.us/img820/1337/ttempsmooth4.png

ttempsmooth(maxr=7,strength=5,lthresh=20,cthresh=20)

http://img849.imageshack.us/img849/7944/vaguedenoiser1.png

vaguedenoiser(nsteps=6, wiener=true, wratio=1.0, chromaT=1, threshold=1.5)

http://img268.imageshack.us/img268/1498/vaguedenoiser2.png

VagueDenoiser(threshold=0, method=3, nsteps=5, chromaT=0)

http://img52.imageshack.us/img52/8766/vaguedenoiser3.png

VagueDenoiser(threshold=8, method=3, nsteps=5, chromaT=0)


Whew, that's the first sample. Here's the second.
(These pictures are to large to have loaded here)

Original: http://img402.imageshack.us/img402/1671/originalcg.png



http://img32.imageshack.us/img32/9504/degrainmedianmode0.png

degrainmedian(mode=0)

http://img831.imageshack.us/img831/6932/degrainmedianmode1.png

degrainmedian(mode=1)

http://img809.imageshack.us/img809/7113/degrainmedianmode2.png

degrainmedian(mode=2)

http://img195.imageshack.us/img195/1993/degrainmedianmode3.png

degrainmedian(mode=3)

http://img641.imageshack.us/img641/9264/degrainmedianmode4.png

degrainmedian(mode=4)

http://img827.imageshack.us/img827/4897/degrainmedianspecial.png

DeGrainMedian(limitY=5,limitUV=7,mode=0)

http://img831.imageshack.us/img831/4701/dfttest.png

dfttest()

http://img269.imageshack.us/img269/7792/dfttestsigma1.png

dfttest(sigma=1.0)

http://img822.imageshack.us/img822/364/dfttestsigma3.png

dfttest(sigma=3.0)

http://img217.imageshack.us/img217/1039/dfttestsigma05.png

dfttest(sigma=0.5)

http://img846.imageshack.us/img846/5467/dfttestsigma025.png

dfttest(sigma=0.25)

http://img3.imageshack.us/img3/4361/fft3dpic1.png

strength = 6
fft3dfilter(bw=6, bh=6, ow=3, oh=3, plane=0, bt=1, sigma=strength)
fft3dfilter(bw=216, bh=216, ow=108, oh=108, plane=0, bt=1, sigma=strength/8, sigma2=strength/4, sigma3=strength/2, sigma4=strength)

http://img263.imageshack.us/img263/5052/fft3dpic2.png

fft3dfilter()

http://img706.imageshack.us/img706/4894/fft3dpic3.png

fft3dfilter(bw=6, bh=6, ow=3, oh=3, plane=0, bt=1, sigma=6, sigma2=4, sigma3=2, sigma4=1)

http://img834.imageshack.us/img834/6892/fft3dpic4.png

fft3dfilter(plane=0, bt=1, sigma=6, sigma2=4, sigma3=2, sigma4=1)

http://img20.imageshack.us/img20/3663/mcspuds1.png

MC_Spuds(frames=2,strength=1,postprocess=true)

http://img821.imageshack.us/img821/3191/mcspuds2.png

MC_Spuds(frames=2,strength=3,postprocess=true)

http://img705.imageshack.us/img705/3719/mcspuds3.png

MC_Spuds(frames=2,strength=6,postprocess=true)

http://img841.imageshack.us/img841/2650/mcspuds4.png

MC_Spuds(frames=3,strength=3)

http://img838.imageshack.us/img838/929/mctemporaldenoiseverylo.png

MCTemporalDenoise(settings="very low")

http://img707.imageshack.us/img707/1345/mctemporaldenoiselow.png

MCTemporalDenoise(settings="low")

http://img825.imageshack.us/img825/9822/mctemporaldenoisemedium.png

MCTemporalDenoise(settings="medium")

http://img13.imageshack.us/img13/5886/mctemporaldenoisehigh.png

MCTemporalDenoise(settings="high")

http://img844.imageshack.us/img844/4739/mctemporaldenoiseveryhi.png

MCTemporalDenoise(settings="very high")

http://img155.imageshack.us/img155/2532/mdegrain2sad400.png

super = MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)

http://img849.imageshack.us/img849/9130/mdegrain3sad400.png

super = MSuper(pel=2, sharp=1)
backward_vec3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
forward_vec3 = MAnalyse(super, isb = false, delta = 3, overlap=4)
MDegrain3(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400)

http://img15.imageshack.us/img15/980/mvdegrainmultiframes3sa.png

source = last
vectors = source.MVAnalyseMulti(refframes=3, pel=2, overlap=4, sharp=1, idx=1)
source.MVDegrainMulti(vectors,thSAD=400,idx=1)

http://img593.imageshack.us/img593/9638/removegrain1.png

removegrain(1)

http://img855.imageshack.us/img855/3329/removegrain5.png

removegrain(5)

http://img84.imageshack.us/img84/9924/removegrain10.png

removegrain(10)

http://img38.imageshack.us/img38/6006/removegrain19.png

removegrain(19)

http://img215.imageshack.us/img215/8973/tnlmeans1.png

tnlmeans(h=2.2,ax=10,ay=10,bx=0,by=0,sx=3,sy=3,a=1.0)

http://img12.imageshack.us/img12/3018/tnlmeans2.png

tnlmeans(h=2.2,ax=10,ay=10,bx=0,by=0,sx=3,sy=3,a=100)

http://img202.imageshack.us/img202/7641/tnlmeans3.png

tnlmeans()

http://img710.imageshack.us/img710/2927/ttempsmooth1.png

ttempsmooth(maxr=7,strength=5,lthresh=3,cthresh=4)

http://img821.imageshack.us/img821/7099/ttempsmooth2.png

ttempsmooth()

http://img405.imageshack.us/img405/1825/ttempsmooth3.png

ttempsmooth(maxr=7,strength=5,lthresh=50,cthresh=50)

http://img5.imageshack.us/img5/1337/ttempsmooth4.png

ttempsmooth(maxr=7,strength=5,lthresh=20,cthresh=20)

http://img835.imageshack.us/img835/7944/vaguedenoiser1.png

vaguedenoiser(nsteps=6, wiener=true, wratio=1.0, chromaT=1, threshold=1.5)

http://img508.imageshack.us/img508/1498/vaguedenoiser2.png

VagueDenoiser(threshold=0, method=3, nsteps=5, chromaT=0)

http://img718.imageshack.us/img718/8766/vaguedenoiser3.png

VagueDenoiser(threshold=8, method=3, nsteps=5, chromaT=0)


And there you go. Comments please.
DeathAngelBR
22nd September 2011, 04:31
So basicly: mctemporaldenoise wins hands down, tnlmeans in 2nd place, dfttest slightly better than fft3d.

I still prefer ttempsmooth() followed by fft3d or dfttest though.
Mounir
22nd September 2011, 09:29
tnlmeans() appear to retain more fine details compared with mctemporaldenoise (medium) or fft3d (pic4) But i haven't compared with the whole images series .
I think there is a new tool to study signal/noise ratio which is 'better' than PSNR perhaps somoneone who has it can put these images to test and see what comes out ?
Didée
22nd September 2011, 10:01
Rating denoiser-degraining upon still pictures is like rating a song upon a single tone or chord. You might like the chord, but you don't know the melody. :)
SSH4
22nd September 2011, 10:28
I add another 5 cents. Comparing "one button" denoisers with powerfull mdeg1/2/3 and dfttest or dfttest with motion compensation, sstring/ssx/ssy/sst or nfile and 16bit mod is really useless.
Gser
22nd September 2011, 11:51
Using HD material would have been better.
Nightshiver
22nd September 2011, 11:51
Rating denoiser-degraining upon still pictures is like rating a song upon a single tone or chord. You might like the chord, but you don't know the melody. :)

Going along with this, I can make separate encodes of each of the samples if you'd like. Mind, that will take some time, but it would be a valuable addition. I think I'll do that.

To gser: Not neccessarily. For the 300 pictures, I did used HD material, but as we all know, that grain in 300 was added artificially. But, seeing as how I do not have a pile of cash laying around, I can't really get my hands on a whole lot of HD material. Besides, most (note that I did not say all) HD material these days try to clean every spot of grain out, or at least quite a bit. I actually like using SD material because it shows the differences a little more clearly to me.


However, as I stated at the beginning of my post, if there are specific samples you would like, please provide them or at least tell me about them so I can attempt to get them.
SSH4
22nd September 2011, 12:16
It's thankless task compare tools with so many options.

dfttest() != dfttest(sigma=2) != dfttest(f0beta=0.5,U=false,V=false,nfile="nfile.txt") != dfttest(sstring = "0.0:0.0 0.15:10.0 0.85:10.0 1.0:0.0") != dfttest(ssx="0.0:2.0 1.0:8.0",ssy="0.0:2.0 1.0:8.0",sst="0.0:4.0 1.0:10.0",lsb=true).ditherpost(mode=7,ampn=2,staticnoise=true)

It's all one dfttest.
So what you want compare?
aegisofrime
22nd September 2011, 12:37
I have seen members using an image hosting site that seems specially suited for image comparisons; You basically mouseover the image to see the difference. Perhaps you could upload to that site? I can't for the life of me remember what it's called though, perhaps someone else can tell us. :D
SSH4
22nd September 2011, 12:53
screenshotcomparison.com
jmac698
22nd September 2011, 16:32
First you study the settings of each tool, determine the best options, then compare the tools.

About the many options, some of those don't directly affect denoising but are affecting artefacts (like overlap, blocksize). You can still compare denoising and assume some setting can be found to minimize artefacts.

I think MCtemporaldenoise looks awesome. I think my theory is correct, a script which is tuned by a person is going to have some value, so a recommendation is a way to narrow your choices.
jmac698
22nd September 2011, 16:47
http://screenshotcomparison.com/comparison/82133/picture:0
Degrain median 5 modes and special, in order

http://screenshotcomparison.com/comparison/82134/picture:0
Dfttest(sigma=default,0.5,1.0,3.0)

http://screenshotcomparison.com/comparison/82137/picture:0
fft3dfilter(?)

http://screenshotcomparison.com/comparison/82138
MC_Spuds(?)

http://screenshotcomparison.com/comparison/82140
MCTemporalDenoise(settings="very low","low","medium","high","very high")

ok I'm bored now, could someone else finish this?
Some of these don't seem to be changing but I swear I uploaded the correct files...

http://avisynth.org/mediawiki/Denoisers Updated.
SilaSurfer
22nd September 2011, 16:48
Rating denoiser-degraining upon still pictures is like rating a song upon a single tone or chord. You might like the chord, but you don't know the melody. :)

I agree. With single screenshots you can see only lower amount of grain but can't rate if the output is calm enough (presence of temporal fluctuations).
jmac698
22nd September 2011, 18:45
Results from msu noise estimation. It seems higher values are less noise. They don't always make sense.

MSU Noise(1) Ver. 0.3 (Y converted from RGB-not accurate)
Settings Noise
original 0.3303
degrain m0 0.6078
degrain m1 0.5217
degrain m2 0.3503
degrain m3 0.0874
degrain m4 0.1342
degrain sp 0.6134
dfttest 0.9850
dft s=0.5 0.2315
dft s=1.0 0.1218
dft s=3.0 0.2163
mcspuds4 0.6051
fft3d4 0.6236
mclow 0.5669
mcmed 0.9481
mchigh 1.4498
mcveryhigh 1.5565
rmgrain19 0.6205
tnlmeans3 1.0159
ttemp4 0.8834
vague3 1.0312


I think there's a way we could come to a conclusion; if we rate by best denoising with no atefacts and preserving detail, I think that's what we all want for light denosing.
The way to do this is using a film grain simulator to add noise to a low-noise orignal (like some RED raw footage), also to test images, we can get real noise reduction measurement and quality metrics for other atefacts. Could could sort thumbnails by denoising or blurring or PSNR, SSIM, etc. then you just pick the thumbnail you want, the settings are shown.
For film grain, it shouldn't matter the type of source, except that some sources mask artefacts maybe.


Blurring Metric
original 11.7414
degrainm0 10.08
mcmed 11.1169
mcveryhigh 9.8426

Here we can see mctempmed is definitely superior to degrain, it has less noise and less blur.

low number means more blur
Nightshiver
22nd September 2011, 21:22
It's thankless task compare tools with so many options.

dfttest() != dfttest(sigma=2) != dfttest(f0beta=0.5,U=false,V=false,nfile="nfile.txt") != dfttest(sstring = "0.0:0.0 0.15:10.0 0.85:10.0 1.0:0.0") != dfttest(ssx="0.0:2.0 1.0:8.0",ssy="0.0:2.0 1.0:8.0",sst="0.0:4.0 1.0:10.0",lsb=true).ditherpost(mode=7,ampn=2,staticnoise=true)

It's all one dfttest.
So what you want compare?

It's not the same thing.... you quite honestly don't understand what I am attempting to do.

I am currently encoding the samples so that we may have moving evidence.
chainring
22nd September 2011, 21:49
ok I'm bored now, could someone else finish this?
Some of these don't seem to be changing but I swear I uploaded the correct files...

http://avisynth.org/mediawiki/Denoisers Updated.
Confirmed with at least the MCTD screens.
jmac698
23rd September 2011, 00:28
can u post uncompressed samples of a few frames? I was thinking just a few frames is enough for the various tools.
It might be even better and more useful to the research community to use the standard test videos.

http://media.xiph.org/video/derf/
includes parkjoy (which I know is available in 10bit), sintel trailer, susie/foreman etc.
Nightshiver
23rd September 2011, 01:40
Well, I'm almost done with my encodes for the samples. I'll see what I can do. But I'll be including the original samples with the encoded ones as well. Thanks for the site, never heard of it before. Are the ones you are mentioning (parkjoy, sintel trailer) one's that you'd like for me to do?

A small note to all, I won't be able to provide an encoded sample for the tnlmeans parts. They are just taking wayyy to long (like, ~30 hours for a small 1200 frames).
Settings used in MeGUI for the videos was unrestricted 1pass lossless into .mp4 container.
aegisofrime
23rd September 2011, 02:25
Well, I'm almost done with my encodes for the samples. I'll see what I can do. But I'll be including the original samples with the encoded ones as well. Thanks for the site, never heard of it before. Are the ones you are mentioning (parkjoy, sintel trailer) one's that you'd like for me to do?

A small note to all, I won't be able to provide an encoded sample for the tnlmeans parts. They are just taking wayyy to long (like, ~30 hours for a small 1200 frames).
Settings used in MeGUI for the videos was unrestricted 1pass lossless into .mp4 container.

Hmmm, try NLMeansCL, if you have a capable GPU?

http://forum.doom9.org/showthread.php?t=158925
Nightshiver
23rd September 2011, 03:33
For the degrainmedian samples for the 300 clip, it skips/freezes the video for a few seconds, but it's not to big of a deal as you still get several seconds in the beginning and end. I'll have to test out NLMeansCL tomorrow afternoon.
jmac698
23rd September 2011, 04:18
I could suggest that you use 120 frames from tnlmeans, that would take ~3 hours, but at least I can look through frame-by-frame for artefacts, but also I want to take some still frames and do objective measurements metrics (blur+noise).

Also, I went looking for some very high quality frames to test, and finally found them. Here ya go:
http://media.xiph.org/sintel/sintel-1k-png16/
So the question then becomes, how to load 16bit png into Avisynth? Well, that lead to ImageMagick, which finally lead me to write a high bit-depth image import for Avisynth :) :) Now we can have extremely high quality noiseless images to test with, but also I know how to add highly accurate modelled film grain.

The final result will be the ultimate reference quality test images, and also measurements of the different tools with objective measurements. Gonna be a lot of work though, but thanks for your efforts so far, we'll have to share some efforts.

Update:
Well I got this far,

convert %08d.png[8970] -evaluate and 65280 -set filename:f msb%t.bmp +adjoin %[filename:f]

convert %08d.png[8970] -evaluate and 255 -evaluate leftshift 8 -set filename:f lsb%t.bmp +adjoin %[filename:f]

I can do a lot better though.
Chikuzen
23rd September 2011, 09:12
cd /d d:\sintel_rgb48
ffmpeg -f image2 -i %08d.png -pix_fmt yuv444p16le -vcodec rawvideo -an -f rawvideo d:\sintel_raw_yuv\yuv444p16le.yuv

#read_16bit_yuv.avs
LoadPlugin("RawSource26.dll")
LoadPlugin("flash3kyuu_deband.dll")
RawSource("d:\sintel_raw_yuv\yuv444p16le.yuv", width=2048, height=436, fpsnum=24, fpsden=1, pixel_type="i444")
f3kdb_dither(stacked=false)
Nightshiver
23rd September 2011, 11:44
Well, I'll be uploading now, but it will take a long time. Over 10GB of video clips to upload.
aegisofrime
23rd September 2011, 13:39
Well, I'll be uploading now, but it will take a long time. Over 10GB of video clips to upload.

Thanks for the hard work you put into this! :goodpost:
Nightshiver
24th September 2011, 15:23
Whew, alright people, the first batch is up. It's the first sample, the one of the blackboard/classroom.

original
http://www.mediafire.com/?dqsoait8r28hf2k

degrainmedianmode0
http://www.mediafire.com/?5ro85gnj4o9y58w

degrainmedianmode1
http://www.mediafire.com/?xpss50humuj4mox

degrainmedianmode2
http://www.mediafire.com/?8ea0modmtounxdx

degrainmedianmode3
http://www.mediafire.com/?5trapye0dn5ar6z

degrainmedianmode4
http://www.mediafire.com/?83880nwa8b38dxw

degrainmedianspecial
http://www.mediafire.com/?o78ce5aa6nc1cso

dfttest
http://www.mediafire.com/?ld0a7t2tdvm7v05

dfttestsigma05
http://www.mediafire.com/?e0u7jp7iqog3fx1

dfttestsigma1
http://www.mediafire.com/?5hnf52rkysahsb6

dfttestsigma3
http://www.mediafire.com/?6cjxwivgqrjbuqa


fft3d pic1
http://www.mediafire.com/?9hc1nph9911on55

fft3d pic2
http://www.mediafire.com/?0d2d6k8wwj5ckwe

fft3d pic3
http://www.mediafire.com/?0bco2tdqhq5j4qt


fft3d pic4
http://www.mediafire.com/?ssr67m3nelakl8x

mcspuds1
http://www.mediafire.com/?21nmy3h6inhpp5t

mcspuds2
http://www.mediafire.com/?xtwak5708eq6qle

mcspuds3
http://www.mediafire.com/?ghnnjhyg2qonbbf

mcspuds4
http://www.mediafire.com/?h0l4v44stlrb4d4

mctemporaldenoiseverylow
http://www.mediafire.com/?ujfx2qoc44h84zn

mctemporaldenoiselow
http://www.mediafire.com/?sb8d1s32fzae32m

mctemporaldenoisemedium
http://www.mediafire.com/?bpfb33o9ze6z642

mctemporaldenoisehigh
http://www.mediafire.com/?pqsli2q5fiwscyp

mctemporaldenoiseveryhigh
http://www.mediafire.com/?ezcka7yyzs1obfa

mdegrain2sad400
http://www.mediafire.com/?daonvq01xd7x2ai

mdegrain3sad400
http://www.mediafire.com/?bgbl9o0g48m8vc8

mvdegrainmultiframes3sad400
http://www.mediafire.com/?edjd42pgzddh78n

removegrain1
http://www.mediafire.com/?lwsj3cg6ujrls55

removegrain5
http://www.mediafire.com/?x2m4pr5g9tjed15

removegrain10
http://www.mediafire.com/?9x32b1gaiai033w

removegrain19
http://www.mediafire.com/?w1cktifn1s9tg6w

ttempsmooth1
http://www.mediafire.com/?nnkwab6cjhs43l5

ttempsmooth2
http://www.mediafire.com/?wpvs532omigb2qp

ttempsmooth3
http://www.mediafire.com/?tc8f9na7bw8mhzz

ttempsmooth4
http://www.megaupload.com/?d=WMQOV54P

vaguedenoiser1
http://www.mediafire.com/?v0ptlpjh2tjn9qg

vaguedenoiser2
http://www.mediafire.com/?ec4zylbub0dqyl8

vaguedenoiser3
http://www.mediafire.com/?crwdt03dithc1im
Dogway
24th September 2011, 18:43
smdegrain(tr=6,thSAD=600,contrasharp=true,lsb=true,refinemotion=true,prefilter=2)
http://www.mediafire.com/?81o4zz0in25muwr

I also think these tests have no real meaning. You are mixing spatial denoisers, with temporal, and each of them in different strengths. Quality has also many broad interpretations.
Nightshiver
24th September 2011, 19:14
1) I never tested smdegrain because it is the same thing as mdegrain or mvdegrain.
2) If they have no real meaning, then why are you posting here? I'm glad that you think I'm wasting many hours of my time, but keep it to yourself next time.
3) I am simply showing others what the different options out there do so that they can see what they do. Granted, anyone could do this by themselves, but not everyone is going to try to test every option (or several) to see what fits the bill. This is a quick, or somewhat quicker way of determining what may be the right option for them. Also, what is wrong with mixing different denoisers? I did that on purpose. There is no one option, and many functions deliver different results. (As this should be painfully obvious)
Dogway
24th September 2011, 19:39
calm down. Indeed smdegrain is a wrapper of mdegrain, but it can use 16 bit and higher radius with the cretindesalpes mod. Also it adds refinemotion(MRecalculate) which not many wrappers do. Most of your tests involves mdegrain so I add my 2 cents here, but its strange when you also add dfttest, tnlmeans, removegrain, etc
Different sources need different denoisers, here source has a lot of grain, so it's only one example to test denoisers against. I also found mctemporaldenoise to perform the best in this example, mainly because its two-pass option, and fizz noise reduction. I just don't use it much because it requires many plugins, and is slow.
I'm not throwing down your work, I only wasn't agree from the point of view, but its only my opinion. I also did some tests (http://forum.doom9.org/showthread.php?p=1479909#post1479909) a few months ago.
Didée
24th September 2011, 19:59
I'm glad that you think I'm wasting many hours of my time, but keep it to yourself next time.
Let's put it like this: considering that you started a project that is quite time-consuming, perhaps you should have put a little more time beforehand into strategy planning.


I am simply showing others what the different options out there do so that they can see what they do.
[...]
Also, what is wrong with mixing different denoisers? I did that on purpose. There is no one option, and many functions deliver different results.
Fair enough. But then, fact is that for purposes of degraining (and your samples are mostly dealing with film grain removal), a spatial filter is a quite different story than a temporal or spatio-temporal denoiser. However, for the non-expert users, it is not even clear which of the filters are of which kind.

Add to that that you are e.g. using FFT3dFilter in 4 different (!) configurations, but all of them are only spatial (albeit the bigger strength in grain removal comes from temporal filtering).
In comparison, dfttest (being a "mightier relative" of FFT3D) is used only in spatio-temporal mode, never in spatial-only. That's not really a "fair" comparison of what either filter possibly can do. It smells more like "random settings".


Again: there's big respect for the effort, but the strategy is lacking.
Nightshiver
24th September 2011, 20:28
Unfortunately, I cannot possibly post every option there is. I understand where you are coming from, but I do not exactly have oodles of time available to me, and so I chose the options that are, for the most part, used most often in the operation of removing film noise.

For instance, I only used the rather simple calls for dfttest because that is what is used most often, both from my experience and what I have been able to find by searching the forum.
The same goes for fft3d. More often than not, the different sigma's are set and very few other options are every messed with, at least from what I could tell. I know that I haven't possibly searched every post or even most of them, so I just never saw many uses of fft3d in any other function other than spatial, and the same for dfttest. I suppose one of the reasons I chose these particular settings was to show off the more popular settings basic users might be prone to messing with and what they might more commonly use.

I know that you know far more than I do, Didee, about the different options that can be used. So, I would like to request that if you have time, would you mind considering providing me some settings that you would find preferable to see?

I will still continue to upload the other samples from 300 and I will also attempt to get the TNLmeans samples up. Judging from these recent posts, I will also try to find out other syntax's for using fft3d in temporal mode and dfttest in spatial mode.

As to the lack of a strategy, I quite agree. I was inspired to do this because of the recent thread for the MeGUI filter revival. But I currently work in the military, and I was only able to conduct my tests at the end of the day where I only have a few hours to myself. My main job takes up most of my time, so I didn't really have the time to try to flesh out more thoughtfulness. I started doing the tests the same night I saw the thread. But I'd still like to press on and continue until it does work :)

Another edit: @Dogway

I do apologize for the offensive way I responded, but to me, your short comment of "these tests have no real meaning" came off to me as "therefore, this thread has no meaning" and I took it a little insultingly because of the amount of time and effort I had put into this. I don't think I ever saw a thread before about denoiser comparisons like this (as large or trying to be as comprehensive) and so I have nothing to go off of, as I've also never done anything like this before.
jmac698
24th September 2011, 20:55
@Night
I, for one, congratulate our new denoising overlord! I agree, I've never seen such a comparison before. I was already happy to see some screenshots, but now videos? I can immediately see that mctemporaldenoise is superior than other tested settings, and that's good enough for me!
However I'm still working on my own objective tests, in a more scientific way. I have some very high quality, noiseless video, which is also copyright free, and I want to test artefacting and filmgrain.

@all
About the new tests, starting with a very high quality 16bit, noiseless source, I'd like to run denoisers on it with/without added noise, to measure artefacting/denoising. I can provide usual error measures, ssim, mse, psnr, msu noise estimation, msu blur estimation. I want to focus on one filter at a time and it's settings, to determine which affect denoising. Assuming this leads me to find the best settings for each tool (that the settings form a linear trend in performance), I can then compare tools. Data will be categorized by spatial, temporal, or both. Any other ideas for methodology? Please note I'm researching my own questions, I can't necessarily take every request.
Didée
24th September 2011, 21:48
starting with a [...] noiseless source, I'd like to run denoisers on it with/without added noise,
Depending on what you want to test, this scenario may or may not be realistic at all.

- It is a very different ballgame if you have a high-bitrate BR source (with inherently very "clean" grain reproduction), or if you're starting with an oldschool DVD source (or even an SD DVB capture). In a sense, the latter is more difficult, because grain reproduction of bitrate limited Mpeg-2 is not clean. Mpeg-2 compression of (stronger) grain usually introduces lots of DCT noise, and more often than not even DCT flicker (i.e. the low frequencies, or even the DC component, are not temporally stable).
In addition to this, using just "AddGrain(x)" is not a overly good approximation of realworld film grain. These artificial grain generators aim for zero-mean in rather small localisations. Film grain usually is much more "rough" than that - it might be reasonably close to zero-mean too, but only in a (much) bigger spatio-temporal window.

It's a bit like if you can make some zero-to-hundred sprint tests with a new car, but what you really want to find out is if the new car is suited for city traffic or holiday journey: The test is simple and scientific, but practice is much more complex.
jmac698
24th September 2011, 22:37
@didée
Good point - however, starting with my source I can compress to mpeg2. Anyhow, this makes me think that we don't want denoisers per se, but 'DCT denoisers', siince the properties could be quite different.
As for noise, I'm not using addnoise - I'm using a carefully measured noise model for specific films. You are right that they have different properties to guassian noise. Now at least I know why you say 'it depends on the source' - you could at least, be referring to the ability to 'DCT denoise' different compressed noise. I could compress pure noise on a flat frame to see what happens?
Dogway
25th September 2011, 01:23
http://en.wikipedia.org/wiki/Image_noise
http://en.wikipedia.org/wiki/Compression_artifact
http://en.wikipedia.org/wiki/Noise_%28electronics%29

@jmac698: For some time now I have been more interested in noise itself than the denoisers. A more pictural example of different types of noise and ways to combat it. From the inherent image grain up to the compressed grain artifacts, through analog electronic devices artifacts.
lossless
|
|
digital________|_________analog
|
|
heavy compressed
Plus a 3rd axis indicating clean-------------grainy

This is very simplified, since video can go through different mediums, but you get the point.
Mounir
25th September 2011, 02:27
I'd be more interested in denoisers comparisons regarding vhs / laserdisc noise. Then after that perhaps a test for DCT-errors noise (mpeg2, mpeg4).
jmac698
25th September 2011, 03:03
I'd have to say in some cases noise is noise - don't get too carried away about there being different 'types' of noise. I would say temporal denoisers for example reduce *anything* which is not consistent with the image, including specks etc. It's possible for even spatial denoisers to be adaptive in some sense; there certainly are tools which gather a 'noise profile'.
I do have some VHS noise available however, so perhaps I can start by discussing the various types of noise and see how they differ.
Off the top of my head, there's about 3 kinds of electronic noise which is based in physics and they have different frequency responses (like pink noise, brown noise etc.).
And by DCT-noise we mean normal noise which was compressed; thus it's not the original noise anymore and will have different properties.
As for digital noise I think quantization errors (round-off noise) is generally white noise.
Noise isn't always bad either, it's useful in different situations. It probably helps with banding without having to dither?
jmac698
25th September 2011, 07:11
@Chikuzen,

RawSource("d:\sintel_raw_yuv\yuv444p16le.yuv", width=2048, height=436, fpsnum=24, fpsden=1, pixel_type="i444")

There is a problem here - if I want to use the HD version, I should use width=3840, however width is max 2880.
Do you have any suggestions as a workaround?
Thanks
Chikuzen
25th September 2011, 07:54
@Chikuzen,
There is a problem here - if I want to use the HD version, I should use width=3840, however width is max 2880.
Do you have any suggestions as a workaround?
Thanks

what?
I must already have changed MAX_WIDTH into 4096.
http://forum.doom9.org/showthread.php?p=1504259#post1504259
jmac698
25th September 2011, 08:04
@Chikuzen,
Thank you for your speedy reply. I am referring to this file:
http://avisynth.org/warpenterprises/files/rawsource_25_dll_20060728.zip
I see the line you are referring to:

#define maxwidth 2880

However, I do not have a setup to compile plugins at the moment. Do you have a compiled mod available?
Do you see any reason to restrict the width? In fact we need even higher to support all samples. There are even some 4k files available. It's only gonna get bigger too.
Chikuzen
25th September 2011, 08:30
RawSource26_max_width_65536.zip (http://www.mediafire.com/download.php?9jkkc7h8ch9lglt)
Nightshiver
26th September 2011, 12:03
Alright, here's the last batch, the 300 HD scenes.

original
http://www.mediafire.com/?u5e8n1ppa8cupfy

degrainmedianmode0
http://www.megaupload.com/?d=2LXN08OP

degrainmedianmode1
http://www.megaupload.com/?d=SW6A6XUV

degrainmedianmode2
http://www.megaupload.com/?d=5IKH76Z6

degrainmedianmode3
http://www.megaupload.com/?d=CPQ5Q1ON

degrainmedianmode4
http://www.megaupload.com/?d=REMWREJ3

degrainmedianspecial
http://www.megaupload.com/?d=S1UHW8EM

dfttest
http://www.mediafire.com/?bco4lp6sasycbjl

dttestsigma025
http://www.megaupload.com/?d=AT8CFNZD

dfttestsigma05
http://www.megaupload.com/?d=5HB2YKVF

dfttestsigma1
http://www.mediafire.com/?6rdv2rh8i2n9wcn

dfttestsigma3
http://www.mediafire.com/?61axnw517h61q6d


fft3d pic1
http://www.mediafire.com/?pv8p9a721s78m35

fft3d pic2
http://www.mediafire.com/?xkg8lb7jput6peo

fft3d pic3
http://www.mediafire.com/?fj85z2isbiycru6


fft3d pic4
http://www.mediafire.com/?vrv0eytot12m0nl

mcspuds1
http://www.mediafire.com/?0ouak22760mbze9

mcspuds2
http://www.mediafire.com/?bwk5efdfb2obfuy

mcspuds3
http://www.mediafire.com/?hooud4f0nsrc4q4

mcspuds4
http://www.mediafire.com/?5n2zxzt5yf9ay3r

mctemporaldenoiseverylow
http://www.megaupload.com/?d=RX91D53R

mctemporaldenoiselow
http://www.megaupload.com/?d=SD4LC3V3

mctemporaldenoisemedium
http://www.mediafire.com/?a3eu53kde8esmgu

mctemporaldenoisehigh
http://www.mediafire.com/?fbofc7c0nubt450

mctemporaldenoiseveryhigh
http://www.mediafire.com/?dd1azc12oe64g3f

mdegrain2sad400
http://www.mediafire.com/?1agwz5x8dcdxon8

mdegrain3sad400
http://www.mediafire.com/?4tj3zx4ai3xa58s

mvdegrainmultiframes3sad400
http://www.mediafire.com/?cqy8q4cgqc873f0

removegrain1
http://www.megaupload.com/?d=O27I4A25

removegrain5
http://www.megaupload.com/?d=8RWNJ0TG

removegrain10
http://www.megaupload.com/?d=Q44GDCZ1

removegrain19
http://www.mediafire.com/?bytuhoom4b1ufa1

ttempsmooth1
http://www.megaupload.com/?d=8MH3JYTN

ttempsmooth2


ttempsmooth3
http://www.mediafire.com/?ah9zvv055m8dpwc

ttempsmooth4
http://www.mediafire.com/?aaln9y51rrcajkz

vaguedenoiser1


vaguedenoiser2
http://www.mediafire.com/?l4l1cltv2161ybs

vaguedenoiser3
http://www.mediafire.com/?71g7w0hx4h6jzig


Will get those last two upped later.
jofafrazze
13th February 2012, 15:12
Thanks a lot for those efforts! I'm afraid the FBI ate the megaupload videos, so I'd like to request a repost, or even a .torrent creation. I for one wouldn't mind seeding some fine work like this!
Mole
2nd April 2012, 07:39
Thanks for your efforts.

However, in the classroom sample, you should not have cropped and resized the denoised sample.

What resizer did you use BTW?

Also, did you denoise before or after resize?
Mole
4th April 2012, 09:42
This has been denoised by Neat Video 3.0 Pro in VirtualDub.

I just used Auto Profile on first frame and default settings.

It's the 300 HD sample.

http://d01.megashares.com/?d01=WvYb4k0
http://www.filefactory.com/file/61sq216puudn/n/300neatdefault.avi
xekon
3rd January 2013, 09:50
I just wanted to post a Huge thank you for taking the time to do this. For one, there are so many different plugins available.

For me the most time consuming part of learning and testing the various denoisers is knowing what settings will work and also how to structure them for 2 pass + settings, and in some cases a combination of 2 denoisers/filters.

I really like the idea of have a list of popular/optimized settings that generally work for a type of source.

For me I was more interested in your classroom scene than the other one because there were more large flat areas with grain, and I mostly encode Anime, so I could see some of the results you got with different denoisers under different settings. I also noticed a few of the denoisers+settings you used because I had tried the same settings after finding posts by seasoned avisynth users posted in the forums.

For me I am mostly interested in Denoisers that will do a HEAVY amount of denoising but without losing line details.

I have been mostly using the 2pass fft3dgpu filter/settings and it does a great job of maintaining geometry shape, but I have noticed under scenes with motion or thinner lines that the thin lines can get washed out completely.

The ones in here of the classroom that impressed me most were:

1.
MCTemporalDenoise(settings="high")

2.
source = last
vectors = source.MVAnalyseMulti(refframes=3, pel=2, overlap=4, sharp=1, idx=1)
source.MVDegrainMulti(vectors,thSAD=400,idx=1)

3.
dfttest()

4.
tnlmeans(h=2.2,ax=10,ay=10,bx=0,by=0,sx=3,sy=3,a=100)

Right now I am seriously considering switching from fft3dgpu to MCTemporalDenoise, speed is of no concern to me only the final product.

I would be very interested to see more of the most highly regarded/popular denoisers with more settings used by seasoned pros at Denoising, Although a lot of settings were certainly covered here.

For instance if it would have not been for a post by *.mp4 guy (http://forum.doom9.org/showthread.php?p=1154066#post1154066)

I honestly would have NEVER thought to use FFT3Dgpu 2 pass like that.