Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

 
 
Thread Tools Search this Thread Display Modes
Prev Previous Post   Next Post Next
Old 10th August 2006, 23:05   #1  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
What denoiser is the worst ? Objective test attempt

What denoiser is the best? It is good, but not-allowed question here at the forum.
Why? Becourse it is subjective, and dependent from your impression and the source type, and the noise type, codec, etc, etc.
Subjective rating is not well defined, and is possible probably by some poll only.
Legal question: what denoiser from the list is the currently better for some given video with given noise value and type, by some objective criterium?
But what compare, and how to compare?
In scentific publications it is not a new question.

Typical procedure:
1. Use some standard test sequence (clean, free of noise).
2. Add given noise amount, with normal distribution.
3. Clean this noisy sequence by denoiser.
4. Compare cleaned result with original sequence, calculate some metric (signal to noise ratio by PSNR, SSIM, VQM criterium)

(This method is also often used in image processing with test images "Lena", etc.)

I suggest use similar method here.
Lets make our own comparison!
(I often use it for my filter development).

Step 1.
There are many standard freely-redistributable test sequences (used in video codec development, etc).
I found and downloaded some of them.
Lets use so called "Flower" sequence. It has quite small file size and has some movement in it.
The original is at
http://bmrc.berkeley.edu/ftp/pub/mul...eg/EncodeData/
First part (frames 0 to 29) is in the file "flower.0-29.tar.z"
Direct link: http://bmrc.berkeley.edu/ftp/pub/mul...wer.0-29.tar.z
This archive file can be unpacked by 7-zip, Winrar or some other archiver tools.
Archive contains image files sflowg.0.yuv to sflowg.29.yuv.
This raw YUV files has I420 format (planar 4:2:0 format similar to YV12, but with swapped U, V planes). Width=352, height=240

(not interlaced).
The file sequences can be open with ImageSequence plugin by WarpEnterprises
(Get it at http://www.avisynth.org/warpenterprises)
and we get normal YV12 video.

Code:
rawsequence("h:\flower\sflowg.%d.yuv",width=352,height=240,pixel_type="I420").trim(0,29)
Realy, video is not quite normal (it has colors range from 0 to 255), so lets convert it to recomended TV range (16-235):
Code:
source=coloryuv(levels="PC->TV")
Step 2.
Lets add some grain with AddGrain plugin by trbarry (or use modified version AddGrainC by Foxyshadis for YUY2, chroma).
(Get it at http://www.avisynth.org/warpenterprises)
Added noise amount is controlled by first parameter. It is really sigma*sigma (or noise variation),
where sigma is the noise standard deviation.
Let's use only luma (Y) channel for testing.
Lets use sigma=5 for test. It is rather big value, and the noise is visible.
(In scientific articles larger sigmas are usually used, about 10-20).
Noise variation parameter = 5*5=25.

Code:
noisy=source.addgrain(25,0,0)
For temporal denoisers, noise must not be the same for all frames, so we can not use seed option of AddgrainC.

Step 3.
Use your favorite denoiser here!
The null-transform denoiser:
Code:
denoised=noisy
Of course, I use FFT3DFilter as example.
Code:
denoised=noisy.FFT3DFilter(sigma=4.0, bt=4, bw=32, bh=32, ow=16, oh=16)
Step 4.
Lets use SSIM metric for camparison of denoised with original source images. (SSIM is better than PSNR).
Well known SSIM plugin by Lefungus
(Get it at http://www.avisynth.org/warpenterprises).
EDIT: It seems, that various versions of SSIM plugin exists, without source.


Code:
SSIM(source,denoised,"nul","nul")
I firstly used "nul" as dummy output files.
Now I use full info:
Code:
SSIM(source,denoised,"ssim.txt","ssimA.txt")
Open script in VirtualDubMod.
We get SSIM values typed on frame.
Skip some first frames 0,1,2,3 (for temporal denoisers), lets will consider frame 4.
The values for noisy video (null-transforn denoiser) is:
n=4
Y=0.9343 U=1.0000 V=1.0000
SSIM=0.9474 Wieght=1.0000

In this test, by my first suggestion,
we are interested not in summary SSIM, but in Y (luma) SSIM value (=0.9343) only! Chroma is not changed.
Reported last summary SSIM is average of channels (0.8*Yssim + 0.1*(Ussim + Vssim)).
But since this plugin write summary SSIM (not luma) to output file SSIM.TXT,
we now will use this summary per-frame SSIM for comparing.
So, play scipt in any player, and output files will be created.
File SSIMA.TXT will contain average clip SSIM.

--------------------------------------------------
Full test script:
Code:
rawsequence("h:\flower\sflowg.%d.yuv",width=352,height=240,pixel_type="I420").trim(0,29)
coloryuv(levels="PC->TV") 
source=last
noisy=source.addgrain(source,25,0,0)
denoised=noisy#.DENOISER(parameters) # put your real denoiser script here
SSIM(source,denoised,"ssim.txt","ssimA.txt")

Here is result for FFT3DFilter v.1.85.
My best settings is:
Code:
denoised=noisy.FFT3DFilter(sigma=4.0, bt=4, bw=32, bh=32, ow=16, oh=16)
FFT3D result:
n=4
Y=0.9794 U=1.0000 V=1.0000
SSIM=0.9835 Wieght=1.0000


DeGrainMedian best script (almost default!):
Code:
denoised=noisy.degrainmedian(mode=1,limity=4,limitUV=0)
But result is not so great:
n=4
Y=0.9525
SSIM=0.9620

So, FFT3Dfilter SSIM score is above DegrainMedian.
FF3DFilter is better in this thread.
I will tune Vaguedenoiser, MVDegrain soon.

Please download the test clip "Flower" (link is above), tune your favorite denoiser plugin (or script) and post results!


You may also post speed results (but they are system-dependent).
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 8th September 2006 at 19:33.
Fizick is offline   Reply With Quote
 

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:28.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.