Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th May 2006, 08:24   #1  |  Link
AVIL
Registered User
 
Join Date: Nov 2004
Location: Spain
Posts: 408
New plugin TNLMeans

Hi all,

A new plugin is born (silently?).

Name: TNLMeans.
Age: v1.0 Beta 1
Parents: Tritical
Job: 3D Denoiser

I'm surprised. First for the quality of denoising. Second for the absence of news on this forum. I advice to try it. Only a complaint: its slow.

Thanks Tritical.
AVIL is offline   Reply With Quote
Old 19th May 2006, 12:13   #2  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,556
I read the papers linked to the filter on his site, and it looks like an awesome little denoiser. I always prefer to know how something works and comparisons to existing methods before I dig in. Anyway, I'm interested regardless of speed and I'll report back once I've given it a whirl. I know Tritical has been quite busy lately though, cool that he's managed to make this.

Last edited by foxyshadis; 19th May 2006 at 12:15.
foxyshadis is offline   Reply With Quote
Old 19th May 2006, 12:13   #3  |  Link
buzzqw
HDConvertToX author
 
Join Date: Nov 2003
Location: Cesena,Italy
Posts: 6,552
wow...

where ?

BHH
__________________
HDConvertToX: your tool for BD backup
MultiX264: The quick gui for x264
AutoMen: The Mencoder GUI
AutoWebM: supporting WebM/VP8
buzzqw is offline   Reply With Quote
Old 19th May 2006, 12:52   #4  |  Link
AVIL
Registered User
 
Join Date: Nov 2004
Location: Spain
Posts: 408
At buzzqw:

Quote:
Originally Posted by foxyshadis
AVIL is offline   Reply With Quote
Old 19th May 2006, 12:54   #5  |  Link
buzzqw
HDConvertToX author
 
Join Date: Nov 2003
Location: Cesena,Italy
Posts: 6,552
ok found on Tritical site ! thanks foxyshadis

just one note... is very very slow (from 9.3 fps to 0.7... and it has broken multithread on x264...now the second core is near sleeping... don't know about this)

BHH
__________________
HDConvertToX: your tool for BD backup
MultiX264: The quick gui for x264
AutoMen: The Mencoder GUI
AutoWebM: supporting WebM/VP8
buzzqw is offline   Reply With Quote
Old 19th May 2006, 13:24   #6  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,556
It doesn't break multithreading, it's just that the x264 part of processing (the second core) is only 10% of what it was, because fetching the avisynth frame is so much slower. If you used MT plugin, you'd probably see both cores at full steam (and an amazing 1.2 fps, heh). I guess that's why it's labeled beta, probably mostly unoptimized. xD

Oh, and if you think it's slow now, try enabling temporal processing (az=1+). 3 times as slow for radius 1 as 0. =D

Last edited by foxyshadis; 19th May 2006 at 13:49.
foxyshadis is offline   Reply With Quote
Old 19th May 2006, 19:06   #7  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Yeah, I didn't post anything about this filter because for the moment it is really too slow to be usuable, though the method does produce nice results. The reason it is so slow is that it for every pixel in the frame you do a convolution of size (2*Ax+1)*(2*Ay+1) (the search window) and for every pixel in the search window you have to calculate the gaussian weighted sum of squared differences of its support window (Sx*2+1)*(Sy*2+1) to the center pixel's support window. In the paper they suggest using a search window of 21x21 (Ax=10,Ay=10) and a support window of 7x7 (Sx=3,Sy=3)... which takes about 100 seconds per frame for a 720x480 frame on my comp (1.6 pentium M).

In the paper they suggest two methods to speed the process up: 1.) do blocks instead of single pixels and 2.) a multiscale version that uses Ax=Ay=1 at the initial resolution and Ax/2,Ay/2,Sx/2,Sy/2 at 1/2 the original resolution. I have implemented the block based approach (Bx/By), and it does cut computation time by roughly (2*Bx+1)*(2*By+1). However, the results really start to suffer with Bx/By greater than 2. The multiscale version should speed things up by about 16x, but I haven't implemented it yet and don't know how much it impairs the output. Possibly the combination of a multiscale version and Bx=By=1 should get it into the fps range instead of the spf range.
tritical is offline   Reply With Quote
Old 19th May 2006, 20:06   #8  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
tritical's filters are always high quality.
Here he beat my early fft3d by slowness ?
I will try it, thanks.

But I really like this:
Code:
TO DO LIST:

     - add multiscale version
     - Release a v1.0 final
Intrerestly, what tritical's filter will be first "final v.1.0" filter.
Fizick is offline   Reply With Quote
Old 19th May 2006, 20:19   #9  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
tritical: What about using a fft to speed up the convolution? Something like the true gausian function in variable blur?
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Old 21st May 2006, 13:41   #10  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
The weights for the search window around each pixel are dependent on the local image structure and aren't known until the gaussian weighted ssd for the neighborhood of each pixel inside the search window is calculated. Therefore, the fft approach wouldn't help (I think anyways). I did realize one simple optimization for the non-block based method which takes advantage of the fact that the weights are symmetric (if pixel A is assigned weight y in pixel B's window then pixel B will also be assigned weight y in pixel A's window). Therefore, as you process the image you can buffer results and cut the computation time in half. I also implemented a multi-scale version and it works reasonably well. I had to make some adjustments to the method in the paper because it tended to mess up fine details/lines... so the overall speed increase isn't quite as much as expected. Some timings on my comp for a 720x480 frame with ax=ay=10 and sy=sx=3:

no multiscale, bx=by=0: ~69 seconds
no multiscale, bx=by=1: ~15 seconds
no multiscale, bx=by=2: ~5 seconds
no multiscale, bx=by=3: ~3 seconds
multiscale, bx=by=0: ~10 seconds
multiscale, bx=by=1: ~2 seconds
multiscale, bx=by=2: ~1 second

For comparison, tbilateral with diameterL and diameterC set to 21 takes about ~1.5 seconds per frame on my comp.

Anyways, I'll hopefully have a new version up in a few days with the multiscale option and bx=by=0 2x speed up. As far as results go I would say that nl-means is superior to bilateral filtering for zero mean gaussian noise, but it seems to be less effective than bilteral filtering for typical mpeg compression artifacts.

Last edited by tritical; 21st May 2006 at 13:45.
tritical is offline   Reply With Quote
Old 21st May 2006, 14:21   #11  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
i can't seem to be able to access the site. uni of missouri (huh, rhymes) gives me a 404 and invites me to check out their athletics program amoung other things

could some kind soul mirror this so i, and similarly download handicapped lurkers could try it out? sounds promising.
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 21st May 2006, 15:03   #12  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
@tritical,

Is your homepage down?

edit: i should have read Mug's post

Last edited by Wilbert; 21st May 2006 at 17:12.
Wilbert is offline   Reply With Quote
Old 21st May 2006, 16:10   #13  |  Link
Serbianboss
Registered User
 
Serbianboss's Avatar
 
Join Date: Mar 2006
Posts: 232
Any mirror for download?
Serbianboss is offline   Reply With Quote
Old 21st May 2006, 18:43   #14  |  Link
Backwoods
ReMember
 
Backwoods's Avatar
 
Join Date: Nov 2003
Posts: 416
4th on the request.
Backwoods is offline   Reply With Quote
Old 21st May 2006, 19:07   #15  |  Link
buzzqw
HDConvertToX author
 
Join Date: Nov 2003
Location: Cesena,Italy
Posts: 6,552
sorry i have now only the dll
www.64k.it/andres/TNLMeans.dll

BHH

( i hope tomorrow to recover the full zip file)
__________________
HDConvertToX: your tool for BD backup
MultiX264: The quick gui for x264
AutoMen: The Mencoder GUI
AutoWebM: supporting WebM/VP8
buzzqw is offline   Reply With Quote
Old 21st May 2006, 19:10   #16  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,901
This link works fine for me:

http://www.missouri.edu/~kes25c/

Maybe it just came back up.
Guest is offline   Reply With Quote
Old 21st May 2006, 22:54   #17  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
There was a power problem of some kind on campus yesterday which effected a few computer systems so that might be the reason it was down. Also, if anyone is trying the filter a few faster settings to try would be:

tnlmeans(ax=3,ay=3,sx=2,sy=2,bx=1,by=1) or
tnlmeans(ax=4,ay=4,sx=3,sy=3,bx=2,by=2)

those should give pretty good results without being unbearably slow (ax=3,ay=3,sx=2,sy=2,bx=1,by=1 will probably be the new defaults). The 'b' parameter that I added to limit the minimum amount of weight that the center pixel can be given seems to be set too low, and impedes more than it helps. Try setting it to a really large value ~1000000.0 to disable its effect.

Last edited by tritical; 21st May 2006 at 23:04.
tritical is offline   Reply With Quote
Old 23rd May 2006, 04:36   #18  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
Thanks tritical, this is a very interesting filter, and I'm sure it will improve rapidly...

Here are some nice screenshots made with virgin transport streams @ 1920x1080p.

For the fft3d versions, I used:

FFT3DFilter(sigma=4,plane=4)

and for the TNL versions:

tnlmeans(ax=3,ay=3,az=10,sx=2,sy=2,bx=1,by=1)

-------------------------------------------------
Gladiator frame 8408, original
Gladiator frame 8408, fft3d
Gladiator frame 8408, TNL
Gladiator frame 8408, Neat Image

Its biggest problem is areas that are not finely detailed, ie. vague/blurry. These areas are blurred alot, kind of like what Neat Image does. Check these images. Note how it retains detail nicely, but the vague area in the lower right-corner gets severely blurred:

Gladiator frame 41923, original
Gladiator frame 41923, fft3d
Gladiator frame 41923, TNL
Gladiator frame 41923, Neat Image

The image could be broken down into blocks and those blocks split until similarity yields enough advantage for cross-window averaging to have maximum profitability with minimum penalty.

Small pixel windows could be expanded until decreasing similarity makes further expansion unprofitable.

The window is compared to other windows located on a rectangular grid defined by window width/height? Overlap or exhaustive search might yield significantly better matches, even if its range is shorter.

Use motion vectors to calculate rotations and other non-shift translations between frames, then apply before window search, could create much more and better same- and cross-frame matches.

Last edited by Isochroma; 23rd May 2006 at 18:39.
Isochroma is offline   Reply With Quote
Old 23rd May 2006, 10:23   #19  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Have you tried decreasing the h parameter? The h parameter controls the strength of the denoising (higher = stronger, lower = weaker). The default of 3.0 is definitely way too much for most sources.

Quote:
The image could be broken down into blocks and those blocks split until similarity yields enough advantage for cross-window averaging to have maximum profitability with minimum penalty.

Small pixel windows could be expanded until decreasing similarity makes further expansion unprofitable.
I don't understand exactly which windows you are talking about. The search window (ax,ay), support window (sx,sy), or base window(bx,by)? Generally, bx=by=0 so that the base window is a single pixel and ax/ay as large as possible will yeild the best results. The size of the support window and the standard deviation of the gaussian weights for the ssd calculation 'a' are where there is no easy rule. The larger the support window the better fine texture will be preserved, but less noise will be removed. Likewise, the smaller 'a' is the better fine texture will be preserved, but less noise will be removed. When bx/by are > 0 I've coded it so that the weights for the support window are all 1.0 for pixels up to the base window size and then it starts to drop off according to the 'a' parameter. So using bx/by > 0 is somewhat like using bx=by=0 with 'a' set to a larger value.

Quote:
The window is compared to other windows located on a rectangular grid defined by window width/height? Overlap or exhaustive search might yield significantly better matches, even if its range is shorter.
I assume you are talking about the base window and bx or by is greater than 0? In this case the frame is indeed split into blocks, but when comparing neighboring blocks to be averaged, the comparisons are done to all possible blocks of that size on the image inside of the search window not just to other bx/by blocks. Though in Beta 1 there was bug that causes it to miss 2*Bx+2*By blocks (increase ax/ay by the values for bx/by as a workaround).

Quote:
Use motion vectors to calculate rotations and other non-shift translations between frames, then apply before window search, could create much more and better same- and cross-frame matches.
It could, but it would make the current version look like the fastest filter ever .
tritical is offline   Reply With Quote
Old 23rd May 2006, 17:17   #20  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
Thanks for the reply!

I tried increasing the ax/ay parameters, they did increase the denoise, but only slightly from 3 to 6 at the expense of ax*ay time.

It was far more effective to increase the az parameter, but beyond 10 frames further noise decreases were also only slight.

It seems this filter is really good to start, I like its low memory usage. Some optimization could yield better speed without any quality loss, such as caching window-matches between frames and on the same frame. After all, it uses 1/4 the memory of fft3d, so it can still bloat up significantly and remain quite usable.

I read all three papers, and was very impressed with the entropy-inverse post-denoise image comparisons for various denoisers. It struck me as very excellent that the TNL-means algorithm left an almost random EI-PDI. This gives me hope that your implementation can improve to reach the quality level demonstrated in the paper. Specifically, they mention that noise removal doesn't harm fine detail, and from the nature of the EI-PDIs, I would assume that it can be the best possible denoiser. Since denoisers remove high frequencies more than low ones, there is no particular reason why they must be so hard on vague areas, which contain few high frequencies.

Last night I also tested Neat Image at medium and aggressive noise-reduction settings on the two images above. Check the added links in the previous post. The aggressive settings was indistinguishable from the TNL-means version, and took about 1/8th the time to generate. Neat Image uses a highly optimized wavelet transform to remove noise, and has been independently rated as the best still-image denoiser.

Last edited by Isochroma; 23rd May 2006 at 18:44.
Isochroma is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.