Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th July 2002, 08:10   #1  |  Link
bb
Moderator
 
bb's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 2,665
Smooth in time and space?

Well, I don't really know if there is already a filter available covering this idea:

The smoothing/denoising filters I know are either spatial or temporal filters. What about smoothing spatially AND temporally at the same time?

E.g. you could average a 3x3x3 block of 3 consecutive frames, i.e. the middle pixel would be averaged by its neighbours in the same frame and the neighbour pixels of the neighbour frames in the time dimension.

I hope the explanation wasn't too complicated to understand, but I think using existing averaging algorithms on a three-dimensional cube instead of just looking at a two-dimensional plane would not be too complicated to implement.

What do you think?

bb
bb is offline   Reply With Quote
Old 25th July 2002, 08:41   #2  |  Link
Acaila
Retired
 
Acaila's Avatar
 
Join Date: Jan 2002
Location: Netherlands
Posts: 1,529
AFAIK that is exactly what the General Convolution 3D filter does. However it's made for VDub, not Avisynth.

Something like this can easily be achieve by just chaining two filters, but the problem is that they don't work intelligently to determine edges and sharp lines etc.
Acaila is offline   Reply With Quote
Old 25th July 2002, 09:45   #3  |  Link
poptones
Registered User
 
Join Date: Jun 2002
Posts: 135
AVISynth has generalconvolution. Richard added it himself to the last couple releases. And you can chain them pretty easily, as well as add as much intelligence as you like. In addition to using edge detection filters to create masks, you can use lighten and darken via layer().

There are so many ways to do this now it's become difficult to decide which one is best for an application. There are the included filters and the plugins - between those and being able to layer clips using a variety of operations you have pretty much unlimited possibilities. My biggest gripe right now is with finding a way to remove MPEG artifacts without screwing things up too badly. And I'm working on that one, too.
poptones is offline   Reply With Quote
Old 25th July 2002, 10:57   #4  |  Link
bb
Moderator
 
bb's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 2,665
Hmm, the 3D averaging I was thinking of is not the same as chaining a spatial and a temporal filter. But the General Convolution 3D filter seem to be the solution (don't know if it has scene change detection, which would be nice). I'll try that one.

Is there a General Convolution 3D filter available for AviSynth, too?

If not, maybe someone feels challenged to implement it. There's also much room for tweaking, like edge detection, etc.

I think this could be a superior method of getting rid of noise in complicated sources like DV.

bb
bb is offline   Reply With Quote
Old 25th July 2002, 11:17   #5  |  Link
avih
Capture, Deinterlace
 
avih's Avatar
 
Join Date: Feb 2002
Location: Right there
Posts: 1,964
i think a specific 3d convolution filter should be written since it would be much faster, especially since it using 27 different values. also, it should be adaptive.

it can be adaptive in 2 ways:
1. apply the convolution result if it's not 'further' than 'x' from the original pixel value.

2. take into account only pixels that are not 'further' in value from the priginal pixel by 'x'

the 2nd method gives better results imo, but is considerably slower because of the 'if's during the convolution process itself.

it SHOULD be a great filter though.
avih is offline   Reply With Quote
Old 25th July 2002, 12:43   #6  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Dreaming of SmartSmootherHiQ in 3D ...
Didée is offline   Reply With Quote
Old 25th July 2002, 15:52   #7  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,266
And then instead of taking twice the time to encode, it can take four times the amount.
manono is offline   Reply With Quote
Old 25th July 2002, 16:56   #8  |  Link
sh0dan
Retired AviSynth Dev ;)
 
sh0dan's Avatar
 
Join Date: Nov 2001
Location: Dark Side of the Moon
Posts: 3,480
Not entirely so, because the temporal check would not be a complete 3D-check, it would only check the current pixel against the same pixel in the previous frame (one check more per pixel).

The idea is quite neat - the algorithm could work like this:
*Calculate the blurred pixel as usual.
*Compare the original pixel to the previous frame. The bigger the difference, the less will the pixel get blurred.

That will make low-freqency noise even easier to detect.

Might actually work!

I've also recived a great suggestion for adaptive threshold, that adapts the threshold to the luma value of the current pixel - so there are lots of improvements in the works. First priority is still an MMX-optimized YUV version though.
__________________
Regards, sh0dan // VoxPod
sh0dan is offline   Reply With Quote
Old 25th July 2002, 19:31   #9  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,266
Hi Klaus-

SSHiQ is far and away the best spatial (and someday spatial-temporal?) smoother out there, and I use it frequently. Please don't get me wrong-I understand what you're saying and was just joking around. Much respect and thanks is due to you and your program, and the optimized version is eagerly awaited.
manono is offline   Reply With Quote
Old 25th July 2002, 19:39   #10  |  Link
sh0dan
Retired AviSynth Dev ;)
 
sh0dan's Avatar
 
Join Date: Nov 2001
Location: Dark Side of the Moon
Posts: 3,480
If I get the time, I'll try to throw together a test-version in C - I've been thinking about using temporal data for some time, and this just triggered a good idea.

Maybe I should do it tonight, instead of watching a movie
__________________
Regards, sh0dan // VoxPod
sh0dan is offline   Reply With Quote
Old 25th July 2002, 20:36   #11  |  Link
poptones
Registered User
 
Join Date: Jun 2002
Posts: 135
I don't get it. Both multiplication (convolution) and addition (averaging) are associative and commutative. You can specify thresholds within layer or by making a mask from the original (unfiltered) source. 5+3 = 3+5 every time...

The problem with temporal filters is they are so doggone slow. It seems like every filter that adds another frame fetch from a file doubles the encode time.
poptones is offline   Reply With Quote
Old 25th July 2002, 21:09   #12  |  Link
dividee
Registered User
 
Join Date: Oct 2001
Location: Brussels
Posts: 358
Sorry I don't follow you. Adaptive operations are usually not commutative nor associative. If for example, the operator @ represents the following function:
Code:
a @ b = (a+b)/2  if |b-a| <= treshold
      = a        otherwise
then
a @ b != b @ a (not commutative)
a @ (b @ c) != (a @ b) @ c (not associative)

For instance, let a=10, b=25, c=15 and treshold=10:
a @ b = 10
b @ a = 25
a @ (b @ c) = 15
(a @ b) @ c = 12.5

Quote:
The problem with temporal filters is they are so doggone slow. It seems like every filter that adds another frame fetch from a file doubles the encode time.
Not true! temporal filters usually works in 1D while spatial filters usually in 2D (such as convolution), so temporal filters are usually faster. You just have to be careful with implementation of such filters, using some kind of buffering to avoid fetching too much frames at once and trashing the cache with a lot of parallel pointers.
__________________
dividee
dividee is offline   Reply With Quote
Old 25th July 2002, 21:30   #13  |  Link
Koepi
Moderator
 
Koepi's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 4,455
Isn't it possible to adopt such pixel averaging with toms TomsMoComp filter? I mean that would be the best filter ever.

Apply some spatial smoothing if someone wants that and look where the pixel went an average that out as well

Just to bring in this idea again...

Regards,
Koepi
Koepi is offline   Reply With Quote
Old 26th July 2002, 01:13   #14  |  Link
Marc FD
XviD fan
 
Marc FD's Avatar
 
Join Date: Jun 2002
Location: France
Posts: 907
Quote:
The problem with temporal filters is they are so doggone slow. It seems like every filter that adds another frame fetch from a file doubles the encode time.
I agree with dividee. cnr2 is real-time capable on my AthlonXP 1.4 Ghz
in 640x480. And if richard finish the BitBlt optimisations, it could go damn faster (40-60 fps).

Quote:
Isn't it possible to adopt such pixel averaging with toms TomsMoComp filter? I mean that would be the best filter ever.
Tom said is algo couldn't be used in a temporal filter

PS : I like fast filter and 3D-filter sounds like slow as hell for me.
Like koepi said, the best would be a MC filter, but i would be damn slow too. but will give a much bigger noise-removing effect.
(and would be much much harder to code )

Last edited by Marc FD; 26th July 2002 at 01:17.
Marc FD is offline   Reply With Quote
Old 27th July 2002, 15:12   #15  |  Link
vlad59
Vlad, the Buffy slayer
 
vlad59's Avatar
 
Join Date: Oct 2001
Location: France
Posts: 445
I just coded an avisynth filter that does the 3*3*3 average with a simple matrix till now :

Code:
Prec frame :     Cur frame :   Next frame :
1 2 1            2 4 2         1 2 1
2 4 2            4 8 4         2 4 2
1 2 1            2 4 2         1 2 1
I've also added a luma an chroma treshold like on the example of Dividee to keep edges and take care of scene change.

I work quite good but it's damn slow (in fact it's the slowest filter I've ever seen).

Next to do is MMX a little (the matrix is done for that) and add a little buffering.

If someone already want this filter (it's very very very slow, I warm you), I can post it throught this forum.
__________________
Vlad59
Convolution3D for avisynth 2.0X : http://www.hellninjacommando.com/con3d
Convolution3D for avisynth 2.5 : http://www.hellninjacommando.com/con3d/beta
vlad59 is offline   Reply With Quote
Old 27th July 2002, 15:32   #16  |  Link
Marc FD
XviD fan
 
Marc FD's Avatar
 
Join Date: Jun 2002
Location: France
Posts: 907
Slow, it was already know, but does it really worth it ??
I think that the real question.
I think people who chain several filters to gain quality like
sshq with tempsoften are far away from searching fast filters
Marc FD is offline   Reply With Quote
Old 27th July 2002, 15:35   #17  |  Link
avih
Capture, Deinterlace
 
avih's Avatar
 
Join Date: Feb 2002
Location: Right there
Posts: 1,964
Quote:
I've also added a luma an chroma treshold like on the example of Dividee to keep I've also added a luma an chroma treshold like on the example of Dividee to keep I've also added a luma an chroma treshold like on the example of Dividee to keep edges and take care of scene change.
how did u implement that? with respect to the final result only? or with respect to each of the 26 pixels?

it's also possible to provide different thresholds for the temporal vs 2d operations.

cheers for the work
avih is offline   Reply With Quote
Old 27th July 2002, 15:46   #18  |  Link
vlad59
Vlad, the Buffy slayer
 
vlad59's Avatar
 
Join Date: Oct 2001
Location: France
Posts: 445
@MarcFD

You can not imagine how slow it is ...........
I've attached a zip with the filter. it has only been tested on a full size vob with no other filter.
With a very noisy material, i use this :
Code:
convolution3D (8, 16)
first parameter : luma treshold
second parameter : chroma treshold
@avih

In fact, I've made a slow method (don't know if it's a good method).

For each pixel surrounding the center pixel :
if the absolute différence with the center pixel > Treshold
then the corresponding matrix value become 0 so I don't take this pixel into account.

This method will be hard to optimize so I'm corrently trying something else.
__________________
Vlad59
Convolution3D for avisynth 2.0X : http://www.hellninjacommando.com/con3d
Convolution3D for avisynth 2.5 : http://www.hellninjacommando.com/con3d/beta
vlad59 is offline   Reply With Quote
Old 27th July 2002, 15:59   #19  |  Link
avih
Capture, Deinterlace
 
avih's Avatar
 
Join Date: Feb 2002
Location: Right there
Posts: 1,964
yup. imo your method should produce very good results, but it's indeed hard to optimize.

i had really rough times trying to optimize this method on a 2d (3x3) deringing filter..

the other thresholding technique u can use is calc the convolution anyway, and apply it only if the final result is inside |thresh| from the original pixel. that's how deringing is usually implemented.

good luck anyway
avih is offline   Reply With Quote
Old 27th July 2002, 18:23   #20  |  Link
vlad59
Vlad, the Buffy slayer
 
vlad59's Avatar
 
Join Date: Oct 2001
Location: France
Posts: 445
Can a moderator say me if I have attached a zip file with my last post ????

Sorry for this 0-use post
__________________
Vlad59
Convolution3D for avisynth 2.0X : http://www.hellninjacommando.com/con3d
Convolution3D for avisynth 2.5 : http://www.hellninjacommando.com/con3d/beta
vlad59 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:28.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.