New very slow FFT denoiser

[leonid_makarovsky]

I have some test results with the latest version.

The original file is in Huffyuv AVI format, 704x576, PAL 25fps. The output clip had total of 114025 frames. The total time of the output clip is 76 minutes and 1 second.

I had the very latest FFT3dfilter from Fizick.

I loaded the following AVS script into VirtualDub and saved it as also Huffyuv AVI with the same resolution and the same frame rate.

It took 22 hours 39 minutes and approximately 50 seconds to process the whole thing. I don't know how helpful it is giving the fact that there was PixieDust and it just also takes a while to process Save As operation in VirtualDub.

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\2.0\LoadPluginEx2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\2.0\DustV5.dll")

clip = AVISource("L:\KreatorPAL.avi")
clip = AssumeTFF(clip)

clip = Trim(clip, 16, 21) + Trim(clip, 16, 114024) + blank_clip(clip, 114009, 114024)

clip = Crop(clip, 0, 0, -6, -10)
clip = AddBorders(clip, 0, 0, 6, 10)

clip = SeparateFields(clip)

clip = PixieDust(clip, limit = 4)
clip = fft3dfilter(clip, sigma = 3, bt = 3)
clip = Weave(clip)
clip = Trim(clip, 6, 114030)
return clip

function blank_clip(clip myclip, int mystart, int myfinish, int "mycolor")
{
thepart = Trim(myclip, mystart, myfinish)
mycolor = Default(mycolor, $000000)
blankout = BlankClip(thepart, length = Framecount(thepart), pixel_type="YUY2", color = mycolor)
return blankout
}

[namor82]

Hello,

I have used the fft3d plugin a lot, and I liked especially the sharpenthing, because it was very good (for my subjectiv opinion)...
But the new version 0.9.3 is working not clean I guess, because i got vertical chroma-blocks in my capture, when I use version 0.9.3 .
It is not only because of the sharpen, but the sharpen does not work anyway, anymore (ok it's perhaps, because u dropped it to horizontal only)...

I had InterlacedSource, so I used:

SeparateFields()
e = SelectEven().FFT3DFilter(sigma=1.8, plane=0, bt=3, bh=32 ,bw=32 ,sharpen=1.7)
o = SelectOdd().FFT3DFilter(sigma=1.8, plane=0, bt=3, bh=32 ,bw=32 ,sharpen=1.7)
Interleave(e,o).Weave()

Here the Link, for the Images...
http://home.arcor.de/kwangsup/fft3d.rar

Has s.o. tried out the sharpen-functions ? Have you such bad results with the new 0.9.3 Version too ?

[Fizick]

namor82,
I know about very sharpen problems.
But i do not know a solution right now. And has no time. I must learn about sharpening.
These columns is due to not sufficient overlapping.
As a workaround, increase overlapped width to ow=bw/2
And i never say, that sharpen=1.7 is good value :)

The fft3flilter development is not stopped,
but delayed a little (week ?).
Any suggestions are welcomed.

[Soulhunter]

Isnt there some way to produce this grid pattern without real filtering ???

Then you could subtract it from the sharpened/smoothed result... :\


Bye

[Fizick]

I try put out the very first frequency member sharpening. So the column disappear, but strictly vertical lines will not sharpen too.
And i want add second sharpen parameter for verical sharpening.

[acrespo]

And I like to blend DeGrainMedian with FFT3D... :D


Bye

n00b question:
How can I do this (blend filters)?

Some people at this forum have writing that FFT is for real videos, not for anime. Is it true? I tried the combination of FFT3D(sigma=2,plane=0) and RemoveDirt(mode=16) << removedirt script. The results are good, but removedirt remove more noise than FFT. I tried sigma=3 but removedirt still better.
Other thing I have doubts is in apply only sharpen, only line darken or both.
The source is Samurai X TV.

[DarkFoon]

@acrespo
This filter is not only for live-action, I use it for anime often.
It is good at removing noise in certain situations in anime, but some kinds of noise and some kinds of anime aren't helped by the filter except with a high sigma (5-8) but that produces ghosting.
I have never used removedirt, so I don't know how it compares. I usually pair FFT3d with msmooth, but I am finding more and more that FFT3d on its own is better.

[SirCanealot]

@acrespo
This filter is not only for live-action, I use it for anime often.


It doesn't add its own grain/noise for you?
That's basically why I stopped using it ^^;; — what's the point of using it if I have to smooth after to compensate for FFT adding filesize to random frames...
This is especially the case with clean anime. When there's not much noise, whatever I do, FFT will add noise, heh...

[Soulhunter]

How can I do this (blend filters)?

Source=DirectShowSource("C:\Stuff\Blah.avi")

A=Source.FFT3DFilter(Sigma=1)
B=Source.Deen("a3d",3,3,3,25)
Overlay(A,B,Opacity=0.5)


Bye

[DarkFoon]

@SirCanealot
Actually, it doesn't add noise to my images. I usually work with TV anime captures and my capture card adds quite a bit of noise (which pisses me off so much, because it has hardware noise reducers, but the software doesn't use them)
On clean anime, I probably wouldn't use this filter, or if I did, I would use a low sigma. I have found what I consider a great way to use this filter, although it slows it down. I set my ow/oh = to 0.5*bw/bh. So if my bw/bh = 32, my ow/oh = 16. This way, I can use rather low sigmas, and remove quite a bit of noise, so I don't have to run the filter again.
SirCanealot it would be helpful if you posted a single frame of some of your anime (unprocessed) so I can better understand what you are working with. Also, I'd find it helpful if you could post the same frame, only processed with FFT3d so I can see the noise that it adds.
thanks

[acrespo]

I tried FFT3D in my sources (TV anime captures too), but I need a very high sigma (8-10) value to clean the image. When I apply FFT3D to all planes I need less sigma (3-5), but I always need other smooth (usualy spatial) to complete the job.

Can be implement a plane option to link all planes, or to link chroma planes in one plane, like that:

plane = 0 -> Y plane
plane = 1 -> U plane
plane = 2 -> V plane
plane = 3 -> UV plane
plane = 4 -> YUV plane

Then, I don't need to execute the filter three times to clean all planes.

[DarkFoon]

I asked Fizick for that a long time ago, but he said its just easier to filter each plane with a different instance.
There's no difference in either way. One way is more work for Fizick, and the other is more work for the user (but not really, all they have to do is copy/paste the filter line 3 times)

[acrespo]

I asked Fizick for that a long time ago, but he said its just easier to filter each plane with a different instance.
There's no difference in either way. One way is more work for Fizick, and the other is more work for the user (but not really, all they have to do is copy/paste the filter line 3 times)

But I guess that fft will run more fast if I don't need to start 3 instances. Only Fizick can confirm that of course.

[Fizick]

Released fft3dfilter Version 1.0 - June 27, 2005 - improved edges processing (by padding);
added svr parameter to control vertical sharpening

[trevlac]

@Fizick,

My comment is not really well thought out .... but that has not stopped me lately .. :D

With regard to filtering all channels faster ....

Can you implement UV as a single complex to complex DFT? U & V represent the vector that describes the chroma. I'm not really sure what this would mean to apply a filter. Maybe I'll give it a try and report back. :)

[leonid_makarovsky]

Released fft3dfilter Version 1.0 - June 27, 2005 - improved edges processing (by padding);
added svr parameter to control vertical sharpening

Hi Fizick.

I tried your new version yesterday. I found the couple of problems. The horizontal sharpness suffers from the same problem as in the previous version. There're vertical lines. The higher the sharpness parameter the more visible they are. I also found a bug. If you set sharpness parameter to 0, then no matter how high you set svr parameter, the vertical sharpness won't take place.

--Leonid

[Fizick]

Leonid,
these all are not bugs, but features of method used.
You may set "sharpen" to some small value =0.1 for example.

[DarkFoon]

Uh.. this may sound kind of stupid, but what was wrong with the sharpening algorithm in the first place? I never had any issues using the sharpening back in the versions before fizick first disabled it.
If I set it really high and only processed the Y plane, some black scenes with white text would have this neato "etching" effect coming off the edges. But, I work with anime almost exclusively, so maybe that's why I never had a problem.

[Fizick]

DarkFoon,
You say to me or to Leonid? :)
How about the last version?

[leonid_makarovsky]

Leonid,
these all are not bugs, but features of method used.
You may set "sharpen" to some small value =0.1 for example.

I am not critisizing. I'm giving the feedback. The features are kind of strange. Yes, if I use very small value like 0.1, I don't see the vertical lines. But it doesn't sharp the image either. So there's a proportional dependency between vertical lines (artifacts) and sharpening.

I turned the sharpening off to process my video clip.

--Leonid

[Fizick]

Leonid,
I want just to say, that if you set sharpen=0.1, then you may use svr=10 to get vertical sharpening only (if you want).

And i repeat, that sharpening is still experimental and not a super-puper - I included it to my filter simply because of it was easy for me.

fft3dfilter is not best ever filter - it is a filter what used certain algorithm (fft).
And i do not force anybody use it or its sharpen mode.

[cwk]

Hello Fizick,

Thanks for this great plug-in and many others. I frequently clean up old VHS sources with a lot of noise and this plug-in is indispensable.

When you say that edge processing has been improved in the latest version, do you mean edges of images within a frame, or the edge of the frame itself (borders)??

[leonid_makarovsky]

fft3dfilter is not best ever filter - it is a filter what used certain algorithm (fft).

I find your fft3d filter the best available for cleaning up VHS noise if used with PixieDust. Also wanted to thank you. And my commentary was a feedback on sharpening issues that I thought were bugs.

--Leonid

[Pookie]

Fizick - I wanted to thank you as well. FFT3D is a great denoiser. :)

[Fizick]

cwk,
Frame borders processing is improved, by padding some mirrored pixels to all borders,
and cropping them afrer processing.
Old versions changed (darken) some pixels near borders.

Leonid and all,
I am glad to get any critique,
but i will respond to it :)

[Fizick]

Version 1.0.1 - July 05, 2005 - fixed bug for chroma planes for YUY2 (introduced in v.1.0)

[Fizick]

OK, I create new version 1.1 - impoved sharpening method.
Now it is free of grid artefactes and similar to LimitedSharpen script (but is better a little IMHO :) )
Try for example pure limited sharpener:
fft3dfilter(bt=-1, sharpen=0.7)

[Soulhunter]

Well, in scenes with much texture detail it works very well !!!

But on flat gradients there is still a visible grid pattern... :\

Source:
http://img281.imageshack.us/img281/2888/1844source0jt.png

FFT3DFilter(BT=3, Sigma=1, Sharpen=1)
http://img281.imageshack.us/img281/9806/18448pi.png

Source:
http://img281.imageshack.us/img281/7716/2533source0gw.png

FFT3DFilter(BT=3, Sigma=1, Sharpen=1)
http://img281.imageshack.us/img281/8382/25334yu.png

Bye

[Pookie]

I looked at Soulhunter's stills on several PCs at work - some of the monitors show the grids, others don't.

[DarkFoon]

@soulhunter
I think I have an idea on how to lessen the grid artifacts (but you probably won't like it much)
try settings oh/ow = half the bh/bw.
this isn't all that special, and it slows down the processing (but I find it also produces better looking images) however, I think it might lessen the grid.
I'd appreciate a test of this theory using the same stills as above, if you have the time. (I lack the processing power, time, and sources to run the test(s) myself)

[Soulhunter]

@ DarkFoon

Yes, you are right... ^^

And it seems that "OX = BX / 3" is already enough !!!

FFT3DFilter(OH=16,OW=16,BH=48,BW=48,BT=3,Sigma=1,Sharpen=1)
http://img259.imageshack.us/img259/8555/18441648xxx2fx.png

FFT3DFilter(OH=16,OW=16,BH=48,BW=48,BT=3,Sigma=1,Sharpen=1)
http://img259.imageshack.us/img259/426/25331648xxx2go.png

Bye

[DarkFoon]

glad i could be of help ^^
although, i can still detect some gridding, but its small and insignificant.

[Fizick]

I will try make third attempt to make artifact-free sharpener later.

[Soulhunter]

glad i could be of help ^^
Well, thanks... ^^




although, i can still detect some gridding, but its small and insignificant.
True, on my 19" Syncmaster I can still spot the grid pattern...

But considering that "sharpen=1" is the max recommended value, its venial!


Bye

[woutertje]

I'm getting the following error:
Plugin fftw3.dll is not an Avisynth 2.5 plugin
I'm using AviSynth 2.55

[Fizick]

woutertje,
re-read manual. Do not try load fftw3.dll.

Soulhunter,
I think your settings (OH=16,OW=16,BH=48,BW=48) is the optimal,
i will set them as default in next version.

[cwk]

woutertje,

I had this problem too. fftw3.dll is not the plug-in, but is required for FFT3DFilter to work correctly. It must be placed in a directory specified by your path variable. I am running XP, put fftw3.dll in my c:\windows directory and it works. fft3dfilter.dll must go in the plug-ins directory.

ck

[DarkFoon]

@ Fizick
the bw/h of 48 is a better divisor into the standard DVD/TV resolution of 720x480 (or 720x576 in PAL-land, but I may be wrong...)
I often changed it to 44, or 40 because those divide nicely into the above resolution.
Anyways, a change for the good, I say!

[Fizick]

44=2*2*11 is not very good number, the speed will be slow.

[DarkFoon]

Actually, I think I stopped using 44 after you told me about that stuff. I still remember the number because I used it for a long time before that.

[Fizick]

Version 1.2 - July 12, 2005 - changed default block sizes bw=bh=48 and overlap sizes ow=bw/3, oh=bh/3 to finally prevent grid artifactes in sharpen mode (but speed is decreased).

[ChronoCross]

Feature Request: plane=3 Clean all 3 planes YUV instead of having to run 3 instances of the filter. Thanks

[Fizick]

This feature was already rejected.

But soon i will implement interlaced flag.

[DarkFoon]

This feature was already rejected.

But soon i will implement interlaced flag.

Do you mean that it will properly work on interlaced sources, instead of having to de-interlace before? Or is it something different?

So far, I've been using the "ApplyInterlacedFilter" function on my interlaced clips, but it slows processing down considerably. Would this "interlaced flag" make it so I don't need to use that anymore? If so, that's going to be great!

[Fizick]

Version 1.3 - July 20, 2005 - added interlaced parameter for interlaced video processing.
Now I use simple "unfold - fold" method. I flip second field (up-down) and put it to bottom of first field. Then process this combined "frame" (FFT, filter, iFFT).
Then reverted fold.
So, some small artifactes are possible near bottom.
But speed is not slowed.
Try!

[cwk]

Hello Fizick,

I am getting a problem with the interlaced mode in 1.3. One of the fields appears to be vertically resized incorrectly when they are getting weaved back together.

My source is YUY2, Huffyuv, interlaced. Here is my script:

AVISource("E:\DV\Clips\plain.avi")
FFT3DFilter(sigma=1, sharpen=1, interlaced=true)

Attached images (original.jpg) (After_FFT13.jpg)

[DarkFoon]

looks to me like the the Y plane is being imporperly processed. notice how the messed up part is gray. .... on second thought, maybe its the chroma planes that are being processed wrong. either way, it has to do with colospace. try it with YV12, and see if that changes anything.
if not, just wait a while, fizick will probably have it fixed within a few days.

[cwk]

Yes, converting to YV12 fixed this problem. Thank you DarkFoon.

It appears that the interlaced option currently has difficulty with YUY2 material.

Here is the new script:

AVISource("E:\DV\Clips\plain.avi")
ConvertToYV12(interlaced=true)
FFT3DFilter(sigma=1, sharpen=1, interlaced=true)

[Fizick]

cwk,
thanks for fast report.
Try new fixed version 1.3.1

[cwk]

Fizick,

That did it! Thank you for the fast fix.

cwk