Smooth in time and space?

[Marc FD]

Whow,Whow,Whow !!!
NO RUSH !!!
IT'S NOT A MOTION COMPENSATED FILTER !!!!
IT'S A MOTION ADAPTIVE FILTER !!!!
it would not compensate motion it would adapts to
it !! Nothing to do with ME in codecs like XviD !!!!
And my filter is far from doing amazing results !!
I'm afraid you're overestimating this flter

Currently it just look a _bit_ better in fast scenes
than ie tempsoften2 because it wouldn'nt smooth
what's moving. that's all

[Koepi]

Erm, sorry that's what I meant

I think when you can detect motion (maybe the direction?) you can use that information to smooth "corresponding" regions. But I think it's easier to just detect motion areas and then apply the spatial soften, as proposed. Well, take your imte, I didn't want to rush things, I wanted to test it

Ok, back to add automaisation support to OggMux...

regards,
Koepi

[Marc FD]

Marc, how about first finishing the temporal smoother stuff until it works as you want it to and then you look into the sources for spatial filters and adopt that code? (should be the same code as the temporal smoother, just not in the z (time-)dimension, but in the x and y directions of a single picture, or am I mistaken here?).

In fact,like i said, i don't like to copy very used algos.
Peoples who have coded them are better than me for that.
I want to try a new way of spatial filtering (and maybe it wouldn't work...)
MAD is not a motion compensated filter :
It's an "hybrid" denoiser :
- In movement areas, he would use spatial filtering, because it leads to better results when there is motion, and if there is blending, it wouldn't be seen.
- In still areas, he would use very aggressive temporal filtering, because there is no ghosting, it wouldn't be seen too.

The idea is just to mix the advantages of spatial and temporal filtering.

@Koepi
Sorry, but i'm afraid you will be VERY disappointed if you think my filter will make miracles (i would create an MoComp filter if you want when i would be older.. let's say 18 Ok after all, i've writed my first C lines in the beginning of THIS month)

PS : when i say temporal/spatial filtering, it's not like a temporal/spatial smoother. i try to destroy High-frequency noise, not to blend low-freq noise.

PS@Koepi : If you have got an heavy chroma noised analog, you should try cnr2. I think would work much better than MAD on chroma denoising.

Back to work now. will try to create a pattern detection spatial noise killer... hope it would work better than normal Spatialsmoothers

[bb]

First testing results (I need to do a lot more, of course):

Convolution3D: Still needs some tweaking, but it seems to be superior to a pure spatial or temporal filter. It's slow indeed, but I had an optimizing approach in mind which I didn't want to keep by myself:
What if you cache three frames and interleave them like 3 lines of 1st frame, then 3 lines of second frame, then 3 lines of third frame, then next 3 lines of first frame again, etc. Then you apply the filter; in the end you have to de-interleave again. I think this could greatly improve processor cache hits.

MAD: I had to guess what the parameters mean, so I tested the proposed "Normal", "Hard", and "Soft" settings. "Normal" and "Hard" seemed to be unusable to me because of way too much ghosting. "Soft" simply didn't get rid of the noise - I'm afraid there's still a lot tweaking to be done...
Marc FD, could you please post the meaning of your filter's parameters?

bb

[vlad59]

@bb

Thanks for testing. I think I'll release a new beta today (3 times faster and exactly the same quality), just little bug to track and it will be good.

Till now I've just added some SSE optimizations but no buffering at all, I'll try to buffer when the main algo will be finished (I don't expect a big speedup if I want to keep quality, but I may be wrong).
I'll try to think about your idea tomorrow, but you have to keep in mind that the slowest part of this algo is the way I use the treshold and that's also prevent me from doing to much buffer optimization.
Let's explain with a sample :

I'm computing a Luma sample = 100 and my luma treshold is 10.
My original values are :
Prec frame :     Cur frame :   Next frame :
101 101 100      090 101 100   189 190 80
099 100 100      010 100 100   190 200 80
090 105 100      010 010 100   140 201 80

I use my treshold to detect edges and scene change and :
Prec frame :     Cur frame :   Next frame :
101 101 100      090 101 100   100 100 100
099 100 100      100 100 100   100 100 100
090 105 100      100 100 100   100 100 100

Next I calulate the weight of each frame :
Prec frame :     Cur frame :   Next frame :
1601             1592          1600

Now the new luma value = (1601 + 2* 1592 + 1600) / 64 = 100 
New Cur frame (no change) :
090 101 100
010 100 100
010 010 100

No change with this sample

With this sample you see another pb :
The next frame must be a scene change so for my calculation I change all its values to 100.
Advantage : my convoluted luma value won't become 130 so no ghosting
Drawback : my convoluted luma value has a high probability to keep its original value.

I thought of adding a SAD to check for this case but it will slow again the filter.

What do you think about it

[Ivion]

Ok, also here are my first result using the MAD filter.

Tested on: Rurouni Kenshin Tsuiokuhen OVA 2 (I choose this one cuz RK [Rurouni Kenshin] is usually very noisy and I like this OVA )
Results: Normal And Soft were both not capable of removing all of the noise, altough Normal did a 'decent' job, tough 2D Cleaner by Jaan Kalda was better. Hard didn't work to well, for there was alot of ghosting, and with alot, I mean alot!
Like: there was a scene of two people talking and one moment the 'camera' (it's anime, so there isn't really a camera ) was pointed at the wall, and the second moment it pointed to one of the faces and then there were still some 'pieces' of the wall on the image.
So I think some tweaking is gonna be needed, good luck Marc FD!

I currently can't provide you guys with screenshot for it's now encoding and the result that's posted here is based on some still images of the avs script.

[bb]

@vlad59:
Unfortunately I haven't programmed filters (yet). Therefore I'm not familiar with all the MMX and SSE jargon (what's SAD?). I was just thinking that you need to process a lot of pixels from different frames in a way which is _very_ awful for a cache. Thus the reordering of the frame lines could be a huge advantage.

I have to think about your way of processing. Just a (maybe stupid) question: How exactly do you calculate the weight?

bb

[Koepi]

Hm.... i have some versions to offer:

Search And Destroy (hehe, sometimes it really does!)
See At Definition )I like that one)

or simply

Summation of Absolute Differences.

Regards,
Koepi

[vlad59]

Originally posted by Koepi
Hm.... i have some versions to offer:

Search And Destroy (hehe, sometimes it really does!)
See At Definition )I like that one)

or simply

Summation of Absolute Differences.

Regards,
Koepi

I prefer Search And Destroy

With my previous example :
100 is my computed value.
200 is the same value but a frame later

Absolute difference is abs (100 - 200) = 100
if this AD > Treshold (10 in my sample)
then I don't use the next frame at all (more speed and maybe more quality)

So the new value is (1601 + 2* 1592) / 48.

Back to code ......

[bb]

Sounds like my question was way too stupid...
(You still didn't explain how you calculate the 1601, but never mind, I know by now that your weight matrix was
1 2 1
2 4 2
1 2 1
)

bb

[vlad59]

Originally posted by bb
Sounds like my question was way too stupid...
(You still didn't explain how you calculate the 1601, but never mind, I know by now that your weight matrix was
1 2 1
2 4 2
1 2 1
)

bb

Oooops sorry, it's Koepi's fault , he make me laught so I only answered to your first question, sorry again.

As I always say, the only stupid questions are those you don't ask.

[dividee]

I just wanted to tell you that I have a SceneChangeDetector class ready for avisynth. It's not a filter, just an internal class, but it can easily be ripped from avisynth. It's not a very evoluted SCD, it just compares the current frame and the previous one.
In pseudo-code:

return (% of pixels where |p(n)-p(n-1)| > tolerance) > threshold;

That is, two pixels are considered different if their absolute difference is above 'tolerance'. If the total amount of different pixels is above 'treshold', a scene change is detected and it returns true.
It works in YUY2 and RGB32, is MMX optimized and quite fast. In YUY2, you have the option to only compare luma if you want.
I thought it would be better to implement it as a separate class, so any filter could use it and it can be tweaked independently.
Example:

In you constructor:
scd = new SceneChangeDector(child, 40   , 20, false     , env);
//                          clip  thresh tol  use_chroma
In GetFrame:
scd->detect(n, env)
returns true if frame n is a scene change, else false
you can also call it like that:
scd->detect(frame, n, env)
if you already have a pointer to the current frame.

I think you just need to replace #include "internal.h" by #include "avisynth.h" to use it in a plugin.

I'll test your idea of using a SAD for scene change detection.

[vlad59]

@Dividee
I'll test it tomorrow (If I have time)

@All
I have a new version of the convolution3D for you to test. Please read the text file included.

The first alpha was purely mathematical so it was full quality. But with this alpha2, my goal was speed, I hope there is no quality issue.

Now the problems I have are :
- on my test vob there is always a little noise around the edges (typically the heads of the caracters)
- With my current algo a white spot surrounded by black ones will remain white (due to Treshold)
- To have better results it could be interesting to lower the weight of the center pixel (just an idea).
- Speed

@Moderators
I'll try to put this file in a website to avoid using too much bandwidth. But I just hate HTML

EDIT : Removed attachment, newer version available

[Marc FD]

@Dividee
LOL. Don't need it at all i have an advanced motion detector already for CopySame.
I'm really sorry (because i didn't know you were working on it)

@All
Okay, to business : i've tried 4 differents algos today and each time it wasn't good enough but finally i had THE idea

Here it is : MAM - Motion Adaptive Mixer +
What it does : Mix a spatial filtered clip with a temporal filtered clip : spatial in motions areas, and temporal in still area :

The goal is to avoid the defaults of spatial and temporal filtering:
spatial filtering blurs when too aggressive.
temporal filtering "ghosts" when too aggressive.

Using MAM, you can _theorically_ use very aggressives settings because :
- you won't see the blurring of the spatial filtering, because it's moving.
- you won't see the ghosting of the temporal filtering, because it's not moving.

And it is compatible with all the currents and coming spatial and temporal filters

To download it, just use the link for my dl site.

(please) TEST IT ! NOW !

PS : We should rename this thread to : The slow-as-hell smooth in time and/or space
(because using _both_ temporal and spatial filtering is not really a way to improve speed...)

[Ivion]

Ya know, I'dd really like to test it Marc FD, if I could get it.
Cuz there's no link on your website and http://ziquash.chez.tiscali.fr/mad.zip doesn't work, at least not for me. Could you please upload it, I wanna test it on my killer Kenshin clip.

[Marc FD]

OH SHIT !!!
I worked hard, and finally i forget to upload
too dumb.
thanks Ivion
and it's not MAD but MAM : Motion Adaptive Mixer
I'd really like feedback on very noisy sources or for HQ

It's really powerfull. I'll use it for all my encodes now

If you wanna see the better denoising achived in avisynth , just test it

[MoonWalker]

I tried to make a quick test of MAM but i couldn't

"Avisynt read error : Avisynth : caught an access violation at 0x01161480, attempting to read from 0x044bb000"

I may be doing something wrong ,cause I am a little sleepy right now..

Here is my avs :

LoadPlugin("D:\DivX\GORDIA~1\mpeg2dec.dll")
LoadPlugin("J:\Cnr2.dll")
LoadPlugin("J:\TemporalSoften2.dll")
LoadPlugin("J:\MAM.dll")
LoadPlugin("D:\DivX\GORDIA~1\Avisynth_Spatial.dll")

mpeg2source("J:\Matrix\matrix.d2v")
crop(0,79,718,418)

Temporal=TemporalSoften2(1,4,8).Cnr2()
Spatial=Temporal.SpatialSoftenMMX(1,4,8,false,false,4,4,6,8)
MAM(Temporal,Spatial,4,8,0)

I know there isn't a resizer..I want to see in full size what the filter can do

MoonWalker

[Ivion]

Hummm, it seems that I got the same error, or maybe slighty different. Tough the outcome was the same, we couldn't test the filter. Here's my avs script, generated with the help of gknot (for the auto crop ^.^):

LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\decomb.dll")
LoadPlugin("C:\filters\cnr2.dll")
LoadPlugin("C:\filters\smoothhiq.dll")
LoadPlugin("C:\filters\mam.dll")

mpeg2source("C:\tempdvd\test.d2v")

FieldDeinterlace()

crop(11,5,699,568)

s=smoothhiq(7,25,35,192,15)
t=cnr2()
MAM(t,s,2,4,8)

For MAM I just used the default values, so I could see what the impact of the standard settings were. CNR2 is also default as you can see, and for SSHiQ I use the values I normally use on this DVD without denoising. My plan was to see what the outcome of these values was and 'tweak' them, for I believe my SSHiQ settings are too high.
2bad I couldn't test it, plz look into it Marc FD, for this filter -sounds- promising.

[Marc FD]

Strange.. maybe i've made a dumb error (like always)
I will look into it now and test it on 3-4 different sources.
i will release a new version in less than 1 hour

@MoonWalker : i would never use SpatialSmootherMMX, it will produce ghosting because there is a temporal filter in it ....
Use sshq or spatialsoften instead
EDIT temporal.ssMMX... ARGH you want to kill me. You will spoil everything like this !!!
temporal _should_ be only temporal and spatial _only_ spatial !!!!

Okay let's WORK !!
I'm back in 40-60 min

[hakko504]

Just a thought, but neither Ivion nor MoonWalker is using mod4 resolutions, could this be a source for problems?