New Filters: Conditional Temporal Median Filter and DeSpot filter

[krieger2005]

I have problems with this Filter on "higher" motion in movies. I used

Despot(p1=25,p2=10,pwidth=60,pheight=60,mthres=12,ranked=true,dilate=1, sign=2)

and when there is high motion there the filter produce ugly big artefacts. When i increase mthres (e.g. mthres=17) it seems, that the artifact stay the same, when i lowering the threshold the effect is the same.

Summerized for me is it difficult to decide which factor p1,p2 or mthres should be tuned to get right results. As you can imagine i does not understand the algorithm of this filter so good, so maybe you can get give some points, how to tune it.

-- EDIT
I think the filter should get better motionestimation (not best but better) because i think one must set p1 high und p2 low to reduce artefacts. Because i think a simple difference betweet two pixels is too bad to decide, if it is a spot or motion.

greets krieger2005

[Fizick]

For high motions, you may increase Mwidth, Mheight parameters, and try tune Merode.

But of course, motion compensation is the better. Did you try Depan or MVtools ?

New version Despot 2.1 (June 14, 2004) is released:
Add parameter "mratio" parameter (it was internal =3 in all previous versions).
Change default of "merode" to dependent from "mratio".
Update doc.

At last I begin to understand how the filter works at motion denoising stage

3) Find moving areas of an image by simply comparing each
pixel to the previous frame and considering all those which
are greater than MTHRES.
3b) Mark motion pixels without noise with weight "mratio" in the motion map.
Mark pixels determined both noise and motion as weight 1 in the motion map.
4) Denoise the motion map by erosion and then dilating
(as a whole it is morphological opening operation).
During erode phase, the motion map is eroded with range mwidth/2 and mheight/2,and zones with small summary weight (less than "mratio*merode/100") are decreased or completely removed from motion map. Such zones correspond to small relative (in percent) numbers of motion neighbors within this range or mostly noisy pixels (spots).

During dilate phase, remained motion zones are enlarged with same range.

[krieger2005]

I try first MVTools. But because the source is very noisy MVTools generate a "block-theater", where block were shift down, up ...
But in some scenes does the MVTool "something", so that DeSpot does less artifacts.

I tried also DePan:

i=last
d=depanestimate(range=5, trust=100)
x=depan(data=d)
return StackHorizontal(x,
               \ subtract(x,i).ColorYUV(autogain=true))

The Result was, that i was the same as x. So that DePan has done nothing...

For higher motion-scenes u use:

norm=ConditionalFilter(motion,des,"abs((YDifferenceFromPrevious+YDifferenceToNext)/AverageLuma*100)",">","19")
ConditionalFilter(highmotion,norm,"abs((YDifferenceFromPrevious+YDifferenceToNext)/AverageLuma*100)",">","35")

where des the Result of DeSpot is, motion a blured movie

btw, thanks for the tip with Mwidth, Mheight. I forgot these parameters....

[Fizick]

With Depan, you must use or two Depan(), or DepanInterleave, to compensate global motion for Despot.
Simplest script:

avisource("qwerty.avi")
i=ConverttoYV12()
mdata=DePanEstimate(i)
DePanInterleave(i,data=mdata)
Despot(some parameters)
selectevery(3,1)

BTW, now I try implement some new method of motion detect in DeSpot.

[Fizick]

Changes new DeSpot v.3.0 (June 22, 2004) from 2.1:
Version 3.0 is major release update (probably alpha with bugs):
Add another motion detection method "motpn" (previous to next frame).
Add whole segments removing methods "seg"=1,2. - more safe.
Change luma correction to local in segments (spots) removing mode. Change some defaults.
Code reorganization.

http://bag.hotmail.ru

If new modes will be good in tests, I will remove some old modes in next version.

[tedkunich]

Fizick,

Have been trying out your filter in restoring some captures off of a laserdisc that is experiencing "laser rot". Your filter seems to knock out the white and black spots nicely, but it does not seem to detect/eliminate any chroma information from the removed spots. I have white, red, blue, and green dots - the white ones are gone, but the others have the intensity reduced, but the chroma flair is still visible. Any thoughts, tricks???

I'm using version 3.0 or your filter.


Thanks

Ted

[Fizick]

I think, indedendent color spots detecting may produce false detection with ugly color artefactes.
Probably removing color spot only at places of detected luma spots may be better.
But it is not very simple task becourse of different sampling of color planes in YV12, espessially for interlaced clips.
And it also may produce some artefactes.
Current version of DeSpot can fit luma locally, but how fit color in places of removed spots?
May you propose any algorithm?

[tedkunich]

Fizick,

Probably removing color spot only at places of detected luma spots may be better.

That is exactly what I was thinking of. In my particular case (laser rot) the spots are only visible for one frame - would it be possible to replace the luma and chroma information of the detected spot with a blend or interpolation of frames from either side of the detected defect? I am processing pure progressive frames. Would it be helpfull if I sent you a few frame grabs or a small avi?


Thanks,

Ted

[Fizick]

New version of DesPot.
Changes 3.1 (June 27, 2004) from 3.0:
Add color correction at place of removed spots.

Parameter "color":
true - change color of pixels at places of removed spots to mean value previous, current and next frames,
false - not change color of pixels at places of detected spots.

Try!

[tedkunich]

Fizick,

You are the Man!!!! Looks like the filter works as advertized - I need to play around with the parameters, as I get some smearing around the spot removal, but the results are better than the previous verisons and certainly much, much better than nothing at all!!! Thank You.

If you ever get to the USA, look me up - I owe you a beer!!!!!


T

Originally posted by Fizick
New version of DesPot.
Changes 3.1 (June 27, 2004) from 3.0:
Add color correction at place of removed spots.

Parameter "color":
true - change color of pixels at places of removed spots to mean value previous, current and next frames,
false - not change color of pixels at places of detected spots.

Try!

[Fizick]

It is impossible to create a perfect filter, but one must to strive for it.

[Fizick]

Recently I release new version of DeSpot.
Changes 3.2 (July 4, 2004) from 3.1:
Corrected enabling and disabling of temporal smooth for some modes.
Changed temporal smooth method to more fast but simpler, new tsmooth is similar to old tsmooth*2.
Restored median mode processing as was before version 3.0.
Partial ISSE optimization for speed increasing (Pentium3 or Athlon is needed now).
Updated doc.

[Backwoods]

Will give a go this week, have some 8mm footage to capture.

[Serg Belyansky]

Fizick, I adore you. The DeSpot filter works better than expected (and documented) -- the results are awesome.

Combined with DePanEstimate, it forms a perfect tool for film restoration. Unfortunately one problem persists: any solid surface with a large number of frequent simular elements (e.g. brick wall or polka-dot dress) will exhibit an erratic motion estimation with elements disappearing and reappearing randomly.

It can be partially remedied by increasing the "range" parameter of Depanestimate, but it slows the process and introduces artifacts on other surfaces.

So I'm still scratching my head.

[Fizick]

Serg Belyansky,

... it forms a perfect tool for film restoration. Unfortunately one problem persists...

Threrefore, it is not perfect yet

The rest of your post is belonged to DePan filter?
I will answer there.
http://forum.doom9.org/showthread.ph...n&pagenumber=4

Please post there.

[Fizick]

New Despot version 3.3 (September 19, 2004):
Added "mscene" parameter for scene change detection.
(To prevent artifacts at scene change)

[Fizick]

Changes new version 3.3.1 (October 08, 2004) from 3.3:

Fixed bugs with scene change detection.

[Jedd]

Originally posted by Fizick
you may try DeSpot plugin for Avisynth for a source with some amount of negative dirt (white marks, hair and streaks).
But it is almost offtopic, please go to DeSpot thread if you want.

I've followed here from the RemoveDirt topic, because in my case I have all of them - white marks, hair, streaks and crawling dust.
Pic1 Pic2 Pic3
RemoveDirt removes everything, but details suffer quite a bit.
So I tried DeSpot and indeed it removes some of the white sparks, but I couldn't make him to remove scatches (pic3: right eye), hair lines (pic1: nose) and some of the not so white sparks (pic2: right cheek, pic3: over the left ear)
What are the tweak directions to remove these scratches and bluish sparks on the background?

[Fizick]

Jedd,
Your hair scratches are probably too weak for current Despot.
However, you may try this script:

despot(sign=-1,pwidth=100,pheight=10,p1=15,p2=8,mthres=12)

But I think, that more safe for so low threshold p1 is two-pass call (first for thin long horizontal scratches and second for small compact spots):

despot(sign=-1,pwidth=100,pheight=2,p1=15,p2=8,mthres=12)
despot(sign=-1,pwidth=10,pheight=10,p1=15,p2=8,mthres=12)

But it is not perfect. Try.
Any comment and suggestion for DeSpot inproving?

[Jedd]

I've got a little far than that, now my script looks like:

#big black spots
despot(sign=1,pwidth=45,pheight=45,p1=15,p2=8,mthres=8,color=true)
#thin white lines
despot(sign=-1,pwidth=140,pheight=100,p1=8,p2=1,mthres=5)
#thick hair lines (avoid real hair)
despot(sign=-1,pwidth=140,pheight=100,p1=15,p2=13,mthres=13,merode=24)
#lines contacting motion, avoid motion areas
despot(sign=-1,pwidth=140,pheight=100,p1=15,p2=13,mthres=13,merode=16,seg=1)
#small dots and dust
despot(pwidth=6,pheight=5,p1=15,p2=8,mthres=12,tsmooth=2)

And I have to say results are outstanding! I remove my hat before the author.
Pic3 before
Pic3 after

I was trying to make it more compact, but thin and thick lines and dots - they all whant diffrenet thresholds.
What is difficult:
- to get rid of the light blue dots on the dark green wall.
- gray hair - when the head moves the hair tends to stay where it were. I could partially fix it with the low merode arg.
- to understand why higher thresholds fix big artifacts and don't fix small ones.

Another very strange problem I met - I use dvdrebuilder to compress my video and it seems, rebuilder doesn't like 5 despots in a row. 4 are fine, 4 despots + deen are fine too. VirtualDub takes 5 without any problems. I asked Rebuilder pros here:
http://forum.doom9.org/showthread.php?s=&threadid=84660
but maybe somebody here knows what might be wrong?