Experimental Declick( ) filter

[Guest]

I've made a proof of concept for a declicker. I'd like to get some feedback as well as request any clips suffering from dust and/or scratches, as I want to generalize this filter into a general-purpose dust, scratch, and click filter.

The clip can be obtained from:

[EDIT: Get later version below]

See the next post for a sample before and after frame.

Declick Filter for Avisynth (Version 0.05)
------------------------------------------

This is a proof of concept for a new declicking
algorithm. Typical clicks are generated by motor
noise and other interference.

Currently, the filter handles light horizontal clicks.
I want to add handling of dark clicks and vertical
features. That would turn it into a general dust and
scratch filter as well as a declicker.

This filter currently requires YUY2 input.

Usage:

Declick([named parameters])

lthresh (int, default 35): Luma threshold for click detection.

cthresh (int, default 15): Chroma threshold for click detection.

map (bool, default false): Show detected clicks.

Example:

The distribution contains a sample clip with clicks (real-world
example) that is cleaned up nicely by the default parameters. I
received this clip from a user about 2 years ago asking me to make
a declicker for it. As they say, better late than never.

[Guest]

Here is a sample frame. It's not perfect (what is?), but it's a great improvement. Playing the clip before and after Declick() gives the best impression.

[Le Furet]

Seems very interesting.

Can I ask how you reconstruct the images below the clicks ? Temporal or spacial interpolation ? Or something else ?

[Guest]

I'm still working on that. Right now it's a simple spatial vertical gradient fill using the nearest good pixels. But that produces artifacts when the fill pixel(s) used to define the gradient differs greatly from the average color of the (lost) area to be filled. I plan to detect that case and use temporal interpolation there. What do you think?

[Le Furet]

Didn't make any attempts with temporal interpolation. Interpolation must not be the true word, it's rather a problem of copy, with move detection before.

I suppose it would give good results for you, since the corrupted zones occur only on one frame (I suppose).

Moreover, since these zone are thin, maybe a better spatial interpolation can help (bilinear may suffice).


I'm working on a way to remove hardcoded subtitles, and it gives me similar problems : I must 1) detect text on the picture (almost done) and 2) reconstruct the picture below. I'm studying several methods. But since I have to reconstruct areas on more than one frame, I'm more interested in spatial interpolation algorithms. I found one based on "Radial Basis Functions" that seems not too hard to implement, not too slow and that gives among best results I've seen on the subject (but the nec plus ultra in Image reconstruction seems to be algorithms based on Navier Stokes equations).

[Guest]

Originally posted by Le Furet
Didn't make any attempts with temporal interpolation. Interpolation must not be the true word, it's rather a problem of copy, with move detection before.It could be interpolation, if you use the average of the previous and following frames.

I'm more interested in spatial interpolation algorithms. I found one based on "Radial Basis Functions" that seems not too hard to implement, not too slow and that gives among best results I've seen on the subject (but the nec plus ultra in Image reconstruction seems to be algorithms based on Navier Stokes equations). Please post links to relevant information if you can. I am very interested in this.

[Le Furet]

A readable site on RBF and image processing applications (have a look on the section on scratch removal) : http://www.doc.ic.ac.uk/~hz3/m4rproject/

[Si]

A nice bit of light reading for bedtime tonight :)

regards
Simon

[Guest]

I've improved the click detection a bit so there are less false positives and they are smaller. Get it here:

http://shelob.mordor.net/dgraft/declick/declick006.zip

@Le Furet

Thank you for the link.

[jorel]

@ neuron2,
please...

it's only for avisynth 2.5x?

thanks in advance.

[Guest]

@jorel

It is for Avisynth 2.5. I do not program for obsolete software versions. :)

[HarryM]

Originally posted by neuron2
I'm still working on that. Right now it's a simple spatial vertical gradient fill using the nearest good pixels. But that produces artifacts when the fill pixel(s) used to define the gradient differs greatly from the average color of the (lost) area to be filled. I plan to detect that case and use temporal interpolation there. What do you think?

Combination temporal-spatial (matrix 3x3) is best.

[Guest]

@HarryM

Best for what?

You can't do a 3x3 spatial when your lost area is (much) bigger than a single pixel! Methinks you are considering general noise reduction, which is quite a different problem. Or perhaps I've misunderstood your point.

[morsa]

Guys.Look at Restoreinpaint source code.
See the examples at its home page.
It removes any kind of text from an image recreating the lost parts of the original image.
It is really excellent.

www.sourceforge.net

[jorel]

Originally posted by neuron2
@jorel

It is for Avisynth 2.5. I do not program for obsolete software versions. :)

ok but 2.5 is beta, and we got some problems
using this beta a few days ago.

is stable now?

[Guest]

Please keep the focus. That is not an issue for this thread. I am not making any more filters for pre 2.5 Avisynth versions.

[HarryM]

Originally posted by neuron2
@HarryM

Best for what?

You can't do a 3x3 spatial when your lost area is (much) bigger than a single pixel! Methinks you are considering general noise reduction, which is quite a different problem. Or perhaps I've misunderstood your point.

You must expendable any pixels potentially. I think 3x3 matrix over temporal, e.g.

f f f
r r r
a a a
m m m
e e e
1 2 3

x x x
x o x
x x x

x - substitute pixel
o - bad pixel

[^^-+I4004+-^^]

can it cure this
http://i4004.bizhosting.com/cgi-bin/i/030501-005140.jpg
(or images from same series (030501) on
http://i4004.bizhosting.com/ ?

temp smoothing doesn't help in this case os analog satv "sparklies"
these are black and white dots noise........

why don't i try instead?
well no real use for 2.5 here as i only do capturing...
but if it can resolve such noise i might give it a try on win98 partition....(and i go for win98 oly in case of emergency..)

or in other words;will it spot and remove black dots too(the black dot is visible at http://i4004.bizhosting.com/cgi-bin/i/030501-005201.jpg ),or only white ones?
(don't worry for that noise induced combing which is present too)

[Guest]

@I4004

It can remove black or white. Can you make available some clips so that I can perfect the filter on your real-world material. As a reward I can make an Avisynth 2.0x version for you. :)

[jorel]

Originally posted by neuron2
@I4004

It can remove black or white. Can you make available some clips so that I can perfect the filter on your real-world material. As a reward I can make an Avisynth 2.0x version for you. :)


i don't know what i do reading:
"As a reward I can make an Avisynth 2.0x version for you."

cry or laugh?

:rolleyes:

[^^-+I4004+-^^]

your reward would be just YUMMI!

i can provide you with some samples indeed (just give me few
hours to capture them,see about upload etc.)
as huff is overkill for my modem connection,i'll probably make few
mjpeg's....(same as i always use for capturing)

(i also need to see if i can trick this lousy free web hoster into believing .rar is .jpeg file (by just changing file extension to .jpeg) this way i can post multivolume rar's so you get bigger videos....)
and someone said that there's some 800MB free provider..(hehe)

cheers
/ivo
(samples coming shortly in this spot...shortly is relative term here..hehe)

edit;free web options exhausted->no freakin way
my provider's web is 5MB only.....
will PM graft to see how will he get it...

[Guest]

Thank you for the report. I hope to get back to this filter shortly.

[Shayne]

Great news to hear you are working on such a filter we really need something that works on old spotty movies. The only thing that i could find was spotremover the vdub filter and it is an unfree plugin for a free program ........... should be some sort of crime.

Anyway, i have the original zorro movie in vob format that has black white and just a whole lot of garbage in it if you are interested i think it would be a real challenge.

Alway great to hear on your new filters

Peace

[kevina]

Neuron2, Could you post a link to the source? Or even better an explanation of how it works. Thanks.

[Guest]

Originally posted by kevina20723
Neuron2, Could you post a link to the source? Or even better an explanation of how it works. Thanks. I'm not happy enough with it at this point. It is "experimental".

[kevina]

I'm not happy enough with it at this point. It is "experimental".
Well, I will still like to see the source as I might be able to imporove upon it.

[Shayne]

This filter is not on your web site, could you please post it. I have used this and it shows great promise and would love to see it progression.

Thank you

[celtic_druid]

I'm confused... I was going to point out this (http://forum.doom9.org/showthread.php?s=&threadid=57337) thread, but then I noticed you had already posted in it and before you posted here.