Experimental Declick( ) filter

neuron2 · 27th April 2003, 08:46

I've made a proof of concept for a declicker. I'd like to get some feedback as well as request any clips suffering from dust and/or scratches, as I want to generalize this filter into a general-purpose dust, scratch, and click filter.

The clip can be obtained from:

[EDIT: Get later version below]

See the next post for a sample before and after frame.

Declick Filter for Avisynth (Version 0.05)
------------------------------------------

This is a proof of concept for a new declicking
algorithm. Typical clicks are generated by motor
noise and other interference.

Currently, the filter handles light horizontal clicks.
I want to add handling of dark clicks and vertical
features. That would turn it into a general dust and
scratch filter as well as a declicker.

This filter currently requires YUY2 input.

Usage:

Declick([named parameters])

lthresh (int, default 35): Luma threshold for click detection.

cthresh (int, default 15): Chroma threshold for click detection.

map (bool, default false): Show detected clicks.

Example:

The distribution contains a sample clip with clicks (real-world
example) that is cleaned up nicely by the default parameters. I
received this clip from a user about 2 years ago asking me to make
a declicker for it. As they say, better late than never.

neuron2 · 27th April 2003, 08:55

Here is a sample frame. It's not perfect (what is?), but it's a great improvement. Playing the clip before and after Declick() gives the best impression.

Le Furet · 27th April 2003, 11:57

Seems very interesting.

Can I ask how you reconstruct the images below the clicks ? Temporal or spacial interpolation ? Or something else ?

neuron2 · 27th April 2003, 15:22

I'm still working on that. Right now it's a simple spatial vertical gradient fill using the nearest good pixels. But that produces artifacts when the fill pixel(s) used to define the gradient differs greatly from the average color of the (lost) area to be filled. I plan to detect that case and use temporal interpolation there. What do you think?

Le Furet · 27th April 2003, 17:52

Didn't make any attempts with temporal interpolation. Interpolation must not be the true word, it's rather a problem of copy, with move detection before.

I suppose it would give good results for you, since the corrupted zones occur only on one frame (I suppose).

Moreover, since these zone are thin, maybe a better spatial interpolation can help (bilinear may suffice).

I'm working on a way to remove hardcoded subtitles, and it gives me similar problems : I must 1) detect text on the picture (almost done) and 2) reconstruct the picture below. I'm studying several methods. But since I have to reconstruct areas on more than one frame, I'm more interested in spatial interpolation algorithms. I found one based on "Radial Basis Functions" that seems not too hard to implement, not too slow and that gives among best results I've seen on the subject (but the nec plus ultra in Image reconstruction seems to be algorithms based on Navier Stokes equations).

neuron2 · 27th April 2003, 18:03

Quote:

Originally posted by Le Furet
Didn't make any attempts with temporal interpolation. Interpolation must not be the true word, it's rather a problem of copy, with move detection before.

It could be interpolation, if you use the average of the previous and following frames.

Quote:

I'm more interested in spatial interpolation algorithms. I found one based on "Radial Basis Functions" that seems not too hard to implement, not too slow and that gives among best results I've seen on the subject (but the nec plus ultra in Image reconstruction seems to be algorithms based on Navier Stokes equations).

Please post links to relevant information if you can. I am very interested in this.

Le Furet · 27th April 2003, 18:23

A readable site on RBF and image processing applications (have a look on the section on scratch removal) : http://www.doc.ic.ac.uk/~hz3/m4rproject/

Si · 27th April 2003, 20:14

A nice bit of light reading for bedtime tonight

regards
Simon

neuron2 · 28th April 2003, 04:49

I've improved the click detection a bit so there are less false positives and they are smaller. Get it here:

http://shelob.mordor.net/dgraft/declick/declick006.zip

@Le Furet

Thank you for the link.

28th April 2003, 04:58

@ neuron2,
please...

it's only for avisynth 2.5x?

thanks in advance.

neuron2 · 28th April 2003, 05:09

@jorel

It is for Avisynth 2.5. I do not program for obsolete software versions.

HarryM · 28th April 2003, 19:28

Quote:

Originally posted by neuron2
I'm still working on that. Right now it's a simple spatial vertical gradient fill using the nearest good pixels. But that produces artifacts when the fill pixel(s) used to define the gradient differs greatly from the average color of the (lost) area to be filled. I plan to detect that case and use temporal interpolation there. What do you think?

Combination temporal-spatial (matrix 3x3) is best.

neuron2 · 28th April 2003, 20:30

@HarryM

Best for what?

You can't do a 3x3 spatial when your lost area is (much) bigger than a single pixel! Methinks you are considering general noise reduction, which is quite a different problem. Or perhaps I've misunderstood your point.

morsa · 29th April 2003, 01:57

Guys.Look at Restoreinpaint source code.
See the examples at its home page.
It removes any kind of text from an image recreating the lost parts of the original image.
It is really excellent.

www.sourceforge.net

30th April 2003, 09:16

Quote:

Originally posted by neuron2
@jorel

It is for Avisynth 2.5. I do not program for obsolete software versions.

ok but 2.5 is beta, and we got some problems
using this beta a few days ago.

is stable now?

neuron2 · 30th April 2003, 13:26

Please keep the focus. That is not an issue for this thread. I am not making any more filters for pre 2.5 Avisynth versions.

HarryM · 30th April 2003, 21:06

Quote:

Originally posted by neuron2
@HarryM

Best for what?

You can't do a 3x3 spatial when your lost area is (much) bigger than a single pixel! Methinks you are considering general noise reduction, which is quite a different problem. Or perhaps I've misunderstood your point.

You must expendable any pixels potentially. I think 3x3 matrix over temporal, e.g.

f f f
r r r
a a a
m m m
e e e
1 2 3

x x x
x o x
x x x

x - substitute pixel
o - bad pixel

^^-+I4004+-^^ · 1st May 2003, 01:09

can it cure this
http://i4004.bizhosting.com/cgi-bin/i/030501-005140.jpg
(or images from same series (030501) on
http://i4004.bizhosting.com/ ?

temp smoothing doesn't help in this case os analog satv "sparklies"
these are black and white dots noise........

why don't i try instead?
well no real use for 2.5 here as i only do capturing...
but if it can resolve such noise i might give it a try on win98 partition....(and i go for win98 oly in case of emergency..)

or in other words;will it spot and remove black dots too(the black dot is visible at http://i4004.bizhosting.com/cgi-bin/i/030501-005201.jpg ),or only white ones?
(don't worry for that noise induced combing which is present too)

neuron2 · 1st May 2003, 01:53

@I4004

It can remove black or white. Can you make available some clips so that I can perfect the filter on your real-world material. As a reward I can make an Avisynth 2.0x version for you.

1st May 2003, 09:55

Quote:

Originally posted by neuron2
@I4004

It can remove black or white. Can you make available some clips so that I can perfect the filter on your real-world material. As a reward I can make an Avisynth 2.0x version for you.

i don't know what i do reading:
"As a reward I can make an Avisynth 2.0x version for you."

cry or laugh?

27th April 2003, 08:46	#1 \| Link
neuron2 _ Join Date: Jan 2002 Posts: 16,634	Experimental Declick( ) filter I've made a proof of concept for a declicker. I'd like to get some feedback as well as request any clips suffering from dust and/or scratches, as I want to generalize this filter into a general-purpose dust, scratch, and click filter. The clip can be obtained from: [EDIT: Get later version below] See the next post for a sample before and after frame. Declick Filter for Avisynth (Version 0.05) ------------------------------------------ This is a proof of concept for a new declicking algorithm. Typical clicks are generated by motor noise and other interference. Currently, the filter handles light horizontal clicks. I want to add handling of dark clicks and vertical features. That would turn it into a general dust and scratch filter as well as a declicker. This filter currently requires YUY2 input. Usage: Declick([named parameters]) lthresh (int, default 35): Luma threshold for click detection. cthresh (int, default 15): Chroma threshold for click detection. map (bool, default false): Show detected clicks. Example: The distribution contains a sample clip with clicks (real-world example) that is cleaned up nicely by the default parameters. I received this clip from a user about 2 years ago asking me to make a declicker for it. As they say, better late than never. Last edited by neuron2; 28th April 2003 at 04:49.

27th April 2003, 15:22	#4 \| Link
neuron2 _ Join Date: Jan 2002 Posts: 16,634	I'm still working on that. Right now it's a simple spatial vertical gradient fill using the nearest good pixels. But that produces artifacts when the fill pixel(s) used to define the gradient differs greatly from the average color of the (lost) area to be filled. I plan to detect that case and use temporal interpolation there. What do you think? Last edited by neuron2; 27th April 2003 at 16:01.

27th April 2003, 20:14	#8 \| Link
Si Simply me Join Date: Aug 2002 Location: Lancashire, England Posts: 597	A nice bit of light reading for bedtime tonight regards Simon __________________ http://www.geocities.com/siwalters_uk/fnews.html

28th April 2003, 04:49	#9 \| Link
neuron2 _ Join Date: Jan 2002 Posts: 16,634	I've improved the click detection a bit so there are less false positives and they are smaller. Get it here: http://shelob.mordor.net/dgraft/declick/declick006.zip @Le Furet Thank you for the link. Last edited by neuron2; 28th April 2003 at 04:52.

1st May 2003, 01:09	#18 \| Link
^^-+I4004+-^^ Banned Join Date: Aug 2002 Location: Croatia [local name:Hrvatska] Posts: 552	can it cure this http://i4004.bizhosting.com/cgi-bin/i/030501-005140.jpg (or images from same series (030501) on http://i4004.bizhosting.com/ ? temp smoothing doesn't help in this case os analog satv "sparklies" these are black and white dots noise........ why don't i try instead? well no real use for 2.5 here as i only do capturing... but if it can resolve such noise i might give it a try on win98 partition....(and i go for win98 oly in case of emergency..) or in other words;will it spot and remove black dots too(the black dot is visible at http://i4004.bizhosting.com/cgi-bin/i/030501-005201.jpg ),or only white ones? (don't worry for that noise induced combing which is present too) Last edited by ^^-+I4004+-^^; 1st May 2003 at 01:13.

27th April 2003, 08:55	#2 \| Link
neuron2 _ Join Date: Jan 2002 Posts: 16,634	Here is a sample frame. It's not perfect (what is?), but it's a great improvement. Playing the clip before and after Declick() gives the best impression.

27th April 2003, 11:57	#3 \| Link
Le Furet Registered User Join Date: Apr 2002 Location: France Posts: 51	Seems very interesting. Can I ask how you reconstruct the images below the clicks ? Temporal or spacial interpolation ? Or something else ?

27th April 2003, 17:52	#5 \| Link
Le Furet Registered User Join Date: Apr 2002 Location: France Posts: 51	Didn't make any attempts with temporal interpolation. Interpolation must not be the true word, it's rather a problem of copy, with move detection before. I suppose it would give good results for you, since the corrupted zones occur only on one frame (I suppose). Moreover, since these zone are thin, maybe a better spatial interpolation can help (bilinear may suffice). I'm working on a way to remove hardcoded subtitles, and it gives me similar problems : I must 1) detect text on the picture (almost done) and 2) reconstruct the picture below. I'm studying several methods. But since I have to reconstruct areas on more than one frame, I'm more interested in spatial interpolation algorithms. I found one based on "Radial Basis Functions" that seems not too hard to implement, not too slow and that gives among best results I've seen on the subject (but the nec plus ultra in Image reconstruction seems to be algorithms based on Navier Stokes equations).

27th April 2003, 18:23	#7 \| Link
Le Furet Registered User Join Date: Apr 2002 Location: France Posts: 51	A readable site on RBF and image processing applications (have a look on the section on scratch removal) : http://www.doc.ic.ac.uk/~hz3/m4rproject/

28th April 2003, 04:58	#10 \| Link
jorel Guest Posts: n/a	@ neuron2, please... it's only for avisynth 2.5x? thanks in advance.

28th April 2003, 05:09	#11 \| Link
neuron2 _ Join Date: Jan 2002 Posts: 16,634	@jorel It is for Avisynth 2.5. I do not program for obsolete software versions.

28th April 2003, 20:30	#13 \| Link
neuron2 _ Join Date: Jan 2002 Posts: 16,634	@HarryM Best for what? You can't do a 3x3 spatial when your lost area is (much) bigger than a single pixel! Methinks you are considering general noise reduction, which is quite a different problem. Or perhaps I've misunderstood your point.

29th April 2003, 01:57	#14 \| Link
morsa the dumbest Join Date: Oct 2002 Location: Malvinas Posts: 494	Guys.Look at Restoreinpaint source code. See the examples at its home page. It removes any kind of text from an image recreating the lost parts of the original image. It is really excellent. www.sourceforge.net

30th April 2003, 13:26	#16 \| Link
neuron2 _ Join Date: Jan 2002 Posts: 16,634	Please keep the focus. That is not an issue for this thread. I am not making any more filters for pre 2.5 Avisynth versions.

1st May 2003, 01:53	#19 \| Link
neuron2 _ Join Date: Jan 2002 Posts: 16,634	@I4004 It can remove black or white. Can you make available some clips so that I can perfect the filter on your real-world material. As a reward I can make an Avisynth 2.0x version for you.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode