Possible to use avisynth to eliminate dead pixels on video?

[markshanks]

Hi there,

I have a digital video camera, and unfortunately it now has ONE dead pixel on the CCD. This results in an always white pixel on any video I have recorded. Depending on the scene, it can be very distracting.

It would in fact be easy to fix this. All I need is software where you can specify the location of the dead pixel, and it will then change its colour to that of the average of the surrounding pixels. I am sure that this would completely eliminate the problem in almost all (if not completely all) situations.

Question 1:

Would avisynth be able to do this? I have not used avisynth before but someone on usenet suggested it might be able to.

Question 2:

If so, would someone who is familiar with avisynth be able to write the appropriate script or whatever is required (assuming it is a two-minute job once you are familiar with the program)?

I think that such a program would be useful for quite a few people - when I posted this question on usenet a couple of other people said they had the same problem, and it would be better to find a software solution rather than throwing away a camera that cost thousands of dollars.


Thanks,

Mark

[Bester]

The first that comes in my mind:

Locate the pos of the px. Build a mask ( with px=white, rest=black)
and then use

overlay(clip,clip,y=1,mask=your_mask)

The "wrong" px will be replaced by the px above.

[AVIL]

Hi markshanks,

You can try also the filter Removegrain from kassandro. With mode=1 it sets the value of every pixel in the scope min-max of their eight neighbours. So, your white pixel will be changed by another similar to the environment. You be carefull that this action is performed on the whole frame, not only on your bad pixel. Usually, this effect is positive. If not, you can use a mask to recover the previous value for the rest of the pixels.

Good luck and wellcome.

[markshanks]

Okay, finally got avisynth to work and am very encouraged. I usually edit in premiere, and then convert the dv to mpeg using tmpgenc, so it is very easy to use avisynth as input to tmpgenc and the slow down isn't a hassle as the conversion to mpeg is slow to begin with.

I also worked out how to run removegrain and this does indeed greatly reduce the visibility of the dead pixel. However, I dislike doing removegrain to the whole frame, as it does soften the image slightly and results in a very slight loss of detail.

I tried to work out how to do a mask so that removegrain would only be applied to the offending pixel, but couldn't work out how you define the mask. When you look up mask in the help, it simply takes you to the layer command which I presume you use to combine the images after you have defined the mask and applied the filter to the unmasked area.

How does one actually define a mask (assuming that is what you need to do) so that removegrain only works on a single pixel or a few pixels?

[scharfis_brain]

or try undot()

[Didée]

RemoveGrain(1) == Undot() [YV12 source]
RemoveGrain(1,0) == Undot() [YUY2 source]

If the pixel is always white, a possibility much faster then doing any masking is using a filter from the MaskTools (http://manao4.free.fr/MaskTools-v1.5.8.zip) (just 'import()' it in your script) to apply the RemoveGrain-filtering only to white pixels:

YV12Lutxy(last,last.removegrain(1),"x 234 > y x ?","y","y",U=3,V=3)

[kassandro]

Hi there,

I have a digital video camera, and unfortunately it now has ONE dead pixel on the CCD. This results in an always white pixel on any video I have recorded. Depending on the scene, it can be very distracting.

A digital video camera probably produces interlaced video. On the other hand, Undot or RemoveGrain(mode=1) is for progressive video only. This would explain your complaints about this filters. You would be the first at all, who claims that Undot or RemoveGrain(mode=1) is to aggressive on progressive footage. The denoiser in a high resolution digicam is probably much more aggressive.
If RemoveGrain(mode=1) removes the dead pixel, then it must be truely a single and I can easily write a filter for this problem. In the progressive case the solution is obvious. RemoveDeadPixel(x=321, y=64) applies RemoveGrain(mode=12) (but without the center pixel) solely to the pixel with coordinates (321,64). In the interlaced case one can think of more sophisticated solutions.

[Bester]

How does one actually define a mask (assuming that is what you need to do) so that
removegrain only works on a single pixel or a few pixels?

It seems I only post Overlay stuff... :)

Mask:
You could cap an image of your vid, eg. with virtualdup as sequ. bmp.
Open your bmp in "paint" or another prog.
Paint the dead px or the region that should be processed as white, the others black.
Save it.

Script sth. like

dv=directshowsource(...)
mask_bmp=Imagereader(...)

dv_with_filter=dv.filter()

overlay(dv,dv_with_filter, mask=mask_bmp)

[markshanks]

Dear kassandro,

Sorry, I should have confirmed that I am using progressive video.

Can you write the filter you suggest? As I said, I am sure that it would be helpful to other people as well.

Incidentally, when I made the point about removegrain losing detail, this was only visible in some scenes, and only when I had still frames next to each other (one with removegrain, one without) to compare. Even then, it was very subtle, and I may have been imagining the difference. The main reason I would prefer to just target the pixel is then I don't need to be concerned that removegrain is removing detail in some situations that I have not specifically examined, ie, more of being on the safe side without extensively studying the effects of removegrain.

Again, I think the filter you suggest would be the easiest and best solution if you can write it.

[Mug Funky]

this'd be a great thing to have in a plugin :) specify which dot is dead (possibly in an ini file or something similar) and the plugin interpolates around it while ignoring the rest of the video. should be really fast because only a few pixels are processed, and everything else is just copied.

one thing that concerns me slightly - ringing in the DV codec. even if the white pixel is fixed there'll still be the mosquito noise that it caused surrounding it. i think a filter could even handle this if it's clever enough and DV specific (there'll be a fair idea about where the mosqiuto noise will occur seeing as the pixel never moves).

[kassandro]

You can download the RemoveDeadPixels plugin from here (http://home.arcor.de/kassandro/RemoveDeadPixels.rar). It can be used as follows:
RemoveDeadPixels(clip input, int x, int y, int "select", int ycolor, int vcolor, int ucolor).
The first three variables are mandatory and self explanatory. With "select" you what you want to interpolate. 1 = interploate Y value, 2 = interploate U value, 4 = interploate V value. If you want to interpolate all, then choose select = 1 + 2 + 4 = 7. If ycolor >= 0 then RemoveDeadPixels creates a dead pixel at the specified location. I needed this option for testing and you can use it for locating a dead pixel ycolor=0 creates a dead black pixel and ycolor=255 creates a dead white pixel.
Because YUY2 is the native color space of DV and digicam clips, the plugin is for YUY2 only. When I have time, I extend the plugin to handle several dead pixels in one stroke.
Here is an example script:

avisource("my_yuy2.avi", pixel_type="yuy2")
RemoveDeadPixels(100, 200, 7, 230) # creat a dead pixel
RemoveDeadPixels(100, 200, 7) # and remove it

Instead of simple averaging RemoveDeadPixels inspects all four lines through a pixel and selects that line for interpolation, where the distance between the two neighbours is minimal.

Due to conflicts with one moderator, I no more support my plugins within the doom9 forum. If you want support, go to my free speech support forum (you find a link on the RemoveGrain/RemoveDirt web sites).

[kassandro]

this'd be a great thing to have in a plugin :) specify which dot is dead (possibly in an ini file or something similar) and the plugin interpolates around it while ignoring the rest of the video. should be really fast because only a few pixels are processed, and everything else is just copied.

one thing that concerns me slightly - ringing in the DV codec. even if the white pixel is fixed there'll still be the mosquito noise that it caused surrounding it. i think a filter could even handle this if it's clever enough and DV specific (there'll be a fair idea about where the mosqiuto noise will occur seeing as the pixel never moves).

Yes, RemoveDeadPixels is very fast. It doesn't contain code for deringing, though. On the other hand, I expect the bitrate to be high enough such that ringing isn't a visible problem. For instance, my Canon Powershot S1 needs over 100 MB for 1 minute (mjpeg 640x480@30 fps).

[markshanks]

Hi Kassandro,

I have posted feedback for your Removedeadpixels plugin at the Removegrain/Removedirt forum you suggested.

Cheers,

Mark