How to remove diagonal bars in video?

[Morpheus_xx]

Hi,

I capture series from analog TV using a Hauppauge PVR150 (720x576, VBR max 12MBit).

Only on one channel (German RTL2) I have strange diagonal "bars" on captured video. They become stronger and then weaker and so on, like "flashing".

Screenshot:
http://mf-the-one.netfirms.com/sg1_bars.htm

Does anybody know if there's a good way to remove this???

Or better, what cause this noise and how to avoid it?

Thnx,
Morpheus

ps: where is my attachment :confused:

[Morpheus_xx]

While searching the forum I found the "defreq" plugin.

Now I try to use it. But I can't find good parameters. The sample on the website (http://www.avisynth.org/fizick/defreq/defreq.html) is quite unlike my frequency spectrum. I do not have clear hot spots :confused:

And it's alternating frame by frame (see frame 9750 vs. 9751, middle region).

Images and avisynth script see link in 1st post.


Maybe someone can help?

[Fizick]

Morpheus_xx,
Your picture is too dark,
And i do not see any diagonal lines on it.
Sorry.

[Fizick]

Morpheus_xx,
Your picture is too dark,
And i do not see any diagonal lines on it.
Sorry.

[Morpheus_xx]

well, you're right. was a bad sample ;-) (my display is also quite bright)

this one should show it better.

please also see link above, there's the debug-output from defreq.
any hints for parameters?

thanks in advance
morpheus

[Fizick]

try fx=12, fy=-5
I see small bright sharp dot on spectrum there.

[Morpheus_xx]

thank you for reply.

I tried your settings, but unfortunately the difference of the filtered clip was not visible.

So I made some other tests like:
DeFreq(fx=11,fy=98,sharp=90,fx2=96,fy2=100,fx3=61,fy3=98,fx4=64,fy4=-97)

This seems to help a bit. I had to use such high values to focus some spots.
Is this the right way? Or am I completly wrong?

Can I adjust the "strength" of DeFreq ? I would need the effect "stronger".

[vcmohan]

I have just authored a plugin FanFilter (The name is due to my assosiation with Seismic data processing for Oil and Gas). I have tested this on your two images and find that with default settings the noise is gone. I see some horizontal streaks which probably are on the original.Some smoothening and bit of edge is seen . Smoothening gave a pleasing effect.
The plugin should be shortly available on www.avisynth.org/warpenterprises page for a free down load.

[Morpheus_xx]

Sounds good! I always like to test new filters :-)

It's not on the page yet.

Can you post the filter here?

[vcmohan]

warpenterprises take a few days to post the link on their page. Meanwhile following are the urls for images output by Fanfilter
http://img141.echo.cx/img141/1313/fanfilter08ik.jpg (http://www.imageshack.us)
and
http://img189.echo.cx/img189/2195/fanfilter27qy.jpg (http://www.imageshack.us)

I do not know whether this is the way to do it or delete url tags!

[Morpheus_xx]

wow, amazing! I can't await to test it on the clip!

Can you tell me about the performance? Most of the existing denoise-filters are imho quite slow.

[WarpEnterprises]

meanwhile by proxy of vcmohan:

http://www.avisynth.org/warpenterprises#fanfilter

[Morpheus_xx]

I've made a first test with default values.

As I said before: amazing! For this problem it's the best filter I've tested yet!

And performance is also very good.

I'll do some more tests with the parameters to see the influence to the video.

Thank you, great work!

[Wilbert]

@vcmohan,

Is there a reason you make those great filters without releasing the source?

[trevlac]

Yeah VC,

I'd love to know more details.

I've been fooling with this frequency stuff in my spare time. I loved your post because it gave me some clues.

Threshold the black for example.

Anyway ... so by default does 16 mean we start at 32/width and remove periodic frequencies? What's the period? Do you remove or reduce them? Is the 5% threshold based upon DC as the max?

Here is my try at this..... No thresholding the black. And i did not smooth after the filter. I was really just guessing.


http://trevlac.us/pics/sg1_61190Y.jpg

edit:
Image is only processing lines 100-476 Previous image is as
http://trevlac.us/pics/sg1_61190X.jpg

[vcmohan]

I am happy that you found the filter does a good job.
As for why I do not release the source:
1. I am a novice at C++ programming and am afraid they will be too many poor programming pieces.
2. I find that for many plugins hosted on the warpenterprises page source is not available.
3. If I get feedback I would love to work on my code continuously. I have retired from work in 1992 and can afford to devote time.
4. Of course may be in not too distant future the sources will be made available.

Regarding Parameters:
Lambda is related to wavelength of noise freq. It may be half or some other ratio. The plugin examines data over this moving window.

Threshold is to limit corrections in dark areas. Amp limits in brighter areas.

I examined the actual pixelvalues of the image using pixelvalues plugin and found that normal freq domain methods may not work as the actual level of interference is very low. In very bright areas it is not even seen.

To avoid artifacts one need to ensure that these three parameters have the least values that can remove noise.

The algorithm used is very simple, and so is quite fast. The seemingly diagonal nature is because the scanning is done in a particular order and with some padding values in TV signal. It ( the angle of diagonal) has no effect on working.

I have just introduced RGB32 and RGB24 color formats also.Earlier I had some misgivings about working in these formats. But I find my algorithm works in these formats also. The defaults however left out some noise. It appears it requires higher threshold and amp parameters. This updated version I will release by end of month.

[vcmohan]

Originally posted by trevlac
I'd love to know more details.

I gave some details in my post above.
I use threshold to categorise whether it is noise or signal. If deviation is more than threshold signal.
Eventhough the interference would be same over the image, it occurred to me that our eyes may be playing tricks and so notice it more in darker areas. Thats the reason to introduce amp factor.

Anyway ... so by default does 16 mean we start at 32/width and remove periodic frequencies? What's the period? Do you remove or reduce them? Is the 5% threshold based upon DC as the max?
Lambda default is 16. Lambda has a relation with wavelength (along width axis) of the periodic noise. I use it as a span over which find deviations and using threshold and amp factors correct. Thats the reason some smoothening may be seen in the result. I do not per se use any other smoothening.

[B]Here is my try at this..... No thresholding the black. And i did not smooth after the filter. I was really just guessing.[B]

Its not clear to me whether you are using my plugin or your own. I find that the white streaks in the black areas, a spot in one eye and still slightly visible diagonal noise are somewhat unsatisfactory. There appears to be much less smoothening than in my example. If you are using my plugin then there must be a bug which is producing those white streaks.

[trevlac]

@VC

Thank you for the details. I completely misunderstood what you were doing. I thought you were working within the frequency domain. :)


Originally posted by vcmohan
Its not clear to me whether you are using my plugin or your own. I find that the white streaks in the black areas, a spot in one eye and still slightly visible diagonal noise are somewhat unsatisfactory. There appears to be much less smoothening than in my example. If you are using my plugin then there must be a bug which is producing those white streaks.

I was using a virtualDub filter I created to learn about the frequency domain. And yes ... the white streaks are a problem, but you showed me that they can be thresholded away. I updated my picture. :) And I too see some noise left. As you pointed out ... this might be a limit of the frequency domain. I thought you filtered and then smoothed to remove this.

PS: I am not trying to better your work. I am just trying to learn about an area of interest. Your filter clearly does a great job and is ready for general use. My filter is not in a usable form to do this processing. It's main purpose it to show me the fourier image. To process a video would be terribly slow. ;)

[Wilbert]

Of course you are free to not release your source. If you do people might learn something from it. Anyway, some comments.

As for why I do not release the source:
2. I find that for many plugins hosted on the warpenterprises page source is not available.
3. If I get feedback I would love to work on my code continuously. I have retired from work in 1992 and can afford to devote time.
4. Of course may be in not too distant future the sources will be made available.
2. Most of them have the source included (the ones who haven't can be counted on two hands; or one hand if you exclude your plugins :)).
3. What does that have to do with releasing sources? In general, people will ask the author first if they want to see something changed/added.
4. That would be very nice :)

[Malcolm]

interesting filter!
but if you compare the 2 pictures of the woman (unfiltered and filtered) by switching back- and forth, you can see that in all corners of the picture some slight diagonal bars are introduced. (exactly like the ones that are removed in the middle of the picture).
looks like an 'overcompensation' effect.

Greetings,
Malcolm

[trevlac]

Originally posted by Malcolm
interesting filter!
but if you compare the 2 pictures of the woman (unfiltered and filtered) by switching back- and forth, you can see that in all corners of the picture some slight diagonal bars are introduced. (exactly like the ones that are removed in the middle of the picture).
looks like an 'overcompensation' effect.

Greetings,
Malcolm

Hi,

I assume you are talking about my try. Because I can't see any lines in VC's filter result. I did another pic for you. I just didn't process the top and bottom 100 lines. Looking at the problem in the origninal, it looks like the pattern is only in the middle. Not sure why this is. Maybe just this frame. I do want to repeat that I am goofing with this frequency stuff ... which is farily cool ... and VC is the one with the practical filter. Also, I believe I am doing the basic same thing that Fizick was doing. It might be that the parameters were not set well when defreq was used.


@VC,

Please feel honored that Wilbert is pestering you about the source code. I've been trying to get him to look at mine for months. :)

[vcmohan]

I thought that the interference frequencies being much smaller than the TV signal or scan line frequencies one will invariably notice the interference as being oriented vertical to diagonal. I calculated that the freq of interference in the example above to be around 40khz. I thought that a similar interference appearing horizontal was impossible.
Yesterday I was watching the History channel wherein in some scenes (flash backs) pure horizontally oriented interference is seen. May be they are deliberately done to give flash back impression.

I am curious to know whether any one experienced a horizontal interference noise. What explanation will it have. May be the 40khz is a beat frequency?
In any case I will make a function to deal with this type of interference also for possible use.

[Poutnik]

Originally posted by vcmohan
....a plugin FanFilter (The name is due to my assosiation with Seismic data processing for Oil and Gas).
the name could also be according to bad fans ( ventilators),
causing heavy electric regular noise/interference on power lines :)

[trevlac]

@vcmohan

Here is a 'nice' sample i grabbed from a post on the vdub area. What do you think of this one? It's from VHS. Not sure what stage the interference got there.

http://trevlac.us/pics/msintrf0.jpg

[vcmohan]

Well thats a tough one. Can you give me the url for that virtualdub forum thread where it was posted?
I tried my FanFilter and found that it can hardly do any thing.
Then I examined the pixelvalues using PixelValues plugin (incidentally for some image sizes this plugin gives access violation errors. I did not go deep into that.) I found that the interference amplitude is about 60 i.e. it swings from +60 to -60. Some places it is more.

I realised that I need to formulate a filter on opposite princeples (opposite to my FanFilter). I also expect that in the process of removing such large value noise I will lose quite a bit of signal also.

I changed my FanFilter into VFanFilter, HFanFilter to be effective in vertical and horizontal (these are seemingly Vertical and Horizontal but in reality they are reverse) directions. I also added to this plugin family of functions VFanPass and HFanPass functions. I also had to liberalise restrictions of values for parameters to be able to use in extreme cases but leave it to the user to use his judgement. The Filter is ready and I tested on the image above.
After VFanPass I had to cleanup image of horizontal streaks with HFanFilter.
The resulting image is :
http://img94.echo.cx/img94/1006/fanmsint07uo.jpg (http://www.imageshack.us)

using subtract.levels the noise removed is below:
http://img94.echo.cx/img94/1126/fanmsint20ju.jpg (http://www.imageshack.us)

As expected the image after processing is not as good as one likes it to be. Also I did not experiment with the filter params. This is a first cut. After updating the documentation, I plan to release this newer version of FanFilter Plugin shortly hopefully before my 71st birthday.

[trevlac]

I thought you'd like that one. :)

I think it was here.
http://forum.doom9.org/showthread.php?s=&threadid=91098

Not much was said ... and he removed the pictures. I had just saved that one because I thought it was nice.

Your filter did well. Although from looking at the subtract, I see a few of the edges gone.

I've recently been working on an image spectrum analyzer vdub plugin. Really just for learning purposes. It displays the 2D power spectrum. I did a small write up. Maybe you'd like to take a look?

[edit] Updated url
http://www.trevlac.us/FilterDocs/Filters.htm

-------------------
70 and learning new things every day? Well ... I've still got a few years to catch up. :) I'm 38.

[vcmohan]

Thanks very much for the info. Infact I am looking to get hold of a fft plugin source, an inverse fft plugin source, from which i can get the coefficients to design some filters. Having a bit of aversion to all those mathematical symbols curls, matrix notations etc I can hardly understand those published software to be able to port to avisynth as a plugin for my use. I will study your source try understand.

Your plugin thru freq domain route did remove those noise freq except for a horizontal streak. Theoritically your approach is sounder than mine.Probably looks cleaner in result. But mine working in time domain works faster and does not require too much understanding of 2D fft. Very sharp edges are difficult to remove in freq domain. A short operator in time domain has a long spectrum in freq and vice versa. Please see the thread http://forum.doom9.org/showthread.php?s=&threadid=93229 . Freq domain filtering in this case may not work, I think.

I have been trying to do a notch filter in time domain but got stuck as I could not get hold of the impulse response of a notch filter. These we were using in oil exploration to remove power line interference.

I urge you to please devout some time on this plugin of yours and finish it early.

[tsp]

vcmohan: you can use fftw (http://www.fftw.org/) for the conversion to and from the frequency domain. Also take a look on this page (http://www.jjj.de/fxt/fxtpage.html#fxtbook) also my filter fft3dgpu has the fft code include(although in a perverted form to be used on a graphics card. The source is based on the fft algorithm in "Numerical recipes in C" (good book)

[Arachnotron]

The source is based on the fft algorithm in "Numerical recipes in C" (good book)Be carefull with that book. The licence prohibits you from distributing the source of your programs if you include routines from it (or even the executable itself depending on which licence you have) so using stuff from it in opensource projects can lead to problems.
See here (http://www.numerical-recipes.com/infotop.html#distinfo) especially the second bullit.

I wonder in how far they can enforce the license (how many ways are there to program a bubble sort ?) but I thought I mention it anyway.

[trevlac]

Originally posted by vcmohan
Infact I am looking to get hold of a fft plugin source, an inverse fft plugin source, from which i can get the coefficients to design some filters. Having a bit of aversion to all those mathematical symbols curls, matrix notations etc I can hardly understand those published software to be able to port to avisynth as a plugin for my use. I will study your source try understand.

I too can't really read the math. But I'm working on it. Differential Equations ... yummy. As tsp pointed out. There are certainly better sources for DFT and IDFT. In fact, I just used the FFTW api. They have great doc. It's quite easy.



But mine working in time domain works faster

Probably ... but FFTW is quite fast. Given that and no need to convert to YCbCr ... an avs filter in the freq domain only needs to work on a small set of frequencies to filter them. Not all the pixels. I've got sqrt() and such in my code. There is a lot of room for improvement.


Please see the thread http://forum.doom9.org/showthread.php?s=&threadid=93229 . Freq domain filtering in this case may not work, I think.

I tried and it did not work. The scratches cover too large of a frequency range. It is difficult to isolate them from other parts of the picture. They can be dampened, but your filters do a much better job at removal.


I urge you to please devout some time on this plugin of yours and finish it early.
Thanks for the encouragement. I think I need to play with it for a while to get a better understanding of the Freq domain. I have come a long way .. :) AVS is probably a better place for a real (and fast) FFT filter. Of course Fizick and tsp have really already done that.

PS: I've recently been trying to figure out (phase) cross-correlation. I think they use this in the various deshake/depan plugins. The interesting area being motion estimation.

[vcmohan]

Originally posted by trevlac

Probably ... but FFTW is quite fast. Given that and no need to convert to YCbCr ... an avs filter in the freq domain only needs to work on a small set of frequencies to filter them. Not all the pixels. I've got sqrt() and such in my code. There is a lot of room for improvement.
One thing that comes to my mind if You have not already implemented it is the one dimensional FFT is Symmetric and so only half the coefficients need to be computed. Similarly carrting this further when you take this to 2D FFT probably only a quadrant need to be computed. If FFT or IFFT algorithm requires the other half they just can be copied. It can increase speed.
Can you give a url for a place where I can get a 1D FFT routine that gives me coefficients.


I've recently been trying to figure out (phase) cross-correlation.
Cross correlation is used extensively in seismic data processing. Depending on the cross correlation matrix filters are designed. Autocorrelation gives a great info on the frequencies present. FFT of a autocorrelation is still less work as all coefficients are real, complex part being zero. Decovolution (or Refocus) is designed using autocorrelation matrix.
I tried cross correlation on the first image in this thread to determine whether I can extract noise wavelength and its amplitude.I could get the wavelength to be between say 11 and 12. But could not proceed further when I realised that this much accuracy is not good enough unless I do a cross correlation continuously to determine best filtering position for each pixel. So I gave up.

[trevlac]

@VC

You know much more than I, my friend. :)

But on the symmetrical bit ... yes. That helps quite a bit. To be specific on speed using FFTW ... and doing that bad pic takes ..

- Copy all input luma to an array of doubles ... 640x480 values
- FFT
- Now you have an array of pairs (sin & cos) but only 1/2 the input 320x480x2
- process about a 20 pixel wide band 20x480x2 setting the doubles to zero
- IFFT
- Copy the luma back ... unfortunately you have to scale and clamp these values before you really have luma

- done

------------------
www.fftw.org is the main place.
http://www.fftw.org/#documentation is the doc.


I used version 2 because I could not get a library for v3 nor could I compile it. It says 3 is faster.

Here is v2 Doc and library / .h file info.
http://www.fftw.org/fftw2_doc/
http://claymore.engineer.gvsu.edu/~steriana/software.html

1D transform giving the power specturm ... basicly what I show. 2D has a little different format. Not really much code to do this.

http://www.fftw.org/fftw2_doc/fftw_2.html#SEC5

#include <rfftw.h>
...
{
fftw_real in[N], out[N], power_spectrum[N/2+1];
rfftw_plan p;
int k;
...
p = rfftw_create_plan(N, FFTW_REAL_TO_COMPLEX, FFTW_ESTIMATE);
...
rfftw_one(p, in, out);
power_spectrum[0] = out[0]*out[0]; /* DC component */
for (k = 1; k < (N+1)/2; ++k) /* (k < N/2 rounded up) */
power_spectrum[k] = out[k]*out[k] + out[N-k]*out[N-k];
if (N % 2 == 0) /* N is even */
power_spectrum[N/2] = out[N/2]*out[N/2]; /* Nyquist freq. */
...
rfftw_destroy_plan(p);
}



Enjoy :D

[Fizick]

trevlac,
Why used version 2 because I could not get a library for v3 nor could I compile it. ?

Do you see Defreq source? it used all needed parts of FFTW3 library. The lilrary is dll + def + include file.
I did not use any fftw source.

[trevlac]

Fizick,

I did want to thank you. I don't know if you recall, but in a thread a while back ... I was interested in doing a full frame FFT and quite a few said it could not be done and was not worth it. You offered to help and pointed me in the right direction.

I never did really look at your source. I didn't realize it was in the zip files. Once I saw you used FFTW, I just read their doc. It made getting one working quite easy. Understanding what I am looking at ... that's not as easy. :)

I could not find the lib or def files for v3. I guess I should upgrade when ever I clean up my vdub filter. The format is a little different from v2 to v3. Thanks for the tip. :)

[vcmohan]

2. Most of them have the source included (the ones who haven't can be counted on two hands; or one hand if you exclude your plugins .

Well I reduced those number of fingers. I just posted the FanFilter, Reform and Grid plugins along with the Source codes.
As most of the plugins did not carry the src tag I thought they do not have the source code. That was my mistake.

About EffectsMany and TransAll the source code is large and I find that each time I update a small bit of a function code, I need to not only upload the full dll and documentation, now the source code also which adds to my metered time on net. The documentation for these two dlls is large due to examples which are necessary to get an idea as to what it does. I have to find a solution for it which does not force an user to be on net.

[Fizick]

So, now the most of rest fingers is my ... :eek:

[Wilbert]

Well I reduced those number of fingers. I just posted the FanFilter, Reform and Grid plugins along with the Source codes.
Great! I really appreciate it.


As most of the plugins did not carry the src tag I thought they do not have the source code. That was my mistake.

About EffectsMany and TransAll the source code is large and I find that each time I update a small bit of a function code, I need to not only upload the full dll and documentation, now the source code also which adds to my metered time on net. The documentation for these two dlls is large due to examples which are necessary to get an idea as to what it does. I have to find a solution for it which does not force an user to be on net.
Why don't you put the source in a separate file? Btw, how large and what format are your pics in the help docs?

[vcmohan]

Why don't you put the source in a separate file? Btw, how large and what format are your pics in the help docs?

Yes I have yesterday put the sources of TransAll and EffectsMany into seperate zip files and will post these files seperately. I may update them only occassionally.

The examples are jpeg format. I resized them to about 250 X 200 but I lose some detail. For each of a transition function I need 3 or 4 of such pics. There are over 25 functions. In EffectsMany Plugin I have about 30 functions.

[Fizick]

vcmohan,
I think you must create your own simple website (webpages).