There is an FFdshow 5-tap deinterlacer for AviSynth?

[Raziel6969]

Hello :),

I've been testing some anime trailers that i have in VOB, of course there are interlaced, then i tryed with FFshow deinterlacers, and i like the result of:

1. 5-tap lowpass
2. ffmpeg deinterlacer
3. kernel deinterlacer
4. tomsmocomp

in that order of preference.

Do you know an AviSynth version of the first 3 (the last one has), specially '5-tap lowpass'.

Note: I did do search, and the only near reference says:
TRITICAL: You can do the equivalent of ffdshow's 5-tap lowpass deinterlacer by running a 5x5 convolution with the following matrix with a divisor of 8:
{ 0 0 -1 0 0
0 0 2 0 0
0 0 6 0 0
0 0 2 0 0
0 0 -1 0 0 }
libpostproc also has another 5 tap deinterlacer with these taps
{ -1, 4, 2, 4, -1 } which is more like linear interpolation then blending as the one above. Hopefully this is the correct deinterlacer you were talking about.

I don't know how to this in AviSynth :(, someone knows?
Thanks in advance.

Bye

[scharfis_brain]

the is an AVISynth-internal function called
generalconvolution()

it will do what you want.

[tritical]

Unfortunately, generalconvolution only supports rgb32... so if you are going to use it on rgb32 material (or don't mind converting to rgb32) then it will work for this. Otherwise, you'll need to use something else. If your clips are yv12 you could use yv12convolution from masktools. I don't think there is a general 2d convolution filter for avisynth that can work in all colorspaces, but I could be wrong (if there is, someone please correct me).

Also, the

{ -1, 2, 6, 2, -1 }

tap filter is indeed the 5 tap lowpass deinterlacer. It is a blending deinterlacer and the kernel should be applied to every line.

The

{ -1, 4, 2, 4, -1 }

tap filter is the ffmpeg deinterlacer. It is an interpolating deinterlacer so the kernel should only be applied to every other line (even lines or odd lines depending on which field you want to keep untouched).

[Leak]

3. kernel deinterlacer
Do you know an AviSynth version of the first 3

*cough* (http://www.neuron2.net/kerneldeint/kerneldeint.html)
or
*cough* (http://forum.doom9.org/showthread.php?t=81322)

[foxyshadis]

Tried using DirectShowSource and setting ffdshow as the default handler with all the filters you want enabled? That's the method I've settled for to use ffdshow filters in avisynth, since there's no FFDShowSource that I know of. (I want, because the dual-proc server 2003 box won't use ffdshow except by feeding it a custom-made graph.)

[Raziel6969]

Hello,

Thanks for the quick response.
I've been doing a little test:

MPEGSource("F:\- ISOS\RETURNER\VIDEO_TS\VTS_07_1.VOB")
ConvertToRGB(interlaced=false)
GeneralConvolution(0,
"0 0 -1 0 0
0 0 2 0 0
0 0 6 0 0
0 0 2 0 0
0 0 -1 0 0 ")
AvsTimer(frames=50, name="Time")

and other test with the same clip, with LeakKernelDeint.
(The resulting data is average of whole clip)

Here are the results:
no filtering: 177.94
LeakKernelDeint(1): 97.38
GeneralConvolution {above script}: 9 [is too slow]

- I tried to the yv12convolution with MaskTools, but i can't figure out :(
- I tried DirectShowSource but every time i start the test, vdub freeze :(

Note: To Leak: I don't know that kernelDeint from FFDshow is the same at AviSynth. Now, i know, because the setup is the same :P (sorry)

There is another way to do '5-tap lowpass'? a speedy way?
The same for ffmpeg deinterlace

Bye

[tsp]

YV12Convolution("1","-1 2 6 2 -1",8)

[Video Dude]

If you want to use FFDShow's filters in AviSynth, use ffavisynth and put

FFDShow(preset="default")

in your script. Just save all your settings as a preset.



ffavisynth - ffdshow preprocessing filter for avisynth
http://forum.doom9.org/showthread.php?t=85447

[Didée]

@ tsp


YV12Convolution("1","-1 2 6 2 -1",8, U=3,V=3)

to not loose chroma, if I'm not mistaken.

[tsp]

int Y = 3, U = 3, V = 3;
is the default values for YV12Convolution (take a look at the file MaskTools\Interface.cpp for the default values for the different filters in MaskTools)

[Didée]

Ah, okay. YV12Convolution I use more seldom, and MaskTools current distribution of default modes for U & V over the various filters is not too consistent ...

[tsp]

and even worse the defaults are poorly documented. You will have to look at the source code to figure out what the defaults are.

[Raziel6969]

Hello,

I did some tests with a little modification of my script (only to change deinterlacer), and test the speed, and quality to my eyes. Here are my results (Test: Cowboy Bebop Movie trailer, inside RETURNER DVD NTSC Spanish).

Note: Bigger numbers, faster filter :)

Avg. Deinterlacer
===============
178 no filtering.
111 TomsMoComp(-1,3,0)
97 LeakKernelDeint(1)
25 YV12Convolution("1","-1 2 6 2 -1",8) ;5-tap lowpass?
23 YV12Convolution("1","-1 2 6 2 -1",8, U=3,V=3)
23 YV12Convolution("1","-1 4 3 4 -1",8, U=3,V=3) ;ffmpeg deinterlacer !no
9 GeneralConvolution {script from post #6}

With FFDshow:
484 FFmpeg deinterlacer
174 5-tap lowpass
91 TomsMoComp: Search_effort=3 Vertical filter=no
97 Kernel deinterlacer: Threshold=10 Sharp=no Two_way=no Map=no

Conclusion:
If the filter exist in AviSynth, is faster. The Convolutions is slower than FFDshow versions, YV12Convolution is faster than GeneralConvolution (why is so slow). The FFmpeg deinterlacer equivalent to AVS doesn't interlace (?).
I did see at the interlaced frames some 'stair effect' like aliased lines at the edge of draw with TomsMoComp, FFmpeg deinterlacer, 5-tap lowpass; LeaKernelDeinterlace shows diffrent, some like weak interlaced lines inside the dark areas.

Why FFDshow filters 5-tap lowpass and specially FFmpeg deinterlacer are way too faster than AviSynth filters?

All these filters retain resolution (to my eyes), or am i wrong?

Bye, Thanks for the tips.

[tritical]

I'll answer what I can.

1.) The ffdshow versions of ffmpeg/5-tap low pass will be much much faster because they are mmx optimized for that specific operation. Even if they were only plain c code they would run much faster than similuating the operation with a general convolution filter because it has to support all kernel lengths and values instead of being optimized to that specific case.

2.) General convolution has to process more data than yv12convolution (RGB32 vs YV12) and yv12 convolution is separated (horizontally/vertically) while general convolution is not (helps a lot if one direction is "1").

3.) YV12Convolution("1","-1 4 3 4 -1",8, U=3,V=3) ;ffmpeg deinterlacer !no

First, the taps need to be { -1 4 2 4 -1 }, and second, you can't directly use yv12convolution because that kernel should only be applied to every other line and not every line (it is an interpolater). You could do something with separatefields()/weave() to weave back one of the fields untouched if you want to simulate it with yv12convolution.

4.) All these filters retain resolution (to my eyes), or am i wrong?

It depends on what you mean by resolution :p. 5 tap low pass just blurs everything... it doesn't pass a single pixel through untouched. ffmpeg will pass one field through as is and basically linear interpolate the other (its interpolation does include some information from the field that is thrown away, that does not always improve things though). kernel keeps one field untouched and tries to weave pixels in static areas and interpolate in only moving areas... its interpolation includes some information from the lost field. The left over combing you noticed is from weaving in moving areas. Tomsmocomp passes one field through and then uses very simple motion compensation that should identify static areas as well as very small movement (only 1 or 2 pixels to the side on default setting)... in moving areas that it can't comp it will use a form of ela interpolation. So which you should use depends on what you want most (no residual combing, full detail in static areas, etc...)

5.) All of the interpolation types mentioned and used by the above filters... 5 tap/ffmpeg/kernel will produce some aliasing around diagonal lines/edges. Tomsmocomp's ela interpolation will do better in this area, but can produce artifacts in some circumstances that the others might not.

Finally, since when is cowboy bebop interlaced?

[Leak]

97 LeakKernelDeint(1)
[...]
With FFDshow:[/COLOR]
[...]
97 Kernel deinterlacer: Threshold=10 Sharp=no Two_way=no Map=no

Are you sure you didn't mix up your numbers there? I'd be surprised if the C-only version of KernelDeint in ffdshow was as fast as my MMX optimized version... :eek:

[kassandro]

ffmpeg will pass one field through as is and basically linear interpolate the other (its interpolation does include some information from the field that is thrown away, that does not always improve things though). kernel keeps one field untouched and tries to weave pixels in static areas and interpolate in only moving areas... its interpolation includes some information from the lost field. The left over combing you noticed is from weaving in moving areas. Tomsmocomp passes one field through and then uses very simple motion compensation that should identify static areas as well as very small movement (only 1 or 2 pixels to the side on default setting)... in moving areas that it can't comp it will use a form of ela interpolation. So which you should use depends on what you want most (no residual combing, full detail in static areas, etc...)

If you look closely, the ffmpeg deinterlacer first interpolates the other field by simple vertical bob and then uses the lost field as a vertical sharpening vector for the interpolated field to compensate for the vertical blurring which is always associated with deinterlacing. At first glance a brillant idea, but then, if there is motion, the sharpening vector, essentially the second derivative in the vertical direction, is taken from a very different location and this simply doesn't make sense. (Leak)kerneldeint uses also five interpolation points like ffmpeg, but the weight of the lost field is much smaller (in the sharp case). Probably it uses also the lost field for a slight vertical sharpening, but I don't have the details available. All in all I personally reject the idea of using five vertical interpolation points.

[tritical]

Yes, kernel uses the extra points for the same reason as ffmpeg though they do have less weight overall. With kernel, sharp=false (one way), you get { -1 8 2 8 -1 } ... sharp=true uses 9 vertical interpolation points. As I said it doesn't always make things better but sometimes worse. It will help in static areas, that's why to me it only makes since to use such a kernel with deinterlacers that don't try to detect motion, in which case it can give better vertical detail in static areas. The original patent that kerneldeint is based off of never included any motion detection and obviously ffmpeg doesn't detect motion either. That being said, motion detection (especially on the per pixel level) is never 100% accurate and sometimes the gain in static areas (or very near static areas) from using such interpolation outweights the negatives in moving areas.

[kassandro]

I agree and I am now gentler towards five point interpolation. In static areas it is clearly superior to the 2 point interpolation (simple bob). In motion areas the 2 point interpolation is more correct, but it is vertically less smooth than the five point interpolation, which in motion areas still looks reasonably sharp, because the lost field only enters the interpolation as a second order difference, which usually fairly small due to cancellation. One question remains: what is the mathematically optimal weight of this second order difference within the interpolation?

[tritical]

One question remains: what is the mathematically optimal weight of this second order difference within the interpolation? You're smarter than I am, perhaps you can tell me the answer when you figure it out :p. Seriously though, what exactly would you be trying to optimize when selecting the weights?

[kassandro]

You're smarter than I am, perhaps you can tell me the answer when you figure it out :p. Seriously though, what exactly would you be trying to optimize when selecting the weights?
Perhaps "mathematically optimal" is not the right word. Rather I should have said "mathematically correct". To this end, let -2,-1, 0, 1, 2 be the five interpolation points. Then the points -2, 0, 2 are from the lost field, while the points -1, 1 are from the field which should be preserved. In the sequel f(x) will be a function operating on these five points. Let a(-2), a(-1), a(0), a(1), a(-2). If we impose the following conditions on these weights:

1. a(-2) + a(0) + a(2) = 0
2. (a(-2) + a(-1) + a(0) + a(1) + a(2))*f(0) = a(-2)f(-2)+ a(-1)f(-1) + a(0)f(0) + a(1)f(1) + a(2)f(2) for any polynomial f(x) of degree <= 3

The first condition is necessary such that combs are removed. Now if the weights a(-2),..., a(2) satisfy both conditions, then there exists a constant c such that a(-2) = a(2) = -c, a(-1)=a(1)=4c, a(0)=2c. Thus the ffmpeg deinterlacer is the only one, which satisfies the above two conditions. Of course, as we agree, this interpolation is not appropriate in motion areas, but the influence of the lost field is reduced enough such that the artifacts are not excessive. On can reduce the motion problems of the ffmpeg deinterlacer by using the appropriate average of two consecutive lost fields. The ffmpeg is most suitable for motion compensated deinterlacing. The lost field is simply replaced by the motion compensated lost field. Because the influence of the lost field is reduced, motion compensation errors cannot be disastrous. Finally, why should this interpolation be restricted to the vertical line? One can do it for the two diagonal lines through a pixel as well and select that line for which the sharpening vector has minimal absolute value.
When I have time I will make a plugin which embodies all the above ideas and some more along this line.

[Raziel6969]

I'm happy, because this thread can start a new deinterlacer plugin :)

@Tritical:
The "4.) All these filters retain resolution (to my eyes), or am i wrong?"
was because the faster filters at ffdshow seems to discard 1 frame, then halve the resolution, the non smoother deinterlacers appears full of 'stairs'
(this is a example of 'artifacts'?)

@Leak:
I'm gonna re-do my test with AVS (no FFDshow), kernerdeinterlacer vs leakerneldeinterlacer, and see what happens :)
[maybe the difference is because one is AVS native filter, and the other is the FFDshow version?]

Thanks for the wise words of Kasandro & Tritical

Bye.

[kassandro]

While the ffmpeg deinterlacer is essentially classical bob + vertical sharpening, the "5-tap lowpass" deinterlacer with weights -1, 2, 6, 2, -1 is the ordinary vertical blender + vertical sharpening. The "5-tap lowpass" deinterlacer has a similar mathematical justification as the ffmpeg deinterlacer. With the notation of last posting the "5-tap lowpass" deinterlacer is uniquely determined by the following two properties

1. a(-2) + a(0) + a(2) = a(-1) + a(1)
2. (a(-2) + a(-1) + a(0) + a(1) + a(2))*f(0) = a(-2)f(-2)+ a(-1)f(-1) + a(0)f(0) + a(1)f(1) + a(2)f(2) for any polynomial f(x) of degree <= 3

Thus the interpolation condition 2 is exactly the same. Condition 1 is necessary to remove combs by blending. Again one can do the smae with the three diagonals and optimise over all three lines through the pixel. Motion compensation and temporal blending makes much less sense for the "5-tap lowpass" deinterlacer, though.

EDIT: sorry, this was a quick shot. condition 2 fails for the weights -1, 2, 6, 2, -1 (take f(x) = x^2). On has to take the weights -1, 4, 10, 4, -1 to fulfill conditions 1 and 2.