50 fps deinterlacers

[Terka]

looking for good speed/eficiency 50fps deinterlacer.
tried:
mvbob -really nice results, but slow :(
smoothdeinterlace in vdub, via avisynth got error, now downloaded the right version for avisynth 256
any sugestion? no 50fps test found.
:thanks:

[GodofaGap]

I guess leakkernelbob would be the first thing that comes to mind. But it is always difficult to estimate what other people mean with good speed/efficiency. :)

[foxyshadis]

Give them all a shot, it won't take too long. (Compared to full encoding.)

mvbob
securedeint (part of mvbob, slow but faster than mvbob)
tdeint+eedi2 (slow)
tdeint (moderate speed)
tomsmocomp (moderate)
leakkernelbob (very fast, sharp or normal)
smoothdeinterlace (which you tried)

Perhaps fielddeinterlace and dgbob as well, or even bob/ibob, if you want to be complete about it; they're fast but not so high quality.

[Didée]

BTW, for a quick'n'dirty dumb-bob without bobbing flicker on static parts, one can try

source.bob(-.2,.6).reduceflicker(strength=1)

It *may* produce occasional motion artefacts in spots where repeating patterns run into interference (which is why scharfis_brain once dressed me down for proposing this). ;)

So, surely not a HiQ solution, but fast ... for a *cheap* solution, I quite like it. And it doesn't suffer from "threshold disease".

[scharfis_brain]

ermmm. change *may* into *will*.
like kerneldeint/bob also *will* show these 'occasional' motion artifacts as residual combing.

currently only these deinterlacers are able to avoid them:

tomsmocomp
smoothdeinterlace
securedeint
mvbob

tdeint and kerneldeint only are free of such artifacts with fieldblended or telecined footage.
(you can avoid the artifacts with them, too. But this requires disabling their motion adaption)

[Mug Funky]

a low enough threshold tends to help, but then you get dotty horizontal lines where neighboring pixels are either deinterlaced or left alone, leaving a sharp noise. i see this on progressive-scan DVD players too.

i think kernelbob/leakkernelbob could be slightly modified to expand it's comb mask out 4 pix on each side, and possibly to also check previous and next fields. that would make it work well for everything but wrong field-order footage and very fast scrolling with no motion-blur. both of which we can deal with when they come along...

an adaptive threshold would be good too - something that sort of rides a couple of values above the noise floor of the video, so noisy stuff gets dealt with better (and faster), and clean stuff doesn't suffer either.

[Didée]

ermmm. change *may* into *will*.
No, I won't. This would make it sound as if there would be artefacts as soon as there is any motion. And this would be an exaggerated statement.
like kerneldeint/bob also *will* show these 'occasional' motion artifacts as residual combing.
motion artefacting is a matter of inter-frame texture interferencing. residual combing is a matter of thresholds. How do you manage to put those in the same hat?

currently only these deinterlacers are able to avoid them: Which, now - motion artefacts or residual combing? tomsmocomp
smoothdeinterlace
securedeint
mvbob
TomsMoComp - only safe when used as dumbbob. As soon as searcheffort isn't zero, artefacts may occur.
SmoothDeinterlace - haven't tried in ages, but isn't it using thresholds ... ? :)
securedeint - pretty safe in avoiding the occasional motionartefact, at the cost of letting stay quite some bob flicker. (staticmask is too anxious)
MVBob - it's a great idea. But in relation to the processing time that is needed, the current incarnation produces too much artefacts.

(BTW, there's a standing question about complexity ... :) )


an adaptive threshold would be good too - something that sort of rides a couple of values above the noise floor of the video, so noisy stuff gets dealt with better (and faster), and clean stuff doesn't suffer either.
That would be a good option indeed. Some time I tried sth like

edge = c .littleblur .edgemask
deint1 = c .deinterlace(lowthreshold)
deint2 = c .deinterlace(highthreshold)
maskedmerge(deint1, deint2, edge)

and definetly got a positive effect with it. No residual combing in flat areas, no bob flicker on edge areas that are static.
Should be not too difficult to put this as a "one-step" operation into TDeint (instead of tearing the TDeint mule two times over the mountains).

[Mug Funky]

yeah, that's the trick. making it fast...

expanding the comb mask strikes me as not being too computationally intensive as well... it'd both reduce those jagged horizontals and allow the threshold to be (slightly) raised. it can also deal with some of the "holes" that motion-adaptive deinterlacers typically suffer, though it would only deal with ones less than 8 pixels wide (which is most of them anyway, but there's cases that'd still mess it up, like very fast moving text).

ideally i'd like something like what a standards-converter already does... based on the output of the one at work, it has:

- an adaptive threshold - set it to "1" and it'll handle noisy and clean video with equal quality
- an expanded comb mask - this can be seen when thin objects move - a little area around the object is also deinterlaced, which might seem too agressive, but the good thing is it's a clean edge - it goes straight from untouched to deinterlaced without any stray pixels either side being given the wrong treatment.
- i think it also checks 4 fields, so if combing is detected on 1 frame, it can be checked against another to see that it isn't just very sharp static detail.

now, this box still suffers from "holes", but i've only seen it in 2 cases:

- VHS input with such bad tracking that the picture jumped up and down per-field.
- input was in the wrong field-order (this can even happen in hardware if the editor was doing it wrong...)

normal input is handled fine - no artefacts. this is with every kind of source imaginable (except the above 2).

[tritical]

expanding the comb mask strikes me as not being too computationally intensive as well... it'd both reduce those jagged horizontals and allow the threshold to be (slightly) raised. it can also deal with some of the "holes" that motion-adaptive deinterlacers typically suffer, though it would only deal with ones less than 8 pixels wide (which is most of them anyway, but there's cases that'd still mess it up, like very fast moving text).
I don't think that always expanding the comb mask out 4 pixels in each direction is the best way to do it, though it would be fast. Probably a better way would be to only expand outwards if it would fill a gap that is less than 8 (or some other number) pixels to another combed pixel. You could also denoise the mask first... say remove any pixels that don't have at least 'x' neighbor(s) within 'y' pixels.

- an adaptive threshold - set it to "1" and it'll handle noisy and clean video with equal quality
An adaptive threshold would be nice, but the big question is what algorithm are you going to use to choose it :D? Every automatic noise estimation algorithm I've read about/tried usually fails quite often except for automatically generated, zero-mean white gaussian noise and even then it's not perfect.

i think it also checks 4 fields, so if combing is detected on 1 frame, it can be checked against another to see that it isn't just very sharp static detail.
I don't understand what you mean here. Kerneldeint already uses the standard, backwards-only 4 field motion check... or are you even talking about motion?

currently only these deinterlacers are able to avoid them:

tomsmocomp
smoothdeinterlace
securedeint
mvbob
As far as securedeint goes, it isn't 100% guaranteed to be free of them either. It is definitely less likely since it checks more fields, but that doesn't mean it wont. Plus the downside, as Didée mentioned, is that it sacrifices some static areas.

The other thing is that since it uses the motionmask() function, which is equivalent to performing a [1 1 1] blur in both horz and vert directions on both frames and then doing a pixel by pixel abs() subtraction, it is susceptable to other artifacts. Namely, if you have a single pixel tall horizontal line, that blurring can be enough to make the line drop under the threshold. Say you have the following pixel values in two frames and your threshold is 6:

25 25 25
10 10 10
25 25 25

25 25 25
25 25 25
25 25 25

with just an abs() subtraction you'd get a diff of 15 and are over the threshold. However, with the blurring you'd get a diff of abs((10+25+25)/3)-25) = 5 and you're under the threshold.

I've run into that exact problem when trying to incorporate denoising into a new motion-mask algo for tdeint. However, I was using [1 2 1] blurs instead of [1 1 1] because of their ability to eliminate dotcrawl when used within a field. A [1 2 1] is less susceptable to those type of artifacts than a [1 1 1], but it was still visible. What I finally settled on was to average only in the direction of least difference in the current frame (horz, vert, or one of the 2 diagonals). That eliminated the artifacts, but most of the benefits of the denoising were gone as well. It would be possible to use a more sophisticated denoising algorithm.. something like a smart temporal smoother coupled with a smart spatial smoother, but the tradeoff is speed and the requirement of setting the denoising parameters.

[scharfis_brain]

Two weeks ago I had another Idea for a more secure deinterlacer with lesser remnant bobbing.
It consists of three stages:
1) scene detection (which steers a sliding deinterlacing motionmask window)
2) edge based comb masking with a short temporal mask
3) motion based comb masking with a long temporal mask (like securedeint currently does)

1) and 3) should be clear how to implement.

2) is a little bit different.
I found, that deinterlacing holes are most visible in uniform areas. So I think that creating an detail/edge mask of the current field could improve this situation.
So only detailed areas and edges can be weaved. The weaving however will be steered by this edgemask an a short temporal motion mask.

To be really sure that even weak detailed static background are getting weaved method 3) additionally will be used with a long temporal radius.

methods 2) and 3) are steered by method 1) to guarantee clean scene changes without bobbing.

Of course motionmasking has to be done on even & odd fields individually in order to detect static areas properly.

What about this idea?
Again in short words:
- detailed areas and edges are deinterlaced using a short temporal mask
- uniform areas and weak detailed areas are deinterlaced using a loong temporal mask
- temporal masking is steered by scene detection.
uniform areas

[Terka]

2 Didée: could you finish the TDeint:
edge = c .littleblur .edgemask
deint1 = c .deinterlace(lowthreshold)
deint2 = c .deinterlace(highthreshold)
maskedmerge(deint1, deint2, edge)?

[tritical]

The edge masking idea I like. I wrote a simple motion detecter (current to previous frame only) based on Didée's/Mug Funky's ideas a few days ago. Basically, it first builds an edge mask based on gradient magnitude of the gaussian blurred (std = 1.0) image. Depending on the magnitude it adjusts the motion threshold. The motion threshold is controlled using four parameters: t1, t2, e1, e2. If the edge magnitude is <= e1 then threshold is set to t1, if the edge magnitude is > e2 then threshold is set to t2, and if it is inbetween then the threshold is interpolated between t1 and t2. I also added Mug Funky's expansion idea, though I first denoised the mask and only expanded horizontally outwards if it would fill a gap of 8 or less to another marked pixel. The results seemed pretty good. Having a higher threshold around edges definitely seems like a good idea.

@scharfis_brain
Using a long temporal mask for flat areas and a short one for edges might work well. Reading your post I started thinking about why we are weaving in uniform areas to begin with? It risks holes and residual combing and the weaved pixel is really no better than what you would get using simple linear interpolation. It might be useful to try weaving only in detailed areas.

I think scene detection should definitely be part of a motion-masking algorithm. It should look at all combinations behind/ahead/across the current field but be limited by scenechanges (i.e. a scenechange would be treated as though all pixels were detected as moving). I had that as part of the external motion-masking for tdeint that I was working on.

[Terka]

Sharfi, could you add link to newest version of mvbob?
Have more versions, and besides this- on one PC is mvbob working, on my other is not, dont know the reason (probably wrong mvtools version).
Adn second, is it possible to improve mvbobs speed?

[Ferux]

Which method of deinterlacing is used in flat screen TV's?

Is that method superior to those mentioned in this thread?

[foxyshadis]

Some don't even bother; of those that do, the equivalent of IBob, or Bob, or sometimes a kernel bob even simpler than KernelDeint. But no chip available in consumer grade tvs would be able to do anything close to mvbob in realtime, even if the engineers were allowed to make it.

[Trixter]

Some don't even bother; of those that do, the equivalent of IBob, or Bob, or sometimes a kernel bob even simpler than KernelDeint. But no chip available in consumer grade tvs would be able to do anything close to mvbob in realtime, even if the engineers were allowed to make it.

There are circuits that do the equivalent of mvbob in realtime, but they're not cheap enough to be embedded into displays or projectors. They're used in expensive standards conversion rackmount units (these start at $30K)

[foxyshadis]

Well, he said flat screen tvs, not snell & wilcox ASICs.

[Mug Funky]

the Sony DVP-NS50P i use at work does something strikingly similar to kernelbob with a (too high) threshold.

[Ferux]

Thx for the answers, now I know I can't deinterlace my video's with my TV :)