Advice on deinterlacing (interlaced foreground over static background)

[thewebchat]

These are the results I've managed to obtain so far (left is processed, obviously):
Problem area 1 (http://img41.imageshack.us/img41/1311/almostthere.png)
Problem area 2 (http://img407.imageshack.us/img407/3467/almostthere2.png)

The script I am using is this:
Average(op_ep03,0.125,op_ep04,0.125,op_ep08,0.125,op_ep09,0.125,op_ep10,0.125,
\ op_ep11,0.125,op_ep12,0.125,op_ep13,0.125)
BiFrost(altclip=SSIQ)
Crop(8,0,-8,-4)
adapt = TDeint(edeint=nnedi2,mthreshC=40,mthreshL=40)
deinterlace = SelectEven(MCBob)
firstrepair = Repair(deinterlace,adapt,24)
finalrepair = Repair(firstrepair,adapt,24).MergeChroma(firstrepair).MDeGrain(method=2)
finalrepair = MergeChroma(finalrepair,finalrepair.AddBorders(0,0,0,4).aWarpSharp2.Crop(0,0,0,-4))
deinterlace.Trim(0,136)+finalrepair.Trim(137,251)+deinterlace.Trim(252,589)+finalrepair.Trim(590,704)+
\ deinterlace.Trim(705,948)+finalrepair.Trim(949,987)+deinterlace.Trim(988,1005)+finalrepair.Trim(1006,1044)+
\ deinterlace.Trim(1045,1062)+finalrepair.Trim(1063,1101)+deinterlace.Trim(1102,1119)+finalrepair.Trim(1120,1158)+
\ deinterlace.Trim(1159,1609)+finalrepair.Trim(1610,1839)+deinterlace.Trim(1840,0)

The problem with the sample I uploaded is that although the background is completely static, no deinterlacer that I've tried realises that fact. TDeint and MCBob both create "blur bubbles" around the edges of deinterlaced objects. TGMC destroys the entire frame. My idea to deal with this was to start with a blurred clip from MCBob and then progressively recover details using Repair against a TDeint clip with high enough mthresh to cause ghosting. While this is reasonably successful in recovering the background, it causes heavy artifacts at edges (because of the influence of the TDeint clip). I have tried to postprocess those out without destroying too much of the detail that I spent so much time recovering, but the results are limited.

Any advice on creating a method that is 1) not as unbelievably slow, and 2) higher quality would be greatly appreciated.

A sample is available here (http://www.mediafire.com/?njmmjrwzazx). It is compressed with H.264 lossless and equivalent to the script at the Average() code line. Any requests for "original sources" will be denied because I am not splitting and uploading 8 different files. The call to Average is done to reduce the noise present in the source by combining 8 copies of the same clip from several episodes.

(Also, sorry for posting so many threads today.)

[2Bdecided]

I'm impressed by how well you've handled this already!

My best ("simple") attempt was...
assumetff()
converttoyv12(interlaced=true)
tgmc=tempgaussmc_alpha3()
yadifmod(mode=3,edeint=tgmc)...and I think that's inferior to what you've done.

I'm stunned by how badly tgmc handles this - unless I'm doing something stupid wrong, it completely misses the fact that huge parts of the frame are completely static!

The yadif spatial check has the same problem, so I used temporal only (mode 3). It's the fine noise-like details in this clip that kill these deinterlacers.

Cheers,
David.

[scharfis_brain]

this is something that might be solved with a very outdated function of mine: bobmatcher.

this deinterlacer is able to select appropriate forward and backwards weaving to do the deinterlacing.

but I didn't try it on this source.

[Terka]

What about masking it manually in some video editor?
(Deinterlace only the interlaced part)
the second part is simplier - non moving mask could be done in
avisynth also?

[thewebchat]

I tried BobMatcher, and it unsurprisingly performed similar to TDeint. This motion mask idea sounds interesting, however. How do I implement it?

[Didée]

First off: that's a taff source, definetly a nightmare for any bob or deinterlace filter. And the solution you found seems to work out quite good, indeed!

However, let me elaborate a bit on TGMC: the problem is that y'all are using it wrong for this case. TGMC is capable to handle this source more or less sufficiently !

I'm stunned by how badly tgmc handles this [...] it completely misses the fact that huge parts of the frame are completely static! Yes, but no. TGMC doesn't "miss" the fact of static areas. Per its methodology, it doesn't know about such, and it doesn't care ... for a simple reason: it doesn't need to do so.
The problem lies elsewhere, read on.

TGMC destroys the entire frame. ... when used with default settings, which are unsuited for this source. With adapted settings, it handles this case fairly well. (Not all too impressing, but fairly well.)

Just for fun, try this:tempgaussmc_beta1(1,1,0, 0,0,0, "nnedi",sharpness=4.0,SVthin=8.0,Sbb=3,blocksize=8

# tempgaussmc_beta1(1,1,1,0,5,0,"nnedi",sharpness=4.0,SVthin=8.0,Sbb=3,blocksize=8 # try this, too - Ah, that's much better, isn't it.

The problem really is the extreme (vertical) sharpness, an indicator for the total absence of any antiflicker-lowpassing. Without digging much into detail, it's this characteristic which causes the spatial-repair part of TGMC to fail ... it's quite similar to Yadif's problem, see the difference between mode=1 and mode=3 in Yadif/Mod. There's so much of "detail" that to a filter looks pretty much like combing=motion, that's why all filters are having a hard time. Moreover, this combing-like detail is also irritating NNEDI, it (necessarily!) creates false interpolations on these detailed parts.

Now, completely disabling the rep0/1/2 options of TGMC surely is not "generally recommended", no. But in this case here it is absolutely necessary. Well, one might try a 'lil bit of safety-repair, see the 2nd code line above.


The remaining problems (covering and revealing areas, mostly) is a problem that MVTools are not able to handle sufficiently. Which is a bad starting point for script-filters that rely on MVTools as a ground stone ...

The most promising way would be in the direction of scharfi's suggestion. Check for static, independently for forward- and backward-in-time, and do a plain fieldmatch with the better one when forward / backward are disagreeing largely. (And *only* then.)

[thewebchat]

Could you give me any advice as to how to start implementing this suggestion, Didee? The problem seems to be 1) finding the area of the frame occupied by the interlaced object and then 2) interpolating based on one of the fields of the frame and then 3) replacing the area occupied by the other field with weaved data from a prior/future frame. The problem I'm having is that even #1 is very hard to do. I have not yet found a reliable way to isolate those area programatically.

(Sorry, I can't write filters. I only use them.)

Now, an alternate solution would be to remake these scenes using Generic 3D Rendering Application 3 since the background (should) be fully recoverable with enough photoshopping and the movement of the overlay is (somewhat) trivial.

Edit @ Didee: Your modified TGMC settings already outperform my hackjob. If you could get it even better (ideally, background same as input), that would be awesome.

[Didée]

Sorry, but my motivation to dig for solutions to such complicated special-case stuff is infinitesimally small these days ...

You could have a look into the static-detection part of MCBob (which evaluates both directions together, IIRC there's no distinction between forward- or backward- motion). Or toy a bit and see if the TMM masking (by tritical, somewhat based on MCBob's method) can be used.

The basic idea still holds up: whether a pixel changes by amount "th" to its temporal neighbors, this tells nothing about motion or not-motion. But: if we have a local signal with peak "mm" (local min-max span), then it is safe to assume that a pixel change of, say, 20% of that min-max-span can be considered as "not-motion". (Still needs at least one instance of temporal evidence, though. Interference of moving patterns, and thelike)

[thewebchat]

That is unfortunate.

For the record, I have decided to use Didee's modified TGMC settings, as any more time spent on this may lead to an increased risk of cancer.

[2Bdecided]

The problem really is the extreme (vertical) sharpness, an indicator for the total absence of any antiflicker-lowpassing.Yes, it would be interesting to watch this on an interlaced CRT - I bet it would flicker like mad.

Whereas for most normal sources, the perceived (artefact-free) quality of an interlaced CRT (where your brain as effectively doing the deinterlacing!) is something that most deinterlacers could only dream of approaching. tgmc comes pretty close - usually better IMO.

Cheers,
David.

[Terka]

where to find the bobmatcher?

[thewebchat]

Terka, this is the copy I found:
http://forum.doom9.org/showthread.php?t=71510&highlight=bobmatcher

It requires some pretty archaic filters (lol @ mpeg2dec1) and seems to be a primitive version of TDeint.

[scharfis_brain]

ah, wait. bobmatcher() is the wrong function. indeed it is a kind of predecessor to tdeint. (some ideas of bobmatcher found their way to tdeint).

I really meant intellibob() which is a deinterlacer that only creates the absolute minimal motionmask at the absolute risk creating motion artifacts.
But this scripted function also is pretty old and I didn't touch it for years, cause it was found useless for most stuff.

But I guess the original version of intellibob with its minimalist motion masking has gotten lost within the years :(
There only is a version left in these forums which already uses TCombmask for mask creation. So it's pretty useless for this special case...