Decomb - Telecide with pp by external deinterlacer?

[zettai]

This is a post for discussion regarding a potential addition to the excellent Decomb filter.

If I wanted to use a different deinterlacing than the internal post processing of Telecide, I'd currently have to do Telecide(post=false) and add a deinterlacer later in the script.

Now don't get me wrong, I think that FieldDeinterlace is a pretty good deinterlacing routine but there are other deinterlacers out there that could be used and may have advantages in certain situations.

However, it seems a bit of a waste to have the external deinterlacers process both combed and progressive frames when you could potentially just call your fave deinterlacer to do the work once you've worked out that a frame is combed after Telecide.


Of course, this makes numerous presumptions about the way decomb works and it is possible that feeding the interlaced frames into an external deinterlacer would not work - especially if the deinterlacer requires a stream rather than a frame (likely).

So, in short, I've no idea if this is even remotely possible but if the structure of Decomb was suited to such an operation it would be a potentially useful addition. Discuss :)

[neuron2]

As currently designed, Telecide() user per-frame deinterlacing and cannot do inter-frame based deinterlacing. What per-frame deinterlacers or features did you have in mind?

Note that both TomsMoComp and Xesdeeni's port of Thalins's Deinterlace-Smooth require inter-frame comparisons.

What you suggest is certainly possible, but you'd have to have some compelling reasons. And even then, I'd be more likely to enhance the existing deinterlacing as required. What enhancements are you thinking of?

The other point I would make is that the other deinterlacers should implement the functionality by which only frames detected as combed are touched. Then this whole issue would be finessed.

[stickboy]

Incidentally, should you deinterlace after Decimate() or before? I can think of a few hypothetical cases where the results might be different, but I don't think the differences wouldn't be too significant anyway.

[zettai]

Yeah, this is where I figured the problem would occur. All my fave deinterlacers are inter-frame.

What if instead of sending a frame to be deinterlaced you could copy a frame from a version of the script that was deinterlaced so that

a = AVISource(args).TomsMoComp(args)
Telecide(post=a)

I'm guessing that this would work but be just as cpu-intesive (if not more) than doing the deinterlacing after telecide. It would have the advantage of ignoring frames that don't have combing though.

Yeah, so I guess you are right in that if current deinterlacers could ignore non-combed frames then that would be the solution. Another occasion where filter talkback could be useful?

(BTW, I'm not advocating changing the current per-frame deinterlacing method of Decomb, as I can't think of any ways to improve it without knowing more about how it works, I'm just discussing possibilities)

[neuron2]

What if instead of sending a frame to be deinterlaced you could copy a frame from a version of the script that was deinterlaced so that

a = AVISource(args).TomsMoComp(args)
Telecide(post=a)Telecide needs to see the original frames to work properly. It does its field matching by assembling frames with the least combing. If you remove all the combing first with TomsMoComp, you'll get random results at best.
(BTW, I'm not advocating changing the current per-frame deinterlacing method of Decomb, as I can't think of any ways to improve it without knowing more about how it works, I'm just discussing possibilities)You've already stated it is not your "fav", so there must be something about it that can be improved. What is that?

What do you need to know about how it works? It detects combed areas by looking for spatial frequency of 2 and then either blends or interpolates in those areas. It is based on Gunnar Thalin's area-based deinterlacing approach.

Inter-frame based deinterlacers are conceptually wrong and can introduce ugly artifacts. Yes, I know I wrote one, but now I know better. :)

There is a middle way between the two that involves comparing the fields as if they were frames (change implies "motion"). But experiment reveals that it detects more false motion than the approach of looking for spatial frequency 2 areas in the weaved frame.

[neuron2]

@stickboy

Given a wide enough threshold, a deinterlaced frame might be (correctly) determined to be a duplicate. That would be precluded if you changed the order. I can't see any case where it would be advantageous to deinterlace after decimating. It might improve performance if there were a lot of combed frames slipping through, but not by much.

[-h]

Inter-frame based deinterlacers are conceptually wrong and can introduce ugly artifacts. Yes, I know I wrote one, but now I know better. :)

I wouldn't say conceptually wrong, some implementations produce extremely good results, such as this (http://www.softlab.ntua.gr/~tourapis/papers/4310-101.pdf) and this (http://www.ics.ele.tue.nl/~dehaan/pdf/47_tr_on_ce99.pdf) (indeed the latter resulted in a broadcast-quality hardware implementation). They are capable of better quality than inteprolative filters, since they use motion-compensated actual image data instead of interpolated data from the current field.

-h

[zettai]

Originally posted by neuron2
Telecide needs to see the original frames to work properly. It does its field matching by assembling frames with the least combing. If you remove all the combing first with TomsMoComp, you'll get random results at best.

I think that either me or you isn't understanding what we are saying. It's probably me :)

What I was getting at was that Telecide works as normal but when it cannot reconstruct a progressive frame it would usually give this to the deinterlacer. Instead, for that particular frame, it copies the frame from the deinterlaced source instead of doing the per-frame deinterlacing.

All the progressive frames are constructed exactly the way they normally would, its just the combed ones that are copied from the deinterlaced version.

But anyway...

As for my fave, I don't like blending in my deinterlacing - if I did, I'd have no problem with just using Telecide()

Lets take some possible deinterlacing methods:

Method1:
Telecide(threshold=12,dthreshold=5,blend=false,chroma=true)

Pretty aggressive, but does the job. I find the slightly lower than default threshold useful for interlacing on fades in ntsc fades in mixed sources - however, agressive settings like this has an obvious side effect...

Now, imagine that there is a fully interlaced fade on an anime image. What happens is that the fine detail such as the lip lines etc become pixellated as they are deinterlaced. When this happens a lot (and it is a more general problem in telecide post-production in my experience) then it does become quite noticable.

This is less true of

Method2:
Telecide(post=false)
SmoothDeinterlace(tff=true,doublerate=false,lacethresh=5,edgethresh=20,staticthresh=0)

This is a high threshold for detecting if a frame is interlaced but for telecide post-production seems to work quite well. Some of the individual frames of method1: do seem to have a 'cleaner' interpolation but method 2 is decidedly smoother on fine lines and better at dealing with fine interlacing on things like interlaced fades to white (as discussed above)

I admit, to compare the two, there's not much in it. Sometimes I prefer one and not another depending on the source but I do like the smoothdeinterlace as a general purpose deinterlacer. I'm not really trying to prove anything with these differences, they're just ones I've seen, but as you asked I thought I'd mention it.

I don't know what smoothdeinterlace does, but I do know that it is gentle on lines in the anime sources that I've tried it on so would be tempted to use it occasionally as telecide post processing.

I don't use deinterlacers much but they are useful when required :)

[trbarry]

GreedyHMA does take the approach of trying to do IVTC on a frame and then calling a (interframe) deinterlace method when that fails. It is more general purpose than TomsMoComp for this reason and can probably better handle mixed clips that are mostly telecined but sometimes poorly. But note for telecined material I think that Donald's Decomb is probably now more sophisticated. Some of the IVTC code in GreedyHMA was only added after I ported it over to Avisynth from DScaler and it has nowhere near the design or testing effort of either Decomb or the DScaler code.

TomsMoComp will only do a better job when you are pretty sure you have pure video source. It is optimized for this purpose only, and is really designed to be called in DScaler when video material has been identified. But if you have telecined material with some bad edits or cadence changes you are probably better off with Decomb or GreedyHMA.

I guess the bottom line of all this is there is yet no free lunch. You still have to know what type of clip you have and use the best tools accordingly. Someday I may try to update GreedyHMA to call TomsMoComp for interlaced material but I probably won't get to it soon.

I do however stand firmly behind the belief that there is much useful info that can be used during interframe deinterlacing. But as the amount of motion goes up the amount of interframe data used should go down. For very fast motion you are back to some sort of intra frame blend. TomsMoComp does this. And if you set the Search Effort parm = 0 it becomes totally intra-frame, but with a slighly more elaborate blend than most others.

- Tom

[neuron2]

-h wrote:
I wouldn't say conceptually wrong...Yes, you're right when considering more advanced approaches that include multiple frame data. I was referring to the simple-minded approach of just comparing a pixel to its corresponding pixel in the previous frame and then thresholding.

Thank you for the links. They are quite interesting.

[ErMaC]

neuron - I think what zettai is getting at is if there's a way to replace FieldDeinterlace's part in Decomb with a different function. I assume somewhere in the code there must be the equivalent of

if (frame == combed) FieldDeinterlace(frame)

and instead replace that with

if (frame == combed) SmoothDeinterlace(frame)

or whatever. The same way MAM used to take separate arguments for various smoothers, if Telecide could take an argument for a deinterlacer to call (which would have FieldDeinterlace as its default, naturally) then that would be what zettai was looking for.

I agree with him in that I can't think of a way to improve what FieldDeinterlace does, because while it's goal is the same (deinterlacing) its method is totally different than some others. What we're looking for here is a method of specifying ourselves what kind of deinterlacer to use because it allows us to experiment manually with the various ones and make a call on what we think looks best.

FieldDeinterlace is great - I use it very often on Anime material that I've modified and Telecide screws up on, for instance:

function norm(clip)
{
return clip.Telecide(blend=false,chroma=true,guide=1).Convolution3d(1, 8, 8, 8, 8, 3, 0).Dup(copy=true,threshold=2,show=false,maxcopies=2)
}
function film(clip)
{
return clip.FieldDeinterlace(blend=false,chroma=true)
}
video=AVISource("D:\AMVs\Final DV\CloserV2-HY-Widescreen.avi").ConvertBackToYUY2().Crop(8,0,-24,0)
v1=video.Trim(0,326).film
v2=video.Trim(327,965).norm
v3=video.Trim(966,974).film
etc...

BTW, "film" was because I'd used an Old Film Effect (Sepia) filter on the video there in Premiere which rendered at 60fps and for some reason Telecide would skip backwards temporally as it tried to figure out how to fix it and continued swapping dominant fields, or so I would guess. This was before the manual override function.

...but there are times when I think another algorithm would work better, and not because of any inherent defend in FieldDeinterlace, but because it's basic philosophy is different than other deinterlacers.
FD is like my microwave. It's great, and it cooks stuff fast and does its job well. But just because I can't bake a cake in it doesn't mean it's inferior to my oven (hell, my oven doesn't make Easy Mac). I just have to know which tool to use when.

[neuron2]

@ErMaC

Yes, I understand the point. But to invoke a foreign filter on a given frame only is not easy and would require support from the filter. I am not aware of any easy way to accomplish it. That means your proposed implementation is out. If support is required from the filter, it would be better to just have the filter implement the idea of not touching non-combed frames.

But zettai's idea of passing a second deinterlaced clip into Telecide and selectively using its output might be possible. I will look into it for the next release of Decomb.

[trbarry]

But zettai's idea of passing a second deinterlaced clip into Telecide and selectively using its output might be possible. I will look into it for the next release of Decomb.

That actually might work quite well. At first glance it would seem to have a lot more overhead because it looks like things would be both Telecide'd and deinterlaced. At least that's what I first thought.

But that is not completely true.

Avisynth works in "pull mode" where only those requested record numbers actually get processed. So if Telecide decided (Teledecided ;) ) to only deinterlace 5 records then that is all the processing that would be needed from the deinterlaced clip.

- Tom

[neuron2]

Originally posted by trbarry


That actually might work quite well. At first glance it would seem to have a lot more overhead because it looks like things would be both Telecide'd and deinterlaced. At least that's what I first thought.

But that is not completely true.

Avisynth works in "pull mode" where only those requested record numbers actually get processed. So if Telecide decided (Teledecided ;) ) to only deinterlace 5 records then that is all the processing that would be needed from the deinterlaced clip.
Sounds correct to me. This is beginning to sound like an easy enhancement. :eek: