Motion-compensated deinterlacing filter?

[morsafr]

I was just wondering if there was a deinterlacing filter based on that method.
By the way what is the method used by decomb ? Is it a motion-adaptative per-pixel deinterlacing filter ? :confused:

Really weird questions isn't it ! :)

[trbarry]

TomsMoComp is a motion compensated deinterlacing filter (pure video source only, no IVTC). But it only searches a couple pixels left and right.

Setting the SearchEffort parm greater than 5 can make it search more but seems to cause more artifacts. See:

www.trbarry.com/TomsMoComp.zip

- Tom

[Guest]

Originally posted by morsafr
I was just wondering if there was a deinterlacing filter based on that method.
By the way what is the method used by decomb ? Is it a motion-adaptative per-pixel deinterlacing filter ? :confused:

Really weird questions isn't it ! :) Full block-based motion-compensated deinterlacing is not available in any Avisynth filter that I know of. Anyway, a second phase of normal deinterlacing would be required because the motion compensation not only can produce false block matches but also does not deal with rotations, zooms, coverings/uncoverings, etc.

I suppose TomsMoComp() with a small search is similar to traditional spatiotemporal median filter deinterlacers.

As far as Decomb's FieldDeinterlace() is concerned, it doesn't detect motion in any kind of direct way. Rather, it looks for combed areas using what is in effect a kind of spatial notch filter at spatial wavelength of 2 pixels (originally conceived by Gunnar Thalin). Areas that are combed are deinterlaced by interpolation or field blending.

[MfA]

trbarry, why didnt you choose a block based motion search BTW? That the present code would suffer from extreme aperture problems seems to be a given.

It would be nice to have a good block based predictive ME in avisynth, seperate from the actual effect filters such as deinterlacing and denoising.

[trbarry]

MfA -

I first wrote it as an experiment in a heirarchical block based motion comp deinterlacer. But TomsMoComp as released was the part of it that had most of the benefits and could run real time in DScaler.

Unlike for codecs and compression purposes, I think for deinterlacing there seems to be a rapidly diminishing return from searching very far away. For deinterlacing it's maybe better to just not find the best match, regardless of algorithm, if it is not going to be good enough anyway due to changes in size, brightness, orientation, occlusion, etc.

But maybe I just didn't try hard enough. Those were just my initial impressions and I'll probably write other versions.

- Tom

[MfA]

There are a load of televisions out there which use block based motion compensated deinterlacing, and what a load it is ...

BTW I dont think it makes much sense to use variable block size motion compensation, just use 8x8 blocks for everything, even for motion estimation hierarchy can do more harm than good IMO.

[morsafr]

Originally posted by trbarry
TomsMoComp is a motion compensated deinterlacing filter (pure video source only, no IVTC). But it only searches a couple pixels left and right.

Setting the SearchEffort parm greater than 5 can make it search more but seems to cause more artifacts. See:

www.trbarry.com/TomsMoComp.zip

- Tom

I'd like to understand. If you separate the odd and even fields of a frame with SeparateFields() in two consecutive images, your filter :

1) take each pixel of an image constituted by the the odd fields
2) for that pixel, look for the neighboring pixels in the precedent image (the even fields)
3) calculate the amount of motion
4) determine the percentage of inter-field information to use
5) create the final image with the inter and intra-field information

That means that each resulting frame is different from the precedent one and that's why you can not perform IVTC.

Am I correct ?

Last, what do you think of a "mode" where you can double the frame rate and keeping the full temporal resolution (applying the algorithm for both odd and even fields) like what Gunnar Thalin's "Deinterlace - Smooth" VirtualDub plugin does ?

[trbarry]

1) take each pixel of an image constituted by the the odd fields
2) for that pixel, look for the neighboring pixels in the precedent image (the even fields)
3) calculate the amount of motion
4) determine the percentage of inter-field information to use
5) create the final image with the inter and intra-field information

morsafr -
1) If you do it in a filter there is no reason to actually separate the fields first. You know where the odd and even lines are.

2) Yes, though I use both the prev and next images

3) Sort of. I just try all the combinations and choose the one with the lowest. But I only know the amount of change, not really the horizontal or vertical motion. (don't care)

4) I should have, and that's what I did in GreedyHMA. But for performance TomsMoComp uses either intra or inter for each pixel, not a blend. That was a mistake I'll change one of these days.


That means that each resulting frame is different from the precedent one and that's why you can not perform IVTC.

TomsMoComp is for pure video, not telecined film material. You shouldn't do both. Though in the case of mixed material you could still decimate frames, since you don't need exact field or frame matches, just relative amounts of change.


Last, what do you think of a "mode" where you can double the frame rate and keeping the full temporal resolution (applying the algorithm for both odd and even fields) like what Gunnar Thalin's "Deinterlace - Smooth" VirtualDub plugin does ?

TomsMoComp for DScaler does this now (double to 50,60 fps) but I didn't support it for Avisynth for some silly reason. I get beat up every week or two on this and I actually wrote most of the code to do it over a month ago. But got sidetracked with other issues before finishing.

- Tom

[morsafr]

Thank you all for your explanations. Feel better now ! :)

[smok3]

@tom: greedyhma did some great job on ntsc and pal interlaced stuff, any plans to do some proper manual/guide so i will know what exactly is happening? And 2nd is there a plan to relese vdub version of greedyhma? tia.

[trbarry]

smok3 -

Well, for pure interlaced stuff (NTSC or PAL), my guide is to always use:

TomsMoComp(1,5,1) or
TomsMoComp(0,5,1) depending upon whether the clip is top first.

For mixed or problem material these days I just use DeComb. Donald Graft has invested way more effort and patience into dealing with all the exceptions and requests than I am ever likely to be able to do.

I may in the future come out with an optimized fast IVTC filter of some sort for the general cases. But I have no plans for handling all the special problems and pathological conditions that come up and I have no plans at all for writing Vdub filters of any sort.

I do however have a partially completed version of TomsMoComp that should rate double to 50-60 fps. I'll try to finish and release that soon.

- Tom

[DDogg]

I do however have a partially completed version of TomsMoComp that should rate double to 50-60 fps. I'll try to finish and release that soon.

Tom? :) or did I miss this in another thread?

[trbarry]

DDog -

Sorry. The code got forked for the latest Avisynth 2.5 and I haven't gotten back to it yet. It's complicated somewhat by it being so long I forgot where I was in it and there have also been some new DScaler developements that I might like to incorporate.

But I'll try to get back to it and finish up at least the 60 fps part.

- Tom

[DDogg]

And you thought we forgot :) I could use it as part of the "film look" I mentioned in the other post seems to involve going to 60 fps. Beggars can't be choosy. I just appreciate all the stuff you have done for the group (blatant suck up to twist your arm :) ).

[trbarry]

If you search the various TomsMoComp threads I think someone (Hakko?) already published a TomsMoComp script function that will take it to 60 fps properly. The only difference would be a small amount of run time performance.

- Tom

[hakko504]

Yep, no big deal:Function TMCbob(clip clip,int topfirst)
{
#topfirst=1 => top field dominant
#topfirst=0 => bottom field dominant
video=clip.separatefields().doubleweave()
top=video.selectEven().TomsMoComp(topfirst,5,1)
bottom=video.selectOdd().TomsMoComp(1-topfirst,5,1)
return interleave(top,bottom)
}

[DDogg]

hakko504, thanks much for placing this here. Tom and I both looked for it among the 71 threads on Tomsmocomp to no avail. A jewel hidden in the topic turbulence!

Add: Just to check:

I saved the above as TMCBOB.avsi in my plugins directory.

Using this script:
LoadPlugin("TomsMoComp.dll")
V=avisource("Tape 1 - Clip 001.avi").ConvertToYUY2()
TMCBOB(V,0)

Is this correct? (for BFF DV)

[hakko504]

Yes, that should be the correct usage: open it in VD and find a motion place, if it's incorrect T/B it will jump back and forth.