Questions on TomsMoComp

[Guest]

@trbarry

Tom, may I ask a few questions about TomsMoComp? I ask here rather than PM because I believe others too can benefit from the answers.

1. When you search for a pixel in possible moved locations, do you require the values to match exactly, or must they match only within a margin? If they must match exactly, it seems you'll find few matches due to normal noise. If they do not have to match exactly, what is the margin?

2. You've said that you wouldn't trust anything above SE 5, but I believe that vertical search is enabled only when SE > 5. Also, the search occurs only for horizontal displacements of 1 pixel. So won't these facts severely limit your possible weaving as purely horizontal motion of one pixel position is likely quite unusual?

3. I send a freezeframe'd clip to TMC with SE=1. I expect the video to be passed through untouched. But I get artifacts on diagonals. That implies that even though weave should have been chosen all the time, it isn't. I know your weave value is clipped, and supposedly there is a test to override that for a static scene. Is this mechanism broken? How does the override mechanism work, i.e., how do you declare a static scene?

4. Given 1 and 2 above, do you think there is any reasonable amount of weaving actually occurring? Is there a way to quantify or visualize it?

thank you,
Don

[trbarry]

1. When you search for a pixel in possible moved locations, do you require the values to match exactly, or must they match only within a margin? If they must match exactly, it seems you'll find few matches due to normal noise. If they do not have to match exactly, what is the margin?

There is a hard coded value in member SearchLoopBottom that sets the allowance for noise. Right now it's probably 4 or 5, but subject to change in a future release as I blend things more.

2. You've said that you wouldn't trust anything above SE 5, but I believe that vertical search is enabled only when SE > 5. Also, the search occurs only for horizontal displacements of 1 pixel. So won't these facts severely limit your possible weaving as purely horizontal motion of one pixel position is likely quite unusual?

Because of the interlace, vertical search is actually looking 2 pixels away, not one. And because I'm only testing one pixel at a time it is easy (fairly probable) to get false matches which can cause stray 1 dot artifacts. This increases with vertical search, or Se > 5, or both. In DScaler at real-time I'll actually use up to SE=13 but in Avisynth where people are going to be looking at the output 1 frame at a time and posting me pictures of each wrong pixel I recommend 5.

BTW, I found that I can get better results than might be expected with the horizontal-only search because most moving edges are slighly diagonal. So if a diagonal line happens to move vertially by one pixel it can often be represented okay by a horizontal-only search. It's the darn jaggies on diagonals that get you deinterlacing anyway and this at least catches many of them.

But because of this false-match problem I've pretty much decided the TomsMoComp single pixel search should not be extended to larger area, even if we get faster machines.

3. I send a freezeframe'd clip to TMC with SE=1. I expect the video to be passed through untouched. But I get artifacts on diagonals. That implies that even though weave should have been chosen all the time, it isn't. I know your weave value is clipped, and supposedly there is a test to override that for a static scene. Is this mechanism broken? How does the override mechanism work, i.e., how do you declare a static scene?

I thought we discussed this. Are you by any chance still using YUY2? Maybe there is another bug for YUY2.

But I'm working on TomsMoComp today and will look for this. You should indeed see an unchanged clip, except for the first frame. (if you applied those 2 fixes)

4. Given 1 and 2 above, do you think there is any reasonable amount of weaving actually occurring? Is there a way to quantify or visualize it?

Assuming it's not broken, yes, there should be weaving, and you should usually be able to see the difference on most video clips if you compare closely. Note however that since TomsMoComp is for video source it will NOT necessarily pass through moving progressive material unchanged. Some detail will be lost.

Why don't you try a compare at different search efforts? And if you are using YUY2, please also compare to the results of the Avisynth 2.0 version that is also in the zip. That would point out if there is a new bug for YUY2.

- Tom

[mf]

Too bad it's not designed for progressive material, at one point I've seen it IVTC some material very well, except for flickering displaced pixels on still edges. If the latter could be fixed, it could be interesting to try on hybrids.

[trbarry]

When I did GreedyHMA I designed it for hybrids. But I ran into some cases where I thought I could have done a better job on video if I could just treat it like video.

So that was the assumption when I did TomsMoComp. It's optimzed for video. There's really not much point in doing motion compensated deinterlacing when your material is not interlaced.

For mixed material it's probably better to have a smart higher level function like Telecide that decides whether TMC should ever even see the frame. If it's possible to handle it as a simple field match then we should do that instead.

- Tom