FieldDeinterlace() vs. ToMoComp(), with pictures (and maybe bugs)

[primitive]

Hello everyone. First of all, size warning: there's a lot of pics and clips here to look at, and I think the smallest one is 500k.

I was experimenting last night with decombing tools and got some interesting results. I may have hit a bug too, but I'll let the powers that be decide that. The parameters used for each example were chroma=true for FieldDeinterlace and 0,5,1 for ToMoComp.

First comparison: Eva: D&R.

*edit* (676k)
*edit* (676k)

To my eyes, the ToMoComp looks much nicer. In the FD() example, the vertebrae still look combed, whereas in the ToMoComp example the divisions between the vertebrae are well-smoothed by the vertical filter. HOWEVER, I think here is where I start to see odd behavior: when I open up the two clips I got these bmps from in vdub, and switch between the two, it appears as if the ToMoComp clip has been shifted down one pixel, perhaps because of the way the filter blends pixels. This effect should be apparent in the bmps I posted. Yes, the same cropping settings were used for each clip.

Second comparison: Miyuki-chan in Wonderland.

Round 1:
http://people.vanderbilt.edu/~b.nuttall/6639.bmp (799k)
http://people.vanderbilt.edu/~b.nuttall/6640_decomb.bmp (799k)
http://people.vanderbilt.edu/~b.nuttall/6640_toms.bmp (799k)
http://people.vanderbilt.edu/~b.nuttall/6641.bmp (799k)

In the orignal source, 6640 is a combed frame on a scene change. In motion, ToMoComp's job looks much nicer than FD's; however, a question: that gray artifacting on the chinese woman's arm was originally a combed area that was passed through the filter, which I blended with the vertical filter so it wouldn't look so terribly bad. Is that kind of effect to be expected when using ToMoComp for anime? The artifacting happens in many, many other places, but it's almost never as bad as it is in this example frame.

Round 2:
http://people.vanderbilt.edu/~b.nuttall/clip1-decomb_bad.avi (504k)
http://people.vanderbilt.edu/~b.nuttall/clip1-tomocomp_good.avi (496k)
http://people.vanderbilt.edu/~b.nuttall/clip2-decomb_good.avi (588k)
http://people.vanderbilt.edu/~b.nuttall/clip2-tomocomp_bad.avi (586k)
http://people.vanderbilt.edu/~b.nuttall/clip3-tomocomp_bad.avi (448k)

I encoded the first part of Miyuki-chan in Wonderland with each of the decombing tools. These are example clips I pulled from the final encodes. Clip 1 is a shot of Miyuki running, her hair waving behind. When stepping through the frames in that clip in vdub, it might look like the decomb gives better-looking output, but the blending done in this scene is so obvious that it gets on my nerves. I much prefer ToMoComp's blending in this particular clip.

Clip 2 is where I think I spotted a bug. The question is: if the vertical filter works down the frame, what happens when the motion is moving vertically? I've encoded this silly clip tens of times as test material in the past, and the diagonal lines in clip 2 have always been hell to try to work with. In the ToMoComp, it looks like the motion compensation + the vertical blending is causing the lines to flicker and look generally nasty. This is more apparent in clip 3, which just looks downright bad with ToMoComp. The same clip looks passable when using simple blending like FD wants to do.

I'd appreciate any guidance any of the wizards of the forum could give on this matter.

-p

*edit* fix URL.
*edit2* kill images so I can post more.

[Guest]

It is unfair to criticize FD for blending when you can easily turn it off and have interpolation instead.

I also don't see any combing on the vertebrae with FD. First, direct inspection with a magnifier shows no combing. Second, I did a subtract operation between the two BMPs and there is no sign of interlacing left by FD. The only thing that shows up on the diff image is the smoothing done by TomsMoComp. Please substantiate your claim that there is residual combing. Thank you.

The smoothing done by TomsMoComp can also be seen easily when placing the two BMPs side-by-side. This loss of sharpness may not always be a welcome effect, although it may be welcome in the case of noisy source material.

The artifacts of TomsMoComp you mentioned come from doing interframe comparisons for motion detection. FD does only interfield comparisons and thereby avoids these nasty artifacts.

People need to bear in mind that FD is designed and optimized as a post-processor for Telecide. It is a lean and mean cleanup for combed frames that may be emitted by Telecide. It is not optimized as a general purpose deinterlacer, though it performs quite well as such. If I wanted to make a general-purpose deinterlacer out of it, I would add motion-map denoising at a minimum, but that would slow it down significantly.

[trbarry]

The artifacts of TomsMoComp you mentioned come from doing interframe comparisons for motion detection. FD does only interfield comparisons and thereby avoids these nasty artifacts.


Maybe, but I think clip3 is really from telecined material as the artifacts come in a regular 2 of 5 pattern. TomsMoComp is really only for pure video source.

primitive -

Maybe try IVTC with Decomb or GreedyHMA on this clip. And if you continue to have problems, please post your full script.

It looks like you have done a lot of work on this. Your feedback is appreciated.

And yes, of course the vertical blending is optional in TomsMoComp also, but I usually leave it on to hide the stray deinterlacing artifact that I don't know how to get rid of yet. ;)

- Tom

[Guest]

@trbarry

>Maybe, but I think clip3 is really from telecined material as the
>artifacts come in a regular 2 of 5 pattern. TomsMoComp is really
>only for pure video source.

These artifacts can occur on pure video as well. I can provide test clips if you doubt it. This is an unfortunate consequence of interframe comparison. But interfield can also have unfortunate consequences! You have to pick the lesser evil for any given clip.

[trbarry]

neuron2 -

Wanna bet a nickel? ;)

I think it's true that TomsMoComp can pick up aliasing artifacts on certain speed vertical pans on some material.

But I'd still bet this one was telecined. Maybe primitive would furnish a few frames of the original clip around that problem? Or even just tell us the results of using some IVTC process on it first?

- Tom

[Guest]

@trbarry

Hey, I wasn't saying his clip wasn't telecined!

I was just making the general point that interframe comparisons tend to produce one kind of artifact (like on the girl's arm as mentioned) while interfield comparisons tend to produce a different kind. This is true regardless of the source material.

(And to be a devil's advocate just a little, it shouldn't matter to a deinterlacer if the source is telecined.)

[primitive]

@neuron2

Whoops, I forgot that you can't view xvid. ^^;

First of all, I'm not bashing your tool. It's a good tool with some advantages and disadvantages, just like ToMoComp. The point is that sometimes interframe comparisons give nicer looking output than interfield comparisons, and vice versa.

Also, all these tools are new(er), and look at the platform we're testing on: filters under development, on top of a video processing core that's under heavy development, encoded with an mpeg4 codec that's still alpha. I'm no video encoding wizard, but I'm also not a rank newbie, and when I see something I don't understand I think it's best to 1) RTFM, and if that doesn't help 2) post a msg on a board where the creators of said tools hang out, especially since they make themselves available.

Ok, now to your suggestion.

http://people.vanderbilt.edu/~b.nuttall/6640_interpolate.bmp (799k)
http://people.vanderbilt.edu/~b.nuttall/6641_interpolate.bmp (799k)

These two shots are taken with FieldDeinterlace(blend=false,chroma=true). (I'm assuming that chroma=true is a relatively innocent commandline option that I can keep on all the time without sacrificing quality; is that true?) The decombing in these two shots is quite good, actually. However, to my eyes it's hell on the solid lines; look at Miyuki's chin in 6640, and the arm of the Chinese girl in 6641. Also, blend=false is hell on these solid lines even when there's no motion in the frame at all. I have two still shots (which I can't post right now because I'm out of webspace) that demonstrate this effect.

About the "combing" in the vertebrae, it's a very subtle effect. When I get out of class this afternoon I'll make it clear what I'm talking about. Probably the easiest way to notice what I'm talking about however is to get a sequential image viewer and flip back and forth between the two vertebrae shots, focusing on the divisions between light and dark. It may not even be combing at all.

@trbarry

I know the clip isn't FILM source. I remember posting something in the recent past about difficulty with this same clip with decomb missing mouth flaps. The difficulty was that I was trying to IVTC source that wasn't progressive! I believe neuron2 would remember that, I still have hot feet from the flame ;)

Well, when I say "It's not FILM!" I say this because DVD2AVI reported that the source material was NTSC Interlaced. If you want me to furnish a few frames of the source I will, but I'll have to make all my links in this thread go away to clear out enough webspace to do so, or upload the clip to an FTP. Send me an email if the FTP option is an option, b.nuttall@vanderbilt.edu

I haven't tried GreedyHMA on this clip, I will do that when I get out of class today. Perhaps it would help; I've had one other person recommend that solution to me as well.

@all

My avs is rather pedestrian.


LoadPlugin("<insert decombing solution here>")
LoadPlugin("mpeg2dec.dll")
LoadPlugin("Convolution3D.dll")
LoadPlugin("SimpleResize.dll")

#FieldDeinterlace(blend=false,chroma=true)
#FieldDeinterlace(chroma=true)
#TomsMoComp(0,5,1)

crop(16,0,688,478)
SimpleResize(608,448)
convolution3d(0,8,16,8,8,3,0)


I don't see any complicated mojo happening in this script that would throw off the outout appreciably, but as I have said before I'm no wizard.

-p

[trbarry]

where the creatures of said tools hang out ...

(Misquoted, just the way I read it first. I like the phrase ;) )


primitive -

Nah. Don't bother to clear out all the other carefully wroght images. But I think mostly you will find that posting jpg images instead of bmp still shows all the artifacts at a fraction of the size unless you are telling folks to zoom them a lot. And they sure would download faster.

In case it is IVTC'able, you might try either:

GreedyHMA(1,0,4,0,1,0,60,90) or
GreedyHMA(0,0,4,0,1,0,60,90)

and see if either works. Change the 4 to 1 if you don't want to reduce the frame rate to 24 for some reason.

Let us know.

- Tom

[Guest]

@primitive

>It may not even be combing at all.

Exactly my point. It is image detail that is smoothed away by TomsMoComp.

Interpolation is always hell on anime. Once again, you choose your lesser evil based on the source clip.

[primitive]

@trbarry

Sorry for the large images. I typically work problems like these through with a member of my fansubbing group, and since the stuff we're looking at is sometimes subtle we have gotten in the habit of passing bmps back and forth. Also, paint on win2k doesn't speak in png :(.

Ok, I've tried GreedyHMA. First of all, I'm almost positive that this isn't IVTC material. When I decimate the stream I get lots of combed mouth flaps; sure, when you pass them through con3d they don't look bad, but it's not what we're looking for.

Second, GreedyHMA is much nicer on diagonal lines than ToMoComp.

http://people.vanderbilt.edu/~b.nuttall/5963_greedy.bmp (799k)
http://people.vanderbilt.edu/~b.nuttall/5964_mocomp.bmp (799k)

First of all, it was curious that it was hardly ever the case that both methods produced bad frames at the same time. This must be due to algorithmic differences. What I have instead done is that I picked two frames that were close together in the original clip that represented subjectively the "worst" each algorithm could do on this particular scene. I *think* this kind of comparison is valid, but I'm not sure. In this scene, GreedyHMA bests ToMoComp by a wide margin, as it does in other scenes throughout the clip. I don't think GreedyHMA is more compressible than ToMoComp, but I'll see what happens after a test encode.

So what's the verdict? I realize I'm probably pushing any decombing algorithm too hard by trying to apply it to this source material. Is ToMoComp better suited to live-action material?

-p

[trbarry]

No. TomsMoComp is better suited for pure video source and will beat GreedyHMA on most of these. But in spite of what you stated earlier, I didn't really believe you. Sorry. ;)

So I gave you a GreedyHMA prescription for a badly mastered, mixed, or hard to handle telecined source, where Greedy still has the edge. To be honest, that might include a lot of anime.

But of course neither is perfect.

- Tom

[primitive]

@trbarry

I think I'll try ToMoComp on some less-pathological anime source material and see how it fares.

GreedyHMA on this clip looks quite nice in motion. There's some stuttering in the pans that I'm trying to track down, but I have to make sure it's not in the source before I start looking for witches to hunt ;). I'm also going to go back and reencode the other two to compare the quant distribution and post for comparison.

-p

(P.S.: Abismo and I got on an argument on the channel as to whether or not we should trust the frametype DVD2AVI gives us. Under what situations will DVD2AVI incorrectly detect the types of frames within a clip, if any?)

[ErMaC]

When DVD2AVI says it's not FILM, all it's saying is that the MPEG2 stream is not flagged with the stream flags (rff and tff I believe) which tell the decoder that the actual MPEG2 stream is in progressive and you need to fetch fields from adjacent frames on playback to decode it.

If I take a telecined piece of film, stick it in TMPGEnc, set the MPEG2 interlacing settings to Interlaced and encode, DVD2AVI will say quite correctly that the clip is NTSC interlaced, because that's what the MPEG2 stream is encoded as.

That does NOT mean the stream actually is video. In the case of both of the things you're testing these are most obviously NOT the case because both of them were shot on Film (Eva D&R was shot on 35mm and Miyuki was most likely shot on 16mm). I have the region 2 Evangelion movies 2-disc set (disc one is DeathTrue^2 and Rebirth, disc 2 is Air and Magokoro wo, kimi ni) and they are actually encoded in 99% film according to DVD2AVI. There are a few shots in EoE and in Death where I will notice the occaisional bad combed frame, but it's almost flawless. The Television discs (both in region 1 and region 2) are not encoded in progressive (since the analog NTSC masters of Eva are just that, NTSC, and are VERY poorly telecined to boot, search the divx5 encoding forum for some horror stories) but were still shot from film. However no matter what deinterlace or IVTC method you try on the TV discs they will still be awful. I really wish GAiNAX would learn how to shoot and telecine film (all the hairs in KareKano and Eva on the bottoms of the frames are very annoying).

[trbarry]

The way I look at it, if DVD2AVI says it is film it means that it can handle it as film, and it is mostly right. But if DVD2AVI says it is video then you are on your own. Decomb or GreedyHMA may still be able to IVTC it with better results than TomsMoComp, which only deinterlaces and assumes each top and bottom field are separated in time.

I keep meaning to write some sort of status filter that samples a clip and reports things like TopFirst, film/video, 3:2 patterns, etc., without caring about the video flags.

But I also keep meaning to fix the roof and get a job too, and neither of those seems to be happening either. ;)

- Tom

[ErMaC]

Roofs, and jobs to a certain extent, are overrated.

[MaTTeR]

Originally posted by trbarry
I keep meaning to write some sort of status filter that samples a clip and reports things like TopFirst, film/video, 3:2 patterns, etc., without caring about the video flags.

Oh jeez, how nice that would be and oh how much time it would save me from analyzing/testing these 4:3 flicks manually :-) It seems D2A's detection scheme does alright but still we all know it's inaccurate sometimes. Life and encoding would be so much easier without deinterlacing and IVTC:D

[ErMaC]

That's why I wish the only ATSC stanards were 1080p, 720p, and 480p - none of this 1080i interlaced crap.

[trbarry]

That's why I wish the only ATSC stanards were 1080p, 720p, and 480p - none of this 1080i interlaced crap.

Millions of years ago, before the dawn of recorded history, a bunch of TV engineers and a bunch of computer engineers got into a food fight at a local rock quarry about the interlace issue.

No one remembers the exact details but temper still rise whenever the issue is raised and the two groups still give each other exactly zero credibility on almost anything. ;)

- Tom

[Trahald]

yes.. just ermac said.. you have to take dvd2avi with a grain of salt. ALL it looks for is rff tff kind of telecining and if that doesnt exist its deemed ntsc (and when its mixed you get %'s) it doesnt compare fields at all.