Deinterlacing of 30i+24p video with irregular pulldown pattern

[yukichigai]

So I've been using ivtc_txt60mc (http://forum.doom9.org/showthread.php?p=1466105#post1466105) by cretindesalpes to great effect for a while now. It's absolutely wonderful for dealing with 24p telecined video that has 30i text effects over it. The only downside to it is that it absolutely depends on you knowing the telecine order and it being a solid 3:2 pattern.

Every now and then I've run across some sketchy telecine patterns, but I've managed. I'm no slouch when it comes to editing avisynth scripts and I've tweaked ivtc_txt60mc when needed so that it will support different telecine patterns. The problem is that so far these patterns have been reliable, predictable, and repeating. I have now for the first time run into something I have no idea how to deal with: an irregular telecine pattern with 30i text effects over it.

Let me give you a real example. Taking this irregularly telecined video and applying a simple bob, I wind up with fake 60p video. If I count the number of sequential identical frames before each frame change, I note a "pattern" like this (actual example):

32332332333323233233332332332324323233233242332332324

As you can see, that's basically random. It's certainly not something that SelectEvery could handle, which means ivtc_txt60mc isn't capable of dealing with it. Furthermore, it appears that the resultant framerate is actually closer to 22p, not 24p.

My question is: what do I do here? I don't want the overlay effects to look like crap, but at the same time I want the main video to look decent too. Should I just bite the bullet and sacrifice one versus the other, or is there a solution?

Sidenote: what should I use to determine the true framerate of that weird pulldown pattern?

[manono]

Sample?

[yukichigai]

Alright, here's a little under a minute of video from the beginning. It's an episode of MST3K obviously, Warrior of the Lost World to be specific. The clip's about 34 megs in size.

test.m2v (34.1mb) (http://www.filedropper.com/test_20)

I want to emphasize that I've never seen anything with this crazy a pulldown pattern in MST3K's library before. Not even some of the awful films put out by Film Ventures International, a few of which avoid pulldown in favor of actual frame blending things to 60i. Yeah... awful.

[ChiDragon]

Interesting. I'm used to dealing with 24+ fps rates stuffed into 30i, not slower-than-24. There's always the option of deinterlacing it to 59.94 and living with the pulldown stutter, of course...

Some frames are shifted vertically and aliased, like 771, 786, 888.

[manono]

Interesting clip, I agree. As you said, it's close to 22fps. Using this script:

TFM(D2V="E:\Test\Test.d2v",pp=0)
TDecimate(Mode=0,Cycle=46,CycleR=12,maxndl=5,M2Pa=True)

I got a total of three duplicate frames and no (I don't think) missing frames in the entire sample. That's 22.152fps, so it's very slightly lower than that. If you want to mess with it that's up to you but I didn't feel like doing any more counting. I used PP=0 because the sample fieldmatched perfectly but leaving it off still often kept the shadow figures interlaced. So, what to do about them? Here's a script I used which is a mix of something I learned from jagabo and poisondeathray:

orig=Last
mt_binarize(30, upper=true).mt_expand().mt_expand().Blur(1.0)
mymask=last
orig
Yadif()#Or your favorite deinterlacer
Filtered=Last
overlay(orig, filtered, mask=mymask)

Add that after the IVTC. The '30' tells the deinterlacer used within the mask to operate on any luma less than 30. It deinterlaces only dark stuff. You might want to adjust that number although it works fine with that sample.

I don't know what to do about the jerky nature of the shadow figures when decimated to 22fps. I'd leave it alone as they don't move around much and your eye will follow the movie and not the guys in the audience.

[yukichigai]

Alright, I'm doing pretty good and I've got an idea or two on how to handle the shadow figures. The problem I'm having is figuring out what the precise resulting framerate is from TFM+TDecimate. 22.152 is close but not quite right, but I can't find any suitable source that will give me the actual fully detailed framerate. No framerate, no splice. I feel like I should know how to do this. Is there an informational filter I should be using?

I've tried converting the frame interval to framerate, but the result (1000000/45143) doesn't work.

[ChiDragon]

The precise frame rate is a fraction which you can see with Info().

[yukichigai]

The precise frame rate is a fraction which you can see with Info().

Awesome, thank you, that did the trick. Looks like the fraction is 510000/23023.