(Yet Another) Weird Pattern 25 -> 29.97 telecined

[DoctorM]

I've been banging my head against a piece of interlaced video (29.97) for a couple days.

I tried IVTC'ing with TFM and Telecide (then decimating). The output keeps ending up jittery. (Duplicate and missing frames).

Finally in disgust I used bob() and counted unique fields and discovered the source to be 25 fps. The thing is there is no ghosting, blending or anything wierd. Just 25 unique fields and a very strange pattern of duplicates.

My current script looks like:
qtgmc(Preset="medium")
fdecimate(25)
assumefps(23.976)

I'm assuming there will be speed up in the audio that needs correction, but that's a different can of worms.

The point being, this method is slow (even at Medium).
TFM will inverse telecine to 29.97 but drop frames (that I can see with a bobber).

It would be quicker and produce better quality to IVTC directly, but I know of no filter that is tunable to match fields for frame rates besides 29.97.

Suggestions?

[manono]

Suggestions?
My first suggestion is that you provide a sample. If you're asking here what to do, maybe you don't know how to diagnose what you have either. Samples are always better than descriptions.

Anyway, based on your description, maybe this will work:

TFM()
Tdecimate(Mode=0,Cycle=6,CycleR=1)

But I'd still feel more comfortable offering advice if there was a sample included.
TFM will inverse telecine to 29.97 but drop frames (that I can see with a bobber).
TFM is a field matcher. It's only part of an IVTC. It doesn't drop frames. Your source is also 29.97fps, isn't it?

but I know of no filter that is tunable to match fields for frame rates besides 29.97.
TIVTC can produce a virtually limitless number of framerates. As can Decomb. If there are a bunch of orphaned fields in this thing, then there will be problems and your current solution may be the best.

[TheSkiller]

Please provide a sample.

From your description it sounds like 25p wrapped in 29.97i using pulldown. Nothing weird about that. Manono's suggestion would deal with exactly that (result would be 24.97 fps which can be left as is or speed up to 25 fps).

If you're unlucky, editing has been done in 29.97i.

[DoctorM]

My first suggestion is that you provide a sample. If you're asking here what to do, maybe you don't know how to diagnose what you have either. Samples are always better than descriptions.

Good point. Clip here. (http://www.mediafire.com/?oide7qoigdvusa4)

Anyway, based on your description, maybe this will work:

TFM()
Tdecimate(Mode=0,Cycle=6,CycleR=1)

But I'd still feel more comfortable offering advice if there was a sample included.

TFM is a field matcher. It's only part of an IVTC. It doesn't drop frames. Your source is also 29.97fps, isn't it?

And yet oddly TFM was dropping frames.
Haven't had a moment to test this yet, will a bit later.

TIVTC can produce a virtually limitless number of framerates. As can Decomb. If there are a bunch of orphaned fields in this thing, then there will be problems and your current solution may be the best.

Yes, I realized I mis-spoke when I suggested I needed to IVTC to a different frame rate than 29.97. I meant I needed something besides the standard pattern after frame matching.

If by orphaned fields, you mean single fields without a match, yeah, I've got them. I assumed they'd end up as a remaining combed frame that gets passed to the post processor.

[manono]

And yet oddly TFM was dropping frames.
It's dropping fields. And yes, they're orphaned fields - fields for which there's no match. TFM won't create a duplicate for an orphaned field on which to match, or bob it. It'll just drop it. This is a most peculiar way to convert PAL to NTSC. I'd do it as you showed in your first post, maybe changing the decimate portion to:

TDecimate(Mode=0,Cycle=60,CycleR=35)

But it doesn't matter a whole lot. Ignore the jump on frame 15 because there isn't a full cycle on which it can work.

Edit: I was thinking about this some more. The only IVTC that bobs fields if needed is the old SmartDecimate. I tried it and even it had trouble going straight to the final framerate. But if field-matching/bobbing to 29.97 followed by a decimate, it did better. Here's the script:

SmartDecimate(30,60)
TDecimate(Mode=0,Cycle=30,CycleR=5)

Here's an MP4 I made of it. I can't tell if it's perfect or not but, while it isn't as fast as using TIVTC alone, it's way faster than having QTGMC in there:

https://mega.co.nz/#!kJxwERSK!ZaG5iHAbAdNe1w9ybhIvOlDDfQFU5PB2R-O1ikP4as0

[DoctorM]

Good suggestions, manono.

TDecimate seemed to hit on unique frames more accurately. SmartDecimate preserves a lot more detail. Which onsidering the condition of the source is nice.

I play around with that some more, but that looks to be the best answer... and much faster.

Edit: I cannot get my head around what exactly it is that SmartDecimate is doing. The page doesn't make a lot of sense when I read it.
http://avisynth.org.ru/docs/english/externalfilters/smartdecimate.htm

[manono]

Edit: I cannot get my head around what exactly it is that SmartDecimate is doing.
Do you really have to understand what it's doing? Isn't it enough that it works? :)

Think of it as a regular IVTC with the extra added attraction of being able to use bobbed fields when needed. I believe it was developed as a way to get a single framerate out of hybrid material - things like the Star Trek spinoffs where there's a mix of film and interlaced computer generated material. By being able to use bobbed fields on occasion, it's better at getting nicer-looking results from the video portions than the Decomb or TIVTC methods of blending frames (when converting from interlaced 29.97fps to progressive 23.976fps).

Knowing that, I decided to see if it worked with your sample or other material with orphaned fields. It does, after a fashion. I think it bobbed and kept all the orphaned fields, but because the duplicate pattern was peculiar afterwards, I decided TDecimate was better to remove those dupes. As I mentioned, I used SmartDecimate to get progressive 29.97fps from your source, before then using TDecimate to bring it down to 24.975fps.

You'll notice it uses the AviSynth Bob filter when bobbing. You might want to use a better bobber when needed, maybe even QTGMC. The manual explains how to replace Bob with any bobber you might want to use.

[DoctorM]

Good point. There are some confusing details about the filter. At one point it is just about saying it decombs instead of matching fields and then later you can set how aggressively it matches fields.

Good eyes with the bob= option. Missed that. Unfortunately, Yadif, Yadifmod and qtgmc all cause a crash. Either a silent crash or a program error (not avisynth) (Media Player Classic, VDubMod, etc.) and "R6025 Pure Virtual Function Call". Even dgbob (their example) didn't work. Huh?

Not sure what's going on there.

[manono]

Yeah, I had trouble getting it going too. It's been years since I used it and several AviSynth versions ago. Maybe someone with more knowledge than I can give a more efficient way to set it up, but what made it work for me was sticking the other needed DLLs before the AviSynth_C.dll needed by SmartDecimate. I couldn't get it going with either the LoadCPlugin or StdCallPlugin. Something like this worked:

LoadPlugin("D:\AviSynth Stuff\Dlls\DGDecode.dll")
LoadCPlugin("D:\AviSynth Stuff\Dlls\Yadif.dll")
LoadPlugin("D:\AviSynth Stuff\Dlls\avisynth_c.dll")#Yadif before AviSynth_C.dll
LoadCplugin("D:\AviSynth Stuff\Dlls\SmartDecimate.dll")
LoadPlugin("D:\AviSynth Stuff\Dlls\TIVTC.dll")
MPEG2Source("E:\Test\Test.d2v")
SmartDecimate(30,60,Bob=Yadif(Mode=1))
TDecimate(Mode=0,Cycle=30,CycleR=5)

But as I said, there are probably better ways to set it up.

[DoctorM]

Cheers. I'll give it a go.

[DoctorM]

Still unable to get a custom bobber to work. I figured it was because I wasn't calling the autoloading plugins, but that didn't help.

Btw, I found there was still the occasional incorrect frame after decimation.
Counting SmartDecimate's output I found this pattern: 12344 1234566 12344 1234566

As a result having TDecimate check cycles of 30, a range not divisible by 12, produces one improper decimation every second.

Changing to: TDecimate(Mode=0, Cycle=12, CycleR=2) improved accuracy and produced a serious bump to performance.
The pattern seems to hold throughout the video.

Now if I can just get the bobber fixed...

[manono]

Counting SmartDecimate's output I found this pattern: 12344 1234566 12344 1234566
I noticed that too, but it's not consistent all the way through. Sometimes it switches.
As a result having TDecimate check cycles of 30, a range not divisible by 12, produces one improper decimation every second.
I found the two different decimation methods to produce frame-for-frame identical results with the sample. Of course, yours is faster and if it works throughout the whole complete video, then of course you should go with it.

I'm sorry you haven't got the other bobbers to work with SmartDecimate yet. I don't know what other advice to offer. Good luck.

[johnmeyer]

The pattern in your sample is a little convoluted, but appears to repeat. Therefore the Cycle/CycleR in TDecimate works perfectly, if you count the pattern correctly. Here is the field count I came up with, where each digit represents a frame, and 1, 2, and 3 represent the number of fields in each frame in the original video ("2" fields is, of course, what you want for every frame). I viewed this using a script that is nothing more than "separatefields()" :

3-1-3-2-3-2-3-2-3-2-2-2-3-2-3-2-3-2

If you count the number of digits, there are 18 digits, so there are 18 frames. So, after field matching and decimation, we want to end up with 18 frames (36 fields) from these 43 fields. Therefore, out of every 43 fields, we should decimate 7.

AssumeTFF()
tfm(display=false,order=-1)
tdecimate(display=false,cycleR=7,cycle=43)

However ...

There is something in the TFM logic that I don't quite understand in how it post-processes the individual fields that have no match. If you set "order=1" TFM will correctly post-process the unmatched odd fields. For these single fields (shown by the "1" in the pattern above) it will give you a frame that contains the original unmatchable field plus a synthesized field, created using whatever technique you choose using the "pp" parameter. However, it skips the even fields. If you instead set "order=0," it then does the even fields but skips the odd fields.

So near and yet ...

So, the TDecimate logic works great, but the TFM matching and post-processing doesn't quite do the trick.

Another approach might be to double the frame rate using a bobber or deinterlacer, and then decimate from that.

One final note. I found that the pattern repeats every 43 fields but when I actually walked through the result, putting a separatefields() at the end of the recovered video so I could see what was happening, I found there was still an undecimated result every two cycles. So I doubled Cycle and CycleR and then added one to CycleR in order to remove these remaining duplicates:

AssumeTFF()
tfm(display=false,order=1)
tdecimate(display=false,cycleR=15,cycle=86)
separatefields()

The separatefields() statement is there for debugging purposes, so I can see exactly what is going on, and would of course be removed when using the script to actually generate the final video.

[johnmeyer]

P.S. to my last post. I didn't actually use "MPEG2Source("E:\Test\Test.d2v")" to read the video and perhaps I should. In reading the TFM documentation, there is a parameter that sounds like it might solve the problem:

d2v -

This option is intended to be used if you are using an mpeg2source() with a d2v file.
It sets the name and path to a d2v file, which TFM will analyze to see if there are any
illegal field order changes and optionally set the order parameter using the field
order of the d2v file.

[manono]

If you set "order=1" TFM will correctly post-process the unmatched odd fields.
It doesn't. There are 2 missing fields/frames doing it that way. Using this script:

TFM(Order=1)

Even before decimation there are already two missing frames, before frames 30 and 73. TFM just cannot handle this.

AssumeTFF()
tfm(display=false,order=-1)
tdecimate(display=false,cycleR=7,cycle=43)

That can't be correct. The final framerate is 25.091 when it should be 24.975fps. You still have the same two 'jumps' plus three duplicate frames.

AssumeTFF()
tfm(display=false,order=1)
tdecimate(display=false,cycleR=15,cycle=86)

Nope again, the same two missing frames and the wrong final framerate. Anything that doesn't return 24.975fps is wrong.


In reading the TFM documentation, there is a parameter that sounds like it might solve the problem:
It won't as there are no field order problems with the sample. The D2V setting is mainly for helping when there's a mix of hard and soft telecine.

TIVTC isn't optimum for this video.

[johnmeyer]

OK, I guess I'm an idiot. I'm sorry I spent the time doing this.

[manono]

It's a learning experience sometimes just to discover how many ways a simple video can be completely messed up by the so-called 'professionals'. There are so many ways to screw up PAL2NTSC when it's so easy just to do it correctly the first time. The mind boggles sometimes because it's then up to us amateurs to try and put things right. I was stumped too, and had tried all sorts of things with TIVTC before giving up, but had had experience with SmartDecimate years ago and decided to give it a try. Maybe there are other and more simple ways to solve this one, but that will have to await the efforts of someone else.

[DoctorM]

@johnmeyer - If I read you right, you didn't get TFM to work then? It seems apparent that TFM won't properly process an orphaned field.

Besides, I don't know how I feel about using a 43 frame range. That'll take forever to process.

@manono - Fixed my bobber issue. You need to get the bobbing out of the way before SmartDecimate made things awkward.

MPEG2Source("Test.d2v")
yada=yadif(mode=1)
SmartDecimate(30,60,bob=yada)
TDecimate(Mode=0,Cycle=12,CycleR=2)

Btw, I see what you mean about the odd pattern. The clip I posted generates a 7 7 5 7 5. The full version produces a slightly different one with one 9 frame run.

Yet once fully run through the script there are no duplicate frames or apparent missing ones (the movement is smooth).

With the 30/5 numbers I did see some duplicates and missing frames in the video 1-2 seconds before the clip I posted (when there was a smaller difference for the frame matcher to detect).

[Gavino]

Fixed my bobber issue. You need to get the bobbing out of the way before SmartDecimate made things awkward.
...
yada=yadif(mode=1)
SmartDecimate(30,60,bob=yada)
Not sure what you mean here.
Internally, that code is exactly the same as manono's
SmartDecimate(30,60,bob=yadif(mode=1))

Your earlier problem almost certainly has something to do with incompatibilities between the built-in LoadCPlugin() and the older avisynth_c.dll (which declares its own LoadCPlugin() function).
I suspect Yadif needs the built-in version while SmartDecimate needs the older one, so the order of loading the dll's becomes important. Hence manono's solution of manually loading the plugins in the appropriate order.

[DoctorM]

Except manono's script as he posted it did not work for me. Yes, I loaded the plugins in the same order he did, but for me I had to actually use Yadif before SmartDecimate.

[Gavino]

That's strange, since the change you made makes no difference to the order in which the functions are actually called internally by Avisynth.

Perhaps the underlying problem is really one of random memory corruption, so changing the script slightly affects whether it works or not.

Incidentally, have you tried it without loading avisynth_c.dll at all (and deleting it from the plugins folder)? (I don't know enough about SmartDecimate to know whether or not it should work just using the built-in LoadCPlugin().)

[DoctorM]

No, SmartDecimate needs the author's own avisynth_c to work. It's a shame the filter is so dated since apparently it still has its uses.

As an aside, the audio pitch seems correct so I tried to time expand it from 25 to 23.976, but neglected to remember the actual framerate is 24.975.
So for future interest, the correct adjustment using BeSweet appears to be: -soundtouch( -tempo -4.0 )

Edit: While it might appear the source video was PAL poorly converted to NTSC, I'm more convinced this was some interesting "Time Compression" method used to make it fit a time slot.

[DoctorM]

Thanks to everyone that helped with this. The final video, while not fantastic is tolerably good.

I do see the occasional stutter from a dropped frame about once every 5 minutes. I suspect this is a rounding error from the frame rate becoming 24.975 instead of 25.
Considering it would be outrageous to do something like: TDecimate(Mode=0,Cycle=2997,CycleR=497) to get it bang on, it'll do.

[ChiDragon]

Source looks like VHS. What is the lineage of this sample?

I would need several minutes of the source to provide better advice, but I have solved this clip: WeaveMan result (http://www.mediafire.com/?e7kodx9tz58nads) (XviD)

It would be quicker and produce better quality to IVTC directly, but I know of no filter that is tunable to match fields for frame rates besides 29.97.

It never hurts to be familiar with the brute-force method of standard IVTC: http://avisynth.org/mediawiki/ShowFiveVersions

If there were no dropped fields and you had infinite time and patience, some combination of DoubleWeave().Pulldown(x,y) would remove the pulldown pattern. But here's how I used to use my WeaveMan (http://forum.doom9.org/showthread.php?t=167875) plugin.


MPEG2Source("Iclip.d2v")

TFM(mode=0,slow=2,micmatching=0,PP=1,output="Iclip.TFM.txt") # Run a video analysis pass in VDub



MPEG2Source("Iclip.d2v")

#TFM(mode=0,slow=2,micmatching=0,PP=1,output="Iclip.TFM.txt")
WeaveMan(order=1,input="Iclip.TFM.txt",output="Iclip.WMout.txt") # Just open and close the script to create the output text file


My plugin apparently has a bug with the final group of stats but here is the resulting text file up to the portion that is garbage.

3 cc-pcXpcc-pN = 3 in 16
19 ppN = 1 in 7
26 ccc-XXpN = 2 in 12
38 ccNN = 2 in 12
50 pXNN = 2 in 12
62 ppN = 1 in 7
69 ccRN = 2 in 12
81 pcNN = 2 in 12
93 cXpcc-pN = 2 in 12
105 ppN = 1 in 7
112 ccR = 1 in 7

We can ignore the first line, and try to group the others.


19 ppN = 1 in 7
26 ccc-XXpN = 2 in 12
38 ccNN = 2 in 12
50 pXNN = 2 in 12

62 ppN = 1 in 7
69 ccRN = 2 in 12
81 pcNN = 2 in 12
93 cXpcc-pN = 2 in 12

To start with, focus only on the "X in Y" numbers. In terms of the number of duplicates determined to exist, we have found a repeating cycle: 7 in 43. For a frame rate, that gives us (43-7)/43*30000/1001 = 25.0912. Of course I have no clue if this actually holds up for the entirety of your source.

Here is a script showing what it would look like if you stop here rather than going for all-out manual control.


MPEG2Source("Iclip.d2v")

Bob() # For speed. Replace with a smart bobber that first attempts to match fields. One is: TDeint(mode=1,tryWeave=true,slow=2)
LanczosResize(960,720)
TDecimate(cycleR=25,cycle=43,display=true) # Frame 92 (source 220) is a dupe


I forget the math involved in why cycleR becomes 25 instead of 7 for a bobbed input... I just cheated and guessed until I found the matching frame rate.

For a manual pattern that perfectly undoes the cadence in this sample, you need to copy and paste one of the patterns from the WMout.txt file into a WMin.txt "ovr" file and play around with it. Your goal is to reduce it down to as many N's and c's as possible, using p, X, Y, R, etc. only when absolutely necessary. Eventually I came up with this:


-19,0 ppNccRNccNNcXNN
30 X # inframe 25 looks interlaced otherwise, due to an AGC freakout


"-19" just means that the pattern was determined using frame 19 as the starting point and we want it repeated all the way back to frame 0.

[ChiDragon]

Whoops, I should have responded to these posts first.

Anything that doesn't return 24.975fps is wrong.

As mentioned earlier, time compression means that isn't true. There are also real 25.000 -> 29.970 pulldowns.

There is something in the TFM logic that I don't quite understand in how it post-processes the individual fields that have no match. If you set "order=1" TFM will correctly post-process the unmatched odd fields. For these single fields (shown by the "1" in the pattern above) it will give you a frame that contains the original unmatchable field plus a synthesized field, created using whatever technique you choose using the "pp" parameter. However, it skips the even fields. If you instead set "order=0," it then does the even fields but skips the odd fields.

It's because the orphan exists in the top field in the first instance, and the bottom field in the second. The field matcher would need to compare to surrounding frames to know that in the second instance it should keep the bottom field and throw away the top.

(Disclaimer: there's probably something I'm missing since tritical is obviously considerably smarter than me and invested more time in this than I ever did.)

I've never understood why, but TFM does everything it possibly can to create a field match, even when the matches it generates very obviously cause extra duplicate frames. You have to dig deep into the documentation to disable some of the features that don't seem to serve any purpose (even theoretically) if smooth motion is a goal for the output and not just complete frames.

It defaults to mode 1 even though the documentation notes that it creates a jerky output, and from v0.9.10.0 the micmatching feature has to be disabled for mode 0 to always function how you would expect it to.

Plain 2-way matching (option 0) is the safest of all the options in the sense
that it wont risk creating jerkiness due to duplicate frames when possible,
but if there are bad edits or blended fields it will end up outputting combed
frames when a good match might actually exist.

In the case of a bad edit, this "good match" will either (1) be dropped by TDecimate at the edit point anyway or (2) stay as an extra duplicate. How can this ever be considered a good thing? What you actually want to do for what tritical describes as this 3rd match is interpolate the opposite-order field, since it contains temporal information that is thrown away by mode=1.

[manono]

I would need several minutes of the source to provide better advice, but I have solved this clip
So did we, and a helluva lot more easily than you did, if the explanation in your post is any indication. My main problem was in getting SmartDecimate to run properly, not in figuring out the true framerate. All we had to work with was the sample and I agree that with the entire video maybe a better effort could have been made.


As mentioned earlier, time compression means that isn't true.
Having the 'real' framerate being either 24.975 or 25fps is too much of a coincidence for it to be time compression for a modern TV show. Time compressed videos have all kinds of weird-ass framerates. As I also think, the original source looks a lot like a VHS tape which pretty much rules out time compression I would think. But I'd also be curious as to where this thing came from. I don't think that ordinarily a TV network would use a VHS tape as a source for broadcast, and they'd be even less likely to then perform time compression on it. I've never heard of time compression being used for a DVD, if that's what this is, but there's always a first time, I suppose. But whatever fly-by-night company made this thing wouldn't even know what time compression was, much less purposely use it for no good reason at all. You're thinking the source before the supposed time compression was progressive 23.976fps hard telecined to 29.97fps?

There are also real 25.000 -> 29.970 pulldowns.
That may be, but not often and not here. At least the sample doesn't give any indication of that. And even if DoctorM thinks it should be 25fps based on having the complete video, he can make it so without any great difficulty. Which doesn't negate the value of using SmartDecimate to keep the bobbed orphaned fields in the final output.

[DoctorM]

I can provide a larger clip if you'd all like to take another look. I don't recall anyone asking. How big a clip is ideal?

The source is an old VHS recording from TV. That's why I considered time compression to be a possibility.

Edit: ChiDragon: Yikes. I don't even have an opinion.

[ChiDragon]

So did we, and a helluva lot more easily than you did, if the explanation in your post is any indication.

I thought your comment about a jump on frame 15 was with your SmartDecimate solution, but I see it was above the EDIT. In case you're wondering why I went to all the trouble, I had been looking for a test case to re-learn how to use the filter that I posted in Development recently. I feel that using it is faster than manually counting fields as johnmeyer did, but you may disagree.

Having the 'real' framerate being either 24.975 or 25fps is too much of a coincidence for it to be time compression for a modern TV show.

He did say there is a stutter every ~5 minutes with that frame rate. Since I calculated the frame rate as 25.0912, the difference between that and 25 can't be seen with a clip shorter than 11 seconds.

As I also think, the original source looks a lot like a VHS tape which pretty much rules out time compression I would think. But I'd also be curious as to where this thing came from.

After hanging around the originaltrilogy.com forums one learns that LaserDisc releases used time compression all over the place because of length-per-side issues. Similarly, a lot of retail VHS movies were time compressed since not every movie can fit on a 2-hour SP mode tape or justify two tapes.

You're thinking the source before the supposed time compression was progressive 23.976fps hard telecined to 29.97fps?

My thinking was progressive with a speedup and then pulldown.

@DoctorM: It's not as complicated as it first appears, since most of the work is automated. How about a minute clip for now?

[johnmeyer]

Chiman,

Thank you for your posts. You made me realize that my posts above were not as stupid as I was lead to believe.

I also thank you for your WeaveMan plugin. It may help me reduce the workflow in transferring movie film using my shutterless movie projector. I described that invention here:

High Speed Film to Video Transfer Machine -- Need IVTC Help (http://forum.doom9.org/showthread.php?t=106837&highlight=shutterless)

I currently use TFM in two-pass mode, as you describe above, but I manipulate the metrics inside of an Excel spreadsheet in order to identify the candidates for decimation. It is a pathetic kludge, and I've been meaning to create something like WeaveMan for a long time. I'm hoping I can adapt your work to my situation, so thank you very much for posting it.

[manono]

He did say there is a stutter every ~5 minutes with that frame rate.
Yes he did. But there's lots we don't know. If he used his modified script (modified from the one I proposed), perhaps there was too long of an interval before the cycle repeated sometimes and he got missing frames. If so, there should also be duplicate frames to compensate if it's 24.975fps.

And as you also know, the difference between 24.975 and 25fps is one frame about every 40 seconds, not every 5 minutes. This gets tricky, of course, because sometimes they may be hidden by static scenes. But even so, I'd suspect imperfect decimating over time compression here. If he's right about the frequency of missing frames at 24.975fps, if it's really 25fps not only would the missing frames be added back in, but a bunch of duplicate frames would be in there as well.
Since I calculated the frame rate as 25.0912, the difference between that and 25 can't be seen with a clip shorter than 11 seconds.
Yes, a much longer sample might (and might not) help solve the dilemma. If it's really 25.0912fps then, of course, time compression is the answer. Except for the odd nature of that sample though, I see no evidence at all of that being the case. I chalk it up to an unusual kind of standards conversion.

He also just said it's a VHS tape of a TV show. So, if the tape is ... say ... 20 years old, do you think TV broadcasts used time compression back then?


Thank you for your posts. You made me realize that my posts above were not as stupid as I was lead to believe.
Nowhere did I say your post was stupid, just wrong. You presented results that included missing frames. Even you, in your qualifications, apparently realized your results weren't very good. If so, why post then?

[ChiDragon]

He also just said it's a VHS tape of a TV show. So, if the tape is ... say ... 20 years old, do you think TV broadcasts used time compression back then?

If it's a TV show then I agree with you, but we'll have to wait for him to chime in again since he didn't actually say it's a series. A movie shown on TBS in 1993 with time compression? I'd buy it.

15 years ago (https://bulk.resource.org/courts.gov/c/F3/160/160.F3d.537.96-55991.html): 'Tristar decided to proceed over Apted's objection and hired an editor who made 270 separate cuts totaling 22 minutes, sped up the credits to steal two more minutes, and compressed the rest of the film electronically to gain another four.'

I found that by searching for the phrase "edited to run in the time allotted" which you would commonly see on TV airings of movies in the '90s. Of course they don't disclose in that short disclaimer whether the editing is trimming or speeding-up.

[johnmeyer]

Even you, in your qualifications, apparently realized your results weren't very good. If so, why post then?Because I was trying to help! If perfection is a requirement for posting, then none of us should bother.

[manono]

If it's a TV show then I agree with you, but we'll have to wait for him to chime in again since he didn't actually say it's a series. A movie shown on TBS in 1993 with time compression?
My fault. I wasn't clear. By 'TV show' I meant anything shown on TV, including movies. Maybe the year the tape was made is important. I didn't know time compression was used beginning more than a few years back but apparently your link proves it's been around much longer than I had thought.
...which you would commonly see on TV airings of movies in the '90s.
Right, I always assumed it meant cutting out sections of the movies. Maybe I was wrong and it might have meant more than that.

[DoctorM]

It's quite correct, this is a VHS recording from a cable broadcast (Family Channel) from the 90's (not sure of the exact year).

I'll have to check the tape again, but I don't even remember there being a notice that it was time compressed or anything... but that doesn't mean it wasn't. I assume they meant to hit a target run time of 2 hours and still fit plenty of commercials.

As request, a 1+ minute clip. (http://www.mediafire.com/download/2kzq2sfzzhmq9wz/RAW_IRBR_Clip2(3).mpv)

Also here was my full script:
loadplugin("C:\AviSynth\plugins\dgdecode.dll")
import("C:\AviSynth\plugins\limitedsharpenfaster.avs")
loadcplugin("C:\AviSynth\plugins\yadif.dll")
loadplugin("C:\AviSynth\plugins\fft3dgpu.dll")
loadplugin("C:\AviSynth\plugins\tivtc.dll")
loadcplugin("C:\AviSynth\plugins\yadif.dll")
loadplugin("C:\AviSynth\plugins\avisynth_c.dll")
loadcplugin("C:\AviSynth\plugins\SmartDecimate.dll")

mpeg2source("Source2.d2v",cpu=1)

crop(4,0,-10,-6).addborders(4,0,10,6)
yada=yadif(mode=1)
SmartDecimate(30,60,bob=yada)
TDecimate(Mode=0,Cycle=12,CycleR=2)
assumefps(23.976)

crop(4,0,-10,-6)
spline36resize(716,480).addborders(2,0,2,0)

fft3dgpu(sigma=1.5)
limitedsharpenfaster().limitedsharpenfaster()
converttoyuy2()

[ChiDragon]

For simplicity this is my quick test version of your script.

loadcplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\SmartDecimate.dll")

MPEG2Source("RAW IRBR Clip2(3).d2v")

SmartDecimate(30,60)
TDecimate(Mode=0,Cycle=12,CycleR=2,display=true)

You can quickly see that these aren't the correct cycle/cycleR values, as every few cycles the layout of the uniques and dupes changes. Input frame 206 is dropped despite being unique, as there is only one dupe during that cycle. Same with 727, 1009, 1501, 2064 (probably more between those last two but it's a low-motion scene). 5 in 30 incorrectly drops 210, and 7 in 43 isn't correct either.

I don't suppose there is a full minute of big movements and camera pans anywhere...

EDIT: 21 in 129 (which is just three 43-frame cycles) gets closer but the pulldown pattern is actually hundreds of frames long.


1308 ppN = 1 in 7
1315 ccRN = 2 in 12
1327 ccN = 1 in 7
1334 pcc-ppXNN = 3 in 17
= 43

1351 ppN = 1 in 7
1358 ccR = 1 in 7
1365 pcc-pccNN = 3 in 17
1382 cXNN = 2 in 12
= 43

1394 ppN = 1 in 7
1401 ccR = 1 in 7
1408 pcc-pccNN = 3 in 17
1425 pXNN = 2 in 12
= 43

1437 ppN = 1 in 7
1444 ccRN = 2 in 12
1456 ccNN = 2 in 12
= 31

1468 cXN = 1 in 7
1475 pcc-pppN = 2 in 12
1487 pcc-c-ppcc-pccNN = 5 in 24
1511 ppNN = 2 in 12
= 55

1523 ppN = 1 in 7
1530 ccNN = 2 in 12
1542 ccNN = 2 in 12
1554 ppNN = 2 in 12
= 43

1566 ppN = 1 in 7
1573 ccRN = 2 in 12
1585 ccN = 1 in 7
1592 pcc-pcXNN = 3 in 17
= 43

1609 ppN = 1 in 7
1616 ccRN = 2 in 12
1628 ccNN = 2 in 12
= 31

1640 ppN = 1 in 7
1647 pcc-pppN = 2 in 12
1659 ccNN = 2 in 12
1671 cXNN = 2 in 12
= 43

1683 ppN = 1 in 7
1690 ccRN = 2 in 12
1702 ccNN = 2 in 12
1714 cXpcc-pN = 2 in 12
= 43

1726 ppN = 1 in 7
1733 ccRN = 2 in 12
1745 ccNN = 2 in 12
= 31

1757 ccN = 1 in 7
1764 pcc-pppN = 2 in 12
1776 ccRN = 2 in 12
1788 ccNN = 2 in 12
1800 cXNN = 2 in 12
= 55

1812 ppN = 1 in 7
1819 ccR = 1 in 7
1826 pcc-pccN = 2 in 12
1838 pcc-pcXNN = 3 in 17
= 43

1855 ppN = 1 in 7
1862 ccRN = 2 in 12
1874 ccNN = 2 in 12
1886 cXNN = 2 in 12
= 43


31+55 = 86 but there is that 31 in the middle of two 43's that throws a spanner into the mix. Now you see why I gave up on my TNT captures. Why not just leave it at 60Hz, by the way?

EDIT2: This appears to do the trick for the sample.


LoadCplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins - old C\avisynth_c.dll")
loadcplugin("C:\Program Files (x86)\AviSynth 2.5\plugins - old C\SmartDecimate.dll")

MPEG2Source("RAW IRBR Clip2(3).d2v")

yada=yadif(mode=1)
SmartDecimate(30,60,bob=yada)
#TDecimate(output="IClip.TDec.txt",mode=4) #Run a video analysis pass in VDub
LanczosResize(1440,1080)
TDecimate(input="IClip.TDec.txt",batch=true,mode=0,sdlim=4,cycle=289,cycleR=47,display=true)

#Field match errors at 242 (289) and 1472 (1758)


Note the sdlim parameter, which you definitely want to use for a cycle this long (and is a good idea to use anyway).

This gives 25.096 fps. A sample longer than 208 seconds would be the next step.