Hybrid VFR with 24, 30 and 60FPS

I have a couple of DVDs that I'd like to mux into mkv, but I'm trying to work out the best way to do it. The DVDs are all NTSC and play at 29.97fps. They have telecined film sections, progressive 29.97 sections, and interlaced 29.97 sections.

Previously I've handled this with TFM & TDecimate's mode 4 and 5 functions and it's done the job pretty well. The only catch is in the deinterlacing - I'm stuck with a VFR clip that runs at either 23.976fps or 29.97. In some of the video clips there's quite a bit of fast scene swapping and the best way to capture that would be to bob the clips when they're interlaced, QTGMC being my preference here.

Is there a way to field match the film sections, not touch the progressive sections and merge them all down into a VFR clip?

I've had three somewhat successful attempts so far:
1: TDecimate gave me a 24/30 VFR.
2: TFM, ChangeFPS QTGMC and Dedup combined with gave me the correct frames - but because of the nature of exact dedup the 24fps sections were played as if they were at 30fps.
3: TDecimate gave me a 60/48 VFR. This is probably the best outcome I've had so far as all the timings are correct, there were just a lot of frames that I didn't really need in the file and I'd like to get rid of those if possible.

I've heard good things about AnimeIVTC being able to handle 60/30/24 but as far as I can tell (I have 2.00) I can only achieve 30/24.

If you'd like a sample you're welcome to request one.

[ChiDragon]

I've never really tried VFR myself, but my understanding is this requires a 120fps intermediate step rather than 60fps.

[kuchikirukia]

60fps doesn't capture anything about a 30fps clip better than 30fps.

[ChiDragon]

With regard to actual interlaced content, you're mistaken. 30fps is really 60Hz = 60 unique moments in time captured with half vertical resolution.

Originally Posted by kuchikirukia

60fps doesn't capture anything about a 30fps clip better than 30fps.

It depends on the content. If it's 30fps content without interlacing you're correct, as is true if it's a bad transfer - in this case ChiDragon's spot on. Some of the frames are truly interlaced, meaning when they're properly bobbed I can have 60 unique frames a second (often 30fps is acceptable but on the off-chance there's faster paced content or you want the full experience then there's definitely a reason to get the footage to 60).

Originally Posted by ChiDragon

I've never really tried VFR myself, but my understanding is this requires a 120fps intermediate step rather than 60fps.

That was my goal with the dedup attempt. I was thinking perhaps TFM with no post processing & tdecimate modes 4 and 5. That'd create a VFR clip with both 30i and 24p. I'd output that to an intermediate AVI which could load that up and use VFRtoCFR with 120000/1001 (120fps). The only problem now would be the interlacing, as deinterlacing wouldn't work with all the duplicate frames.

To counteract this I could use a IsCombedIVTC and use clip2 as follows: load the clip up again with VFRtoCFR 30000/1001, apply QTGMC and then ChangeFPS to 120000/1001. Essentially that'd be the same clip, but bobbed and it'd only replace the frames in the original that have combing. Then do a dedup (or exactdedup would be better in this case).

There are two large issues with this though.
1) The process requires four passes.
2) It's inaccurate, sometimes it works, other times the footage is jumpy and bobs either incorrectly or replaces the wrong frames.

I'm thinking there has to be a simpler way to do this and I've over-complicated it, but I certainly agree that getting it to 120fps seems to be the likely way to go as it's the lowest common multiple of all 3.

[ChiDragon]

The TDecimate documentation mentions 120fps with regards to mode 6. Surely there must be posts somewhere about the rest of this workflow.

I've found some, but mostly it's about re-encoding VFR (for example after loading a 24/30 VFR clip, applying the timecodes in avisynth and setting the clip to 120fps, it'll work in the same way ExactDedup would and would remove any exact duplicates). Given it can do that and also create 24/30 clips I'd assume there'd be a way to get it to do what we're asking of it. I'll keep looking into tdecimate documentation and other potential cases of the same problem.

I was working on an idea based off the following:

Movie=DGDecode_MPEG2Source("video.d2v")
Audio=FFAudioSource("audio.ac3")
Source=AudioDub(Movie, Audio)
IVTC=Source.QTGMC()
EvenFrames=Source.TFM(PP=2, clip2=IVTC.SelectEven)
OddFrames=Source.TFM(PP=2, clip2=IVTC.SelectOdd)
Interleave(EvenFrames, OddFrames)
ChangeFPS(Movie.Framerate*4)

With TDecimate using modes 4 and 6 on each respective pass. Those lines are emitted simply to keep this post shorter but would be at the end.

It's good in theory and decimates all the duplicates - but because it doesn't use any TFM input (due to having called TFM twice), the TFM sections are left in the form of abcdd with a double at the end.

A closer examination of the clip without any TDecimate functions shows that the two last frames are similar but not identical.

Movie=DGDecode_MPEG2Source("video.d2v")
Audio=FFAudioSource("audio.ac3")
Source=AudioDub(Movie, Audio)
IVTC=Source.QTGMC(FPSDivisor=2)
Source.TFM(PP=2, clip2=IVTC.SelectEven, output="TFM.txt")

Same deal about the TDecimate lines being omitted, but this time with modes 4 and 5 respectively.

This decimates those sections perfectly, the abcdd sections become simply abcd and everything lines up perfectly.

Inspection of the clip without any TDecimate functions shows that the last two frames are once again very similar but not identical. Because this method does use a TFM input though, the duplicate is removed on the second pass.

I figured that the only reason the mode 6 script above didn't work is because the TFM sections weren't perfect duplicates and I was thinking about it for a bit. I came to the conclusion that if perhaps I did TFM and mode 5, then loaded the VFR clip back up the frames would have to duplicate, then I could do mode 6. I attempted a four pass shot at this:

Pass #1: Ran this as an analysis pass in Virtual Dub.

Movie=DGDecode_MPEG2Source("video.d2v", cpu=4, info=3)
Audio=FFAudioSource("audio.ac3")
AudioDub(Movie,Audio)
TFM(PP=0, Mode=4, output="TFM-Pass1.txt")
TDecimate(mode=4, output="TDecimate-Pass1.txt")

Pass #2: Saved this as a lossless AVI.

Movie=DGDecode_MPEG2Source("video.d2v", cpu=4, info=3)
Audio=FFAudioSource("audio.ac3")
AudioDub(Movie,Audio)
TFM(PP=0, Mode=4, input="TFM-Pass1.txt")
TDecimate(mode=5, hybrid=2, vfrDec=1, input="TDecimate-Pass1.txt", tfmIn="TFM-Pass1.txt", mkvOut="Timecodes-Pass2.txt")

I converted the timecodes file from v1 to v2 to make it compatible with the next script. I named the file "Timecodes-Pass3.txt" for reference.

Pass #3: Ran this as an analysis pass in Virtual Dub.

AVISource("intermediate.avi")
Source=VFRToCFR(times="Timecodes-Pass3.txt",numfps=30000,denfps=1001)
IVTC=Source.QTGMC()
EvenFrames=Source.TFM(PP=2, clip2=IVTC.SelectEven, output="TFM-Pass3-Even.txt")
OddFrames=Source.TFM(PP=2, clip2=IVTC.SelectOdd, output="TFM-Pass3-Odd.txt")
Interleave(EvenFrames, OddFrames).ChangeFPS(Source.Framerate*4)
TDecimate(mode=4, output="TDecimate-Pass3.txt")

Pass #4: Simply had a scrub through the frames in Virtual Dub and saw the problem here, otherwise this would have been the final output.

AVISource("intermediate.avi")
Source=VFRToCFR(times="Timecodes-Pass3.txt",numfps=30000,denfps=1001)
IVTC=Source.QTGMC()
EvenFrames=Source.TFM(PP=2, clip2=IVTC.SelectEven, output="TFM-Pass3-Even.txt")
OddFrames=Source.TFM(PP=2, clip2=IVTC.SelectOdd, output="TFM-Pass3-Odd.txt")
Interleave(EvenFrames, OddFrames).ChangeFPS(Source.Framerate*4)
TDecimate(mode=6, hybrid=2, vfrDec=1, input="TDecimate-Pass3.txt", mkvOut="Timecodes-Final.txt")

The results of this weren't what I hoped though. It worked a little worse than mode 6 on it's own. Without the TFM in the now-identical frames of the film section still weren't decimated. Four passes felt a bit tedious AND because I had decimated and loaded up the file again, the frame timing and placement varied slightly. This caused stuttering back and forth in that parts which used QTGMC like what you'd see if you had the field order wrong. Because of the nature of the VFR clip though, the stuttering wasn't consistent. In some cases it'd stutter whereas in others it'd deinterlace properly.

After typing all this up it's obvious to me why it didn't work, but hopefully it gives someone an idea on potential solutions, perhaps. Experimenting with TDecimate's advanced options in the hopes one of them may help.

[ChiDragon]

Personally I would just encode a 60fps file with 3:2 pulldown intact and forget about VFR. Unless you actually have a display that you run with a 120Hz source.

As a VFR test I tried that tweaked TFM script in place of yours, but I couldn't get TDecimate to recognize the 3:2 pulldown sections as 24fps no matter what I did. Even when I fed it as 60fps instead of 120fps, the actual decimation of the film sections was mostly correct, but the timecodes file just switched between 59.94 and 39.96 every frame.

I'm seriously considering it. Or even manually analyzing each source individually and determining whether 60/30 or 30/24 would better suit it and doing the appropriate one. I do have a high refresh rate display that I use for both PC and TV though and some of the clips I'm trying to do are what I'd classify as my all time favorites, so the passion and want is there - I'm just not sure if there is a way to do what I'm asking and therefore I'm not if it's worth persisting or just giving up for now.

That's about where I'm at with this too. Best case scenario I can get 2 of the 3 desired rates displaying correctly.

I'm a bit of a novice when it comes to this though, I feel like there's something I'm overlooking or an obvious way to do this. I should probably ask, is TDecimate capable of detecting 3 different frame rates? Perhaps the issue is that it's capped at two per file and it starts at the highest. In cases of 60/30/24 it'd get to 60/30 and then try and decide between them for each scene, in 30/24 it'd only have those two to choose from.

Edit: On that note, if we use your script for keeping the pulldown intact and call TDecimate modes 4 and 5 afterwards we get a 60/48 clip with occasional bursts of 36fps.

[ChiDragon]

If you want to keep trying, I never attempted this combo:

My TFM+QTGMC + Dup (make the "D" in ABCDD an exact dupe) + ExactDedup for VFR (instead of TDecimate)

I think it'd work with decimation, but the frame playback times would be incorrect. ExactDedup would have the frames play back as ABCD-ABCD-, the dash representing where the a frame is held for an extra frame. TDecimate detects these sections and manually sets them to 24fps.

Is there a way to use that method but somehow get it to 120fps with the field-matched sections playing at the correct times? Eg: AAAAABBBBBCCCCCDDDDD rather than AAAABBBBCCCCDDDDDDDD.

[ChiDragon]

You're right. I realized today that's also the problem with TDecimate mode 6 for this purpose. It's the one mode that is designed not to change frame timings, since the idea is to input a 120fps file that already uses only integer frame repetitions (5*24, 4*30, 2*60) with no messy 3:2 or 6:4 pulldown patterns.

And the problem with the other TDecimate modes is as you say, they're only designed to handle two frame rates (base decimation rate and hybrid passthrough rate).

I guess that means this is a bit beyond my scope at the moment. What are the chances of being able to get in contact with Tritical? TIVTC would work with some slight modifications to be able to handle this sort of thing.

[ChiDragon]

I believe he stopped developing anything to do with Avisynth years ago.

That's not very fortunate in this case! I guess I'll have to try various obscure methods and hope something gets it right at this point. I have a possible solution, but I'll have a crack at it before I post it here.

Edit: Here's my newest attempt, I actually got pretty close.

Pass #1:

Movie=DGDecode_MPEG2Source("video.d2v")
Audio=FFAudioSource("audio.ac3")
AudioDub(Movie,Audio)
TFM(PP=0, output="video-tfm1.txt")
TDecimate(mode=4, output="video-tdecimate1.txt")

Pass #2: I saved this pass as "Step2.avi"

Movie=DGDecode_MPEG2Source("video.d2v")
Audio=FFAudioSource("audio.ac3")
AudioDub(Movie,Audio)
TFM(PP=0, input="video-tfm1.txt")
TDecimate(mode=5, hybrid=2, vfrDec=1, input="video-tdecimate1.txt", tfmin="video-tfm1", mkvOut="video-temporarycodes.txt", tcfv1=false)

Pass #3

Movie=DGDecode_MPEG2Source("video.d2v")
Audio=FFAudioSource("video.ac3")
Original=AudioDub(Movie,Audio)
Step2=AVISource("step2.avi").ConvertToYV12().VFRtoCFR(times="video-temporarycodes.txt",numfps=120000,denfps=1001)
IVTC=Original.QTGMC().ChangeFPS(Original.Framerate*4)
Step2.TFM(PP=2, clip2=IVTC, output="video-TFM-Pass3.txt")

Pass 3 (and then 4) would have been identical except for the use of ExactDedup.
By loading in the already decimated clip at 120fps the timings would be correct even for the film sections and frames would simply be duplicated. Since deinterlacing this clip is out of the question, I called the original clip back in and attempted to deinterlace that instead and use it to replace the combed frames in the already decimated clip. The frames weren't played at the same times though it seems. Adding StackHorizontal(Step2,IVTC) makes the problem more apparent and I can see why it's going wrong, I'm just not sure how to fix it.

[ChiDragon]

As I recall, the process to create a 120fps CFR file involved using a crazy combination of poorly documented and/or untranslated Japanese tools that may not even be available anymore.
http://forum.doom9.org/showthread.ph...172#post286172

I'm not entirely sure they even handled 60Hz sections by bobbing. Most anime only used pure interlacing on credits, and even the hardcores don't care much about perfectly smooth motion there.

Wow, 2003. Thank you, that's a hell of a lot of digging you did to help me out. I'll get to work on acquiring all those plugins and testing out various methods of getting this to work. It does look like it's designed specifically for 30/24 as opposed to 60/30/24, but depending on how the plugins work there may be a way of incorporating bobbed footage. I'll work with bob in place of QTGMC until I get somewhere substantial I think, to save time.

[osgZach]

Originally Posted by thecoreyburton

2: TFM, ChangeFPS QTGMC and Dedup combined with gave me the correct frames - but because of the nature of exact dedup the 24fps sections were played as if they were at 30fps.

So why not take the output from that, and then rewrite the timecodes in V2 format? You can specify which frame ranges should run at what FPS.


Any chance of getting a sample of the source file (with all 3 framerates) ?

I have several sources with varying rates at varying points, that's part of the problem. Some are simply credit sections which are very easy to do manually, others are not so simple and have many moments flicking back and forth in a short time. Manual selection on some of them would be too tedious. I'll have a flick through the sources and see if I can find a portion that has all 3, or even interlaced and film (so 60/24). There's probably a case like this on music DVD I have, I'd assume.