Can TIVTC preserve every field

[sonic41592]

I was using TIVTC on hybrid DVD sources which have sections of telecine and interlacing to produce a VFR video. The timecode file it outputs shows that the interlaced sections are played back at 30 fps after processing, and analyzing the video shows that it deinterlaces the frame using the top field, but discards the bottom field. Is it possible to have TIVTC assume 59.94 fps instead of 29.97 for the interlaced sections, and deinterlace each field instead of omitting one?

My code is the standard TIVTC case:
First pass:
mpeg2source("c:\oursource.d2v")
tfm(d2v="c:\oursource.d2v",output="matches.txt")
tdecimate(mode=4,output="metrics.txt")

Second pass (anime or cartoon):
mpeg2source("c:\oursource.d2v")
tfm(d2v="c:\oursource.d2v",input="matches.txt")
tdecimate(mode=5,hybrid=2,vfrDec=1,input="metrics.txt",tfmIn="matches.txt",mkvOut="mkv-timecodesfile.txt")

Thanks in advance!

Also, can the --tcfile-in be used in FFMpeg? I can always use mkvmerge to mux in the timecodes, but it would be much easier to do in one pass.

[sonic41592]

I think I've finally figured out a good method for this, thanks to some posts from @thecoreyburton several years ago https://forum.doom9.org/showthread.php?t=173147. I'm very happy with the way this is turning out despite it requiring 5 passes and still having a couple bugs to work on.

The first two passes use TIVTC to inversetelecine the source to create a lossless VFR video with timecodes file while leaving the truly interlaced frames untouched and still combed. TIVTC has a feature to use a separate deinterlacer to process the interlaced frames, stored as clip2. Instead of deinterlacing, this will feed the unprocessed combed frames right back into place, giving a video with 24 fps sections that were IVTCed and 30 fps sections of interlaced frames:

Pass 1

FFMPEGSource2("source.mkv",rffmode=1)
b=last
TFM(PP=2,clip2=b,output="TFM1.txt")
TDecimate(mode=4,output="TDecimate1.txt")

Pass 2

FFMPEGSource2("source.mkv",rffmode=1)
b=last
TFM(PP=2,clip2=b,input="TFM1.txt")
TDecimate(mode=5,hybrid=2,vfrDec=1,input="TDecimate1.txt",tfmin="TFM1.txt",mkvOut="Timecodes1.txt",tcfv1=false)

This video is saved as IVTC.mkv or whatever. For the third pass, the original source is bob deinterlaced using whichever method seems best, I used QTGMC, to give a 60 fps output. This output is then changed to 360 fps, so that each frame has 6 duplicates. 360 fps is used because it is the least common multiple of 18, 24, 30, and 60 fps. (Occasionally, TIVTC will perform a double decimation on a group of 5 telecined frames, causing 5 frame sections of 18 fps):

Pass 3

FFMPEGSource2("source.mkv",rffmode=1)
QTGMC( Preset="slow", SourceMatch=3, TR2=1, Lossless=2).changefps(360000,1001)

This video is saved losslessly as QTGMC.mkv or whatever. Before combining the two, the IVTC.mkv video must be changed to 360 fps as well using VFRtoCFR. This causes all the 18 fps sections to be duplicated 20 times, the 24 fps sections duplicated 15 times, and 30 fps sections duplicated 12 times. Now we have two videos with exactly the same length and constant frame rate, so each IVTC and interlaced section of the two videos are properly aligned. The ConditionalFilter is used to find the frames that are still combed in the IVTC.mkv video and replace them with the corresponding frame from the QTGMC.mkv video. This causes the interlaced 30 fps sections to be converted to 60 fps, preserving each field. The video then has to be deduplicated with ExactDeDup, which requires 2 passes, and spits out a final timecode file to be muxed with output. This video will have 24 fps IVTC sections (and the pesky 18 fps sections), 60 fps deinterlaced sections (saving each field), and also maintain 30 fps progressive sections:

Pass 4

clip1=FFMPEGSource2("IVTC.mkv",rffmode=1).VFRToCFR(times="Timecodes1.txt",numfps=360000,denfps=1001)
clip2=FFMPEGSource2("QTGMC.mkv",rffmode=1).Assumefps(360000,1001)
ConditionalFilter(clip1,clip2,clip1,"IsCombedTIVTC","=","True").Assumefps(360000,1001)
ExactDedup(firstpass=true,dupinfo="info.txt",times="Timecodes.txt")

Pass 5

clip1=FFMPEGSource2("IVTC.mkv",rffmode=1).VFRToCFR(times="Timecodes1.txt",numfps=360000,denfps=1001)
clip2=FFMPEGSource2("QTGMC.mkv",rffmode=1).Assumefps(360000,1001)
ConditionalFilter(clip1,clip2,clip1,"IsCombedTIVTC","=","True").Assumefps(360000,1001)
ExactDedup(firstpass=false,dupinfo="info.txt",times="Timecodes.txt")

In the thread I used as a guide for this method, @StainlessS was able to condense multiple passes into a single pass, however I don't know how to go about doing that for these steps. There is also some optimizing to be done, as well as multi-threading some of the passes. Any thoughts and inputs are welcome!

Bugs:
1. Occasionally, there is a single 120 fps frame at the start/end of when frame rate switches from 24 fps to 60 fps. I believe this is caused by bug #2, because interlaced frames are being left in 24 fps sections, causing a mismatched split.
2. Many of the fade in/out scenes are still combed. This can be fixed easily with an antialiasing filter, but I'd rather not rely on that. Some of these combed fade ins/outs are also present on the standard TIVTC NTSC case 7 (2 pass VFR 24fps 30fps output). So these frames are not being detected as being combed by TIVTC. Any suggestions on modifying the comb detection for the original TIVTC and/or ConditionalFilter detection? Here are a couple very short fade in samples. Source1.VOB has no combing after using the TIVTC case 7 output, but does have combing using the 2 pass method IVTC in this post, even though it registers it as 24 fps. These interlaced frames in 24 fps section must cause the conditionalfilter to pull frames from QTGMC.mkv causing a sporadic 120 fps frame. Source1 is from the very beginning of the episode, and theoretically should be 30 fps first and switch to 24 fps, although TIVTC labels it as 24 fps for 4 frames, then 30 fps for 5 frames, then back to 24 fps. Source2 has combing using both the standard and this method. TIVTC does not recognize the fade in on the moon as being a 30 fps section, although there are several back to back interlaced frames.

[sonic41592]

The 'single 120 fps frame' problem comes from interlaced frames being left in the 24 fps sections. TFM is field matching sections that should be video and leaving behind combed frames. Normally, it would recognize the frame as combed and deinterlace it. This can be seen with this short clip. During the fade in, the first frame is black and the second frame is interlaced with one field being black. I'm guessing the two get field matched as a black frame, and TFM passes on to TDecimate that it should be a 24 fps section, even though there is combing on every other frame in the group. Ideally, the variable frame rate file should have frames 0-6 as a 30 fps video section, but it always starts off with a 24 fps film section.

Here's the code I've been playing around with

First Pass:

ffmpegsource2("source1.VOB",rffmode=1)
TFM(PP=1,cthresh=4,MI=80,output="TFM.txt",slow=2)
TDecimate(mode=4,output="TDecimate.txt")

Second Pass:

ffmpegsource2("source1.VOB",rffmode=1)
TFM(PP=1,cthresh=4,MI=80,input="TFM.txt",slow=2)
TDecimate(mode=5,hybrid=2,vfrDec=1,input="TDecimate.txt",tfmin="TFM.txt",mkvOut="Timecodes.txt")

Is there any way to tell tfm not to field match if it leaves behind combed frames? I really just need TDecimate to make all combed frame sections 30 fps.

[StainlessS]

Here single function ForceProcess() plugin. [Separated from TwriteAVI2()]:-
ForceProcess_x86_x64_dll_v1-00_20180724.zip
http://www.mediafire.com/file/xyvpru...80724.zip/file

ForceProcess(), v1.00, By StainlessS @ Doom9

Requires VS2008 CPP runtimes,

ForceProcess(clip, bool "Video"=True,bool "Audio"=false,Bool "Debug"=True) :- Returns 0


If Video (default true),  Force read every frame of clip from 0 to FrameCount-1.
If Audio (default false), Force read every sample of clip from 0 to NumberOfSamples-1.
Useful where a clip outputs some kind of file for use in a second filter, this function would in such a case
forcibly read and therefore write the file, so that it may be available to other filters or to a second pass of  the filter that
initially wrote the file.
Debug, default true. Write Progress to DebugView.

The ForceProcess function forcibly processes both video and audio streams (ie reads each frame/sample in sequence),
so that any eg log files produced by either video and/or audio processing filter will be forcibly written to disk.
ForceProcess is a runtime function not filter and so only returns on completion.

Example:

    Two Pass:

        WhateverSource(...)
        b=Last  # Added
        A_Two_Pass_Function(Pass=1) # Create some kind of deshaker log or whatever.
        ForceProcess()              # Force Pass 1, creating log
        b    # Added
        A_Two_Pass_Function(Pass=2) # Use created log
        return Last

1st 3 passes, Save as QTGMC.mkv

FFMPEGSource2("source.mkv",rffmode=1)
b=last

# Create pass text files
TFM(PP=2,clip2=b,output="TFM1.txt")
TDecimate(mode=4,output="TDecimate1.txt")
ForceProcess()              # Force Pass 1

b  # Source again
TFM(PP=2,clip2=b,input="TFM1.txt")
TDecimate(mode=5,hybrid=2,vfrDec=1,input="TDecimate1.txt",tfmin="TFM1.txt",mkvOut="Timecodes1.txt",tcfv1=false)
ForceProcess()              # Force Pass 2

b  # Source again
QTGMC( Preset="slow", SourceMatch=3, TR2=1, Lossless=2).changefps(360000,1001)

Untested.

Hope you can figure out the rest.
I've just recently realized that TWriteAVI screws up writing AVI files bigger 4GB (or maybe 2GB),
I've never really tried to write files that big with it.
Source code was taken from Vdub 1.4 (I think, by someone else) and I aint got a clue how to
mod it for OpenDML AVI (4GB+).
If it did work for OpenDML style AVI's then could perhaps have made above + the other passes into a single one.

I cant spend any more time on it, its 03:34AM here and I'm in pain with a damn shoulder cramp due to being hunched over my
machine for the last day and a half, I'm in the process of finalizing a plugin and docs and scripts for a plug update, so
I aint gonna gett any shuteye just yet, and I wants some, me is knackered.

good luck.