TDeint and TIVTC

[thetoof]

Could you add a feature in tdecimate to handle 23.976, 29.97 and 59.94?

Hm, just thought about this:

Can I manually edit the timecodes to look like this - I mean, will it work? (I know I could test, but it's faster to get an answer here )

# timecode format v1
Assume 29.970030
0,39,23.976024
775,778,59,940060
994,997,23.976024

[tritical]

It's possible, but I don't have the time or desire to do it. It isn't a simple modification, at least an easy method isn't coming to mind. The code for tfm/tdecimate is a big mess. It might actually be easier to create a separate filter/program to do it based off tfm stat file and tdecimate stat files from tfm output and bobbed output.

I don't know why manually editing the timecodes file wouldn't work... assuming you splice sections (add the 60p video in the right place)/modify frame numbers as needed.

[thetoof]

Alrighty, thanks for the answer.

[Gser]

Originally Posted by @Tron@

Interested in a plan to update the gorgeous plug -TFM!? I take them a long time and would like to know whether in the near future, its updating and some "delicious"))) because there is no limit to perfection, and should always be striving towards the ideal ))

Like support for dga files. :'(

[johnmeyer]

Is there a way to force TDecimate to decimate all combed frames? I have a source where TFM flags really bad frames as combed frames. I want to decimate these first, and then remove the remaining duplicates to achieve a given frame rate. The MIC metric in TFM shows 255 on these bad frames, so TFM is providing the information I need. I don't mind doing a two-pass, but I cannot figure out how to pass a file back into TDecimate that will force it to decimate these "255" frames first, and then use the metrics from TFM to further decimate by removing the true duplicates.

[edit]
I figured it out. This does what I want:

tfm(display=false,mode=1,pp=1,cthresh=25,micmatching=3,mmsco=false)
tdecimate(mode=7,display=false, rate=23.976)

[ikarad]

Originally Posted by tritical

Some options that could speed up processing are:

tfm:
slow = 0
micmatching = 0 or 2

tdecimate:
chroma = false

Of course, changing those could change the output, but in most cases it probably wont be noticeable. I would try slow=0 in tfm and chroma=false in tdecimate, and see if it is fast enough.

TVITC support rgb or only YUY2 and YV 12 colorspace?

[turbojet]

Hi, is there any chance adding support for reading frame type (film or ntsc) from dgi files to TFM like the d2v function?

Here's an example of a dgi file that has both ntsc and film frames.

[D3C0D3R]

i've use TIVTC couple of years. it amazing filter pack.

I've found serious memory leak (about 1MB-2MB per second) when use tdecimate mode=5 (2nd pass of hybrid)
with serious filter chain - couple of them writted by tritical too )) *BIG RESPECT*

i've placed this chain before and fter tDecimate, but leak is still appear I even remove some memory & cpu most hungry filters but it seems doesnt depend from type of filters and it position in script.

my settings

tdecimate(mode=5, hybrid=2, vfrDec=1,mkvOut=mkvTimeCodeOut,input=metricStats, vidThresh=15,dupThresh=10, vidDetect=1, cycleR = 1, cycle=2)

encoded by VDub+x264vfw one of latest versions/revisions

I didnt use TFM to do output="matches.txt" at first pass.
I haven't use TFM at all, only TDecimate - to remove duplicates in progressive stream.

In mode=3 all works fine, but mode=3 produces blackness at the end of stream and dont change count of frames.

But I wanna use 2-pass (mode=4 stats -> mode=5 work) because i run batch and dont want to crop end of every file manually

Maybe exists some option that I havn't set or i've found a bug?

[tritical]

Could you provide an exact full script that causes the issue? I'm not sure how tdecimate could be leaking memory since in mode 5 it doesn't allocate any memory during frame requests... only during script initialization. Does the memory issue happen if you add setmemorymax(64) to the script (i.e. is this 1-2MB per second just due to avisynth's cache filling up at the beginning).

[D3C0D3R]

>>setmemorymax(64)
yes, i did it at immediately when i found this leak. didnt help.

>>> (i.e. is this 1-2MB per second just due to avisynth's cache filling up at the beginning).
no, it grows all time during encode and finally eat all 3Gb and throw Exception

here my a bit simplified script that causes leak

common init part

src = "s02e01.avi"
metricStats = "metrics_01.txt"
mkvTimeCodeOut = "mkv-timecodesfile_01.txt"
SetMemoryMax(80)
LoadVirtualDubPlugin("MSU_cartoon_restore.vdf","MSUCartoonRestore", 0) 
AviSource(src,audio=false)
Lanczos4Resize(width*2, height*2)
ConvertToRGB32()
MSUCartoonRestore("bilateral", 1, 56, 36, 1)
spline64resize(width/2, height/2)
ConvertToYUY2()

1st pass

tdecimate(mode=4, output=metricStats)
crop(0,0,8,8)

2nd pass

tdecimate(mode=5, hybrid=2, vfrDec=1,input=metricStats, mkvOut=mkvTimeCodeOut, vidThresh=15,dupThresh=10, vidDetect=1, cycleR = 1, cycle=2)

[Zep]

1280 x 720p source
Set MT build 2.5.8

=================================

mpeg2source("D:\whatever.d2v",idct=5)
ChangeFPS(last,last,true)


116 FPS


=================================

mpeg2source("D:\whatever.d2v",idct=5)
ChangeFPS(last,last,true)
Selecteven()

58 FPS

=================================

mpeg2source("D:\whatever.d2v",idct=5)
ChangeFPS(last,last,true)
Selecteven()

TDecimate(mode=0,cycleR=1,cycle=5)

39 FPS


=================================


mpeg2source("D:\whatever.d2v",idct=5)
ChangeFPS(last,last,true)
Selecteven()

BilinearResize(320,180)
TDecimate(mode=0,cycleR=1,cycle=5)

28 FPS

More time used to resize down than decimate at full size. interesting!
Though 47 FPS if I MT() the resize but that is for another thread.

=================================


mpeg2source("D:\whatever.d2v",idct=5)
ChangeFPS(last,last,true)
Selecteven()

ResizedVideo=BilinearResize(320,180)
TDecimate(ResizedVideo,mode=0,cycleR=1,cycle=5,clip2=last)


3.1 FPS


I thought it would be around 28 FPS like the previous example.
Is this a bug? Did i do something wrong?



Thanks

[tritical]

Without looking into it in depth, I'd guess it's a cache issue. Try:

mpeg2source("D:\whatever.d2v",idct=5)
ChangeFPS(last,last,true)
Selecteven()

ResizedVideo=BilinearResize(320,180)
TDecimate(ResizedVideo,mode=0,cycleR=1,cycle=5,clip2=last.blur(1.0,0))

Or replace blur(1.0,0) with some other fast filter.

[Zep]

Originally Posted by tritical

Without looking into it in depth, I'd guess it's a cache issue. Try:

mpeg2source("D:\whatever.d2v",idct=5)
ChangeFPS(last,last,true)
Selecteven()

ResizedVideo=BilinearResize(320,180)
TDecimate(ResizedVideo,mode=0,cycleR=1,cycle=5,clip2=last.blur(1.0,0))

Or replace blur(1.0,0) with some other fast filter.

I tried your suggestion and a few other filters and no improvement


thanks

[Zep]

ok just now i tried

TDecimate(ResizedVideo,mode=0,cycleR=1,cycle=5,clip2=ChangeFPS(last,last,true))

and FPS is at 9 FPS


that is the only filter I have passed thus far that has helped.
Since that filter grabs 10 frames and caches them it appears it is
as you said a cache issue.

[mastrboy]

Any chance we will see a updated version of TFM with support for .dgi files (DGDecNV) similar to the support already implemented for d2v files?

[Lyle_JP]

Originally Posted by mastrboy

Any chance we will see a updated version of TFM with support for .dgi files (DGDecNV) similar to the support already implemented for d2v files?

I have no trouble using TFM with DGDecodeNV as my source filter. You just need to set field order manually.

[mastrboy]

Not meaning as a source filter, off course that works. TFM has the parameter called d2v="filename", which aids in the IVTC process, currently it only seems to support the format from DVD2Avi and DGIndex d2v files.

From the manual:

d2v -

        This option is intended to be used if you are using an mpeg2source() with a d2v file.
        It sets the name and path to a d2v file, which TFM will analyze to see if there are any
        illegal field order changes and optionally set the order parameter using the field
        order of the d2v file.  If the d2v file is found to have illegal field order transitions,
        TFM will create a fixed d2v file with the string "-FIXED" attached to the end of the file
        name.  The new file will be located in the same directory as the original.  You can then
        use this fixed d2v file for processing.  If the order parameter is set to "-1" then TFM
        will detect the field order from the d2v file and set the order parameter to match. Depending
        on the value of the "flags" parameter, TFM will also use the d2v info for field matching
        and will pass info from the d2v on to tdecimate to help aid duplicate detection and hybrid
        detection.

            *NOTE:  This option currently supports all d2v formats that I am aware of... which
                    include:  dvd2avi 1.76, 1.77.3 and its variants, all dvd2avidg versions, and
                    all dgindex versions.

            example =>  TFM(d2v="myd2v.d2v")

        Default:  ""  (String)

     flags -

        Controls how much of the info from the d2v file is used when the "d2v" parameter is set.
        Possible options:

           0 - Check the d2v file for illegal transitions and set the order parameter if it
                  is not already manually set.  Also, pass on rff flag duplicate info to
                  tdecimate.
           1 - Same as 0, plus use the trf flags for field matching in film sections (sections
                  where the trf flags follow the 012301... pattern)
           2 - Same as 1, but use the trf flags for field matching in all areas (doesn't have
                   to be in 0123 pattern)  (very much not recommended!)
           3 - Same as 0, but don't pass on any info to tdecimate (i.e. only set order and check
                   for illegal transitions)
           4 - Same as 1, but d2v matches are checked for being combed.  If a d2v match is detected
                   as combed then tfm uses its own matching routine for that frame.
           5 - Same as 4, but d2v matches are only checked for being combed around scenechanges.

           VERY IMPORTANT (MUST READ):  For options 0, 1, 2, 4, and 5 to work correctly, tfm/tdecimate
                                        must either immediately follow the mpeg2source() command or
                                        any filters inbetween them must not modify the order or number
                                        of fields in the stream in any way. Or use a trimIn file to
                                        tell TFM what frames have been discarded.

        Default:  4  (int)

[Stereodude]

I just wanted to post a thank you here for tritical for the great work on TIVC (tfm / tdecimate). The ability to use one clip as the basis for another's decimation with TDecimate ( clip2= ) really helped out my workflow on a few clips I was working on getting back to p24.

Thanks!

[manolito]

I.E. optimum quality vs. speed solution

Just want to find out if this line (posted by tritical a few years ago) is still state of the art:

yadifmod(edeint=tdeint(mthreshL=0,mthreshC=0))

A simple TomsMoComp or Yadif has too many quality issues, and all those ultra high quality (and ultra slow) deinterlacing scripts are just unusable for my rather slow machine. I always liked the quality and the speed I got from the above command. Is there anything new out there which surpasses it in terms of speed vs. quality?


Cheers
manolito

[tritical]

A simple TomsMoComp or Yadif has too many quality issues, and all those ultra high quality (and ultra slow) deinterlacing scripts are just unusable for my rather slow machine. I always liked the quality and the speed I got from the above command. Is there anything new out there which surpasses it in terms of speed vs. quality?

My recommendation would be QTGMC with the slowest preset that meets your speed requirements. I don't think there is much question that QTGMC is the way to go for deinterlacing in Avisynth nowadays. Getting it setup can seem difficult, but Vit has made it very easy if you read the first post. If you can't find a preset that betters your current script in speed/quality report back .