TDeint and TIVTC

[Guest]

@tritical

OK, first time I'm using Tdeint()... :)

You wrote:

That means TDeint could use EEDI2 in the following way (assuming clip is TFF):

interp = separatefields().selecteven().EEDI2(field=1)
tdeint(edeint=interp) I wanted to try this adaptive use of eedi2(). I had a BFF wedding DV clip. So I did this first:

interp = separatefields().selectodd().EEDI2(field=0)
return interp

The result is nice:

http://neuron2.net/misc/no-tdeint.jpg

Then I tried this:

interp = separatefields().selectodd().EEDI2(field=0)
tdeint(order=0,edeint=interp)

But the result is not so nice (note aliased diagonals):

http://neuron2.net/misc/tdeint.jpg

Can you set me straight, please? Thank you.

[tritical]

I stated things incorrectly and oversimplified into two scenarios when there are actually four same framerate deinterlacing cases that need specific handling. The configurations should be:

1.) bff clip, keep bottom field, interpolate top field:

interp = separatefields().selecteven().EEDI2(field=0)
tdeint(order=0,field=0,edeint=interp)

2.) bff clip, keep top field, interpolate bottom field:

interp = separatefields().selectodd().EEDI2(field=1)
tdeint(order=0,field=1,edeint=interp)

3.) tff clip, keep bottom field, interpolate top field:

interp = separatefields().selectodd().EEDI2(field=0)
tdeint(order=1,field=0,edeint=interp)

4.) tff clip, keep top field, interpolate bottom field:

interp = separatefields().selecteven().EEDI2(field=1)
tdeint(order=1,field=1,edeint=interp)


The settings you tried should be changed to either case 1 (just change selectodd to selecteven) or case 2.

There are also the configs for bobbing, but atm the edeint option is broken for bobbed output so I'll post those later when it is working.

[Guest]

Thank you for the clarification. I tried case 1. But the result is not as satisfying as I hoped. Please compare the sharp diagonals to the pure eedi2() picture in my previous post (especially over Ben's right shoulder). Is there anything I can do, perhaps lower the threshold?

EDIT: Even with thresholds at 0, it's not improved. Apparently it is the alternating of passed through content with interp'ed content, as can be seen via the map option, that causes this. I think we need to determine the combed areas and then use the interp'ed data on all scan lines within that area. What do you think?

case 1:

http://neuron2.net/misc/wedding.jpg

[tritical]

@neuron2
Any way that I could get that part of the clip for testing? Also, how fast is the background moving in that sequence? If you set the motion thresholds to -1 do those areas look as they do in the interp clip?

I think we need to determine the combed areas and then use the interp'ed data on all scan lines within that area. What do you think?
Expanding the motion map that way would probably help in this case, but I can't say for sure without seeing which pixels it is weaving and why.

@All
I was thinking last night about making an edeint type option for the motion map that would allow TDeint to take a motion map from another filter. I've realized that I don't want to reimplement all of the options and features of TDeint in TDeint2 (and the code in TDeint is so big and ugly that if I did I would rewrite it all) when the only real changes would be EEDI2 type interpolation, which can already be used in TDeint, and the new motion mapping. So instead, I am going to add an "emap" parameter to TDeint. Luckily, I already have my new motion mapping implemented in another filter :). The downside to doing everything in separate filters is it is slightly slower than if it was integrated and it makes it more complicated to use. On the plus side, allowing everything to be done externally opens up more possibilities and makes it easier to implement new ideas.

[Guest]

Thanks, tritical. I'll upload the clip when I get home tonight. Right now, I'm busy finding out how crappy all the H.264 tools are. How about this: a "Pro" transport multiplexer that won't let you set the PIDs or the transport mux rate (heck, it won't even tell you what rate it ends up with, so you have no way to know what rate to inject it at!). :mad:

[DarkNite]

I wouldn't worry about integration until you had both the time and desire to do so. Things aren't so complicated for those of us who know the lost art of "how to rtfm".

[Guest]

Any way that I could get that part of the clip for testing? Here it is:

http://neuron2.net/misc/short.zip

It's BFF DV. I've been using Cedocida. The snapshots are from frame 0.

Thank you for taking a look at this. I still think the motion areas need to use all the scan lines. I mean, how could that be wrong, since it would be copying whole areas of the good full interp'ed clip, which we know looks fine? I concede that it may not be necessary, but let's see if you find a weaving issue.

If you set the motion thresholds to -1 do those areas look as they do in the interp clip? No, it's fully combed! That suggests that your case 1 is set up wrong, or there is a weaving bug.

[Guest]

OK, I solved it. You need the field parameter on Tdeint() for case 1!

interp = separatefields().selecteven().EEDI2(field=0)
tdeint(order=0,field=0,edeint=interp)

It makes a very sweet adaptive deinterlacer. This results in the best deinterlacing I have ever seen under Avisynth. It blows away LeakKernelDeint(). And it's tolerably fast on my 64FX-55. Bravo, tritical!

Would it be possible for you to update your four cases above with the right field settings for Tdeint(), as this will serve as a reference for use of this outstanding method?

Sweet (who needs Faroudja?):

http://neuron2.net/misc/sweet.jpg

[Guest]

Interestingly, you can get comparable results with my VirtualDub filter. My code is a lot more naive so will produce more artifacts. But it still looks pretty good. Here are the settings:

http://neuron2.net/misc/sd.jpg

[tritical]

Thanks, I'll update the settings... it is quite funny that I can't keep the parameters right for my own plugins. I was assuming field got set to the same value as order by default (field=-1) in TDeint since that is how it works in TIVTC, but that is not true (it uses field=1 by default)... I should read the readme sometime :p. Now I am wondering how many times I have made that mistake when giving settings before.

[Guest]

Don't worry -- a foolish consistency is a hobgoblin of little minds.

[manolito]

It makes a very sweet adaptive deinterlacer. This results in the best deinterlacing I have ever seen under Avisynth. It blows away LeakKernelDeint(). And it's tolerably fast on my 64FX-55. Bravo, tritical!

I totally agree. Thanks very much tritical !

Just one (probably stupid) question:
Is there a general rule of thumb which field should be kept and which one should be interpolated? It would be nice to be able to narrow the four different cases down to just two.

Cheers
manolito

[tritical]

There really isn't any rule. I have heard some people claim that graphics and logos usually tend to be top field aligned.. in which case keeping the top field and interpolating the bottom field would give better results. Usually, it is simply a matter of whatever looks best to you and most of the time it is hard to tell any difference. I personally usually keep the top field for tff material and keep the bottom field for bff material out of habit.

[foxyshadis]

Tritical, is there any way to get TDecimate to create 120 fps files? (I doubt it, but I thought I'd ask.) I can import a timecode file and fill in frames based on the frame rate, so far that's looking like the simplest method. Unless someone can point me to something better?

(I'm trying to finish updating avisynth.org's vfr page with a 120 fps method that doesn't suck horribly. Even cooler would be something to insert drop frames into an avi/mp4 based on a timecode file, but I doubt that exists.)

[tritical]

Tritical, is there any way to get TDecimate to create 120 fps files? TDecimate could output 120fps, but then it would be up to the codec to insert "n-vops" or drop frames for the identical frames. If any processing was done later in the script it could make it such that the identical frames would no longer be identical leading to the codec not inserting drop frames.

The method for encoding 120fps currently described on the variable framerate page of avisynth.org does it such that the output from the avisynth script is a 29.970 fps file similar to what you get out of tdecimate(mode=5). A separate program (AVI60.exe) then goes through and modifies the final encoded file by inserting drop frames as needed. That process would be the same as having a separate program that goes through and inserts n-vops based on the timecode file output by tdecimate.

Even cooler would be something to insert drop frames into an avi/mp4 based on a timecode file, but I doubt that exists. If the source for AVI60.exe was available somewhere I could probably easily modify it to do this for avi, but I don't think that it is. Otherwise, I would have to spend a while looking into the file structures (which I have no clue about) in order to figure out how to do it and what needs to be modified. If there is someplace that has this information easily available I would be interested.

I can import a timecode file and fill in frames based on the frame rate, so far that's looking like the simplest method. Unless someone can point me to something better?
How are you doing this?

As far as 120fps encodes go I am not a proponent, but currently it is pretty much the only way to deal with 24/30 mixes without using mkv and without resorting to blending. I am hoping that mp4box will eventually get the ability to create vfr mp4 files based off a timecode file in the same fashion as mkvmerge.

[foxyshadis]

I use mkv was well, but I'm updating the page to help the people who need or want to use the different methods. Some of them were abominations, and this is the only one I haven't found a good substitute for. A method that requires Avisynth 2.0 and a variety of ancient plugins is just evil.

How are you doing this?
Right now I'm not; my thought was to parse a timecode file given by tdecimate via script or plugin to repeat frames, but as you say, that still depends on the grace of the encoder. :(

I found some information searching around, though; virtualdub can now write drop frames to raise the framerate (or lower it back), so I'm going to dig through the source and ask phaeron how it works. (If you want to see, set it to direct stream copy and change the framerate to, say, 119.88, 4 of every 5 frames will be drops hardly adding any size to the avi.) Of course it's always supported drop frames for captures, but this enables a nice quick comparison.

Edit: It looks as if it's simply a matter of creating zero-length frames, which are frame header "30 30 64 63" and then a 4-byte lenth "00 00 00 00". The index is composed of a collection of: the same frame header, a 4-byte keyframe flag, a 4-byte index (LSB), and a 4-byte length (LSB); drop frames simply enter zero for length. Oh, and a chunk in the beginning with fractional framerate and framecount, 16 bytes from the 4CC start. Otherwise the files are almost byte-identical, there's no groupings of frames like newer containers. I will investigate mp4 methods however.

[Guest]

@tritical

In EEDI2(), I'm confused by the field parameter. The readme says that field=0 should copy through the bottom field and interpolate the top. But this script shows no difference between a and b. If I change to field=1, a difference can be seen. This suggests to me that the field settings are 0=top and 1=bottom. Am I going astray?

a=assumebff.separatefields().selecteven() # bot fields only
b=a.eedi2(field=0).assumebff().separatefields().selectodd()
subtract(a,b) # should compare original bot to interp'ed top

[tritical]

@foxyshadis
Thanks for the info. Now you've got me really interested in making a tool to do this. I'll take a look at a converted file from vdub.

@neuron2
I can't run the script you posted without adding assumeframebased() before the second separatefields call (that was changed between 2.55 and 2.56 i think). Using the following script:

a=assumebff.separatefields().selecteven() # bot fields only
b=a.eedi2(field=0).assumeframebased().assumebff().separatefields().selectodd()
subtract(a,b) # should compare original bot to interp'ed top

definitely shows a difference here. While the following scripts:

a=assumebff.separatefields().selecteven()
b=a.eedi2(field=0).assumeframebased().assumebff().separatefields().selecteven()
subtract(a,b)

a=assumebff.separatefields().selecteven()
b=a.eedi2(field=1).assumeframebased().assumebff().separatefields().selectodd()
subtract(a,b)

show no difference as expected. Not sure what is happening to cause problems with your script.

[tritical]

@foxyshadis
Alright, so I've almost got a working program to do the drop frame insertion for avi. It can parse all the avi header info, modify framerate/count, parse all the frame chunks and insert drop frames, and parse and modify the index. To make it less complicated to code there are currently a few restrictions: the avi can only have one stream and it must be a video stream ("00dc"), the output must be 119.88 fps, the timecode file must be v1 and can only have framerates of 17.982, 23.976, 29.970, 39.960, 59.940, and 119.88 fps, and it only supports avi files with the "idx1" index type and not opendml avi's that have the "indx"+"ix##" index. However, given the relatively specialized use of such a program I don't think that those restrictions should be much of a problem.

[Guest]

I can't run the script you posted without adding assumeframebased() before the second separatefields call (that was changed between 2.55 and 2.56 i think). Adding that fixed my problem. I'm not sure why. According to the documentation, SeparateFields() should have no effect on frame-based clips, but it clearly does. I won't agonise about it since 2.56 apparently prevents one from going astray this way. Until I upgrade, I'll just adopt the policy of always doing AssumeFrameBased() before SeparateFields().

[tritical]

Finally got this thing working. [link removed]. Syntax is:

see here (http://forum.doom9.org/showthread.php?p=759253#post759253)

It will create a new 119.88fps file at "c:\path\avifile-120.avi". All restrictions mentioned above apply. So now it is possible to do a normal vfr encode with tfm/tdecimate(mode=5) with v1 timecode file output... encode the output from tdecimate to avi and then run this program on it to get a 120fps avi file. I'll give it a better name, make a readme, put the source with it, and release a full version sometime later this week. I've also fixed the edeint mode=1 problem with tdecimate and have gotten the external motion mask stuff working so there will be new version of TDeint soon too. Now it's time to sleep :p.

[foxyshadis]

Whoa! Cool! You had a busy sunday. ;)

Since it's meant to be used right out of avisynth rather than on pre-existing avis, that certainly makes sense. Not every tool can be a virtualdub. =p I'll wait for your final copy before I update the page.

[tritical]

Since it's meant to be used right out of avisynth rather than on pre-existing avis, that certainly makes sense. Not every tool can be a virtualdub. =p :confused:


Anyways, I would advise against using the version I linked above for now as it had some bugs in modifying the offsets for the index entries. I've also gotten quite a bit more done since then and expanded the functionality... I've added a PAL mode, that creates a 50.000fps avi file, that goes along with the current 119.880 fps NTSC mode. The possible framerates in the timecode file are now:

NTSC -- 119.880/n for n=1 to 25 (step=1)
PAL -- 50.000/n for n=1 to 10 (step=1)

It now accepts v2 timecode files as well as v1. It's also working in conjunction with mkv2vfr off haali's site, so you can take a vfr mkv file, extract the avi and timecode file using mkv2vfr, and then use this program to go straight to a 120fps avi file (assuming the framerates can be represented). The extra possible framerates also make it so you can use it in conjunction with Dedup (either with or without TDecimate) as long as the maxdrops is no greater than 5 and starting fps was 25, 50, 23.976, 29.970, etc...

Once a final version of this program is done I think I will also make one to go from 120/50fps avi to a timecode file and avi without drop frames (though I think a program to do this already exists :confused: ).

[Marsu42]

Your tfm filter from TIVTC v1.0 Beta 4 is missing mode=7, it throws an error pointing out that there's only up to 6. Beta 2 (fortunately, I it still exists unlinked on your site) is ok and has got this wonderful new mode e.g. for reversal of crappy PAL->NTSC transfers w/ decimation cycle 6. If this is not an error, you should update the documentation or people might wonder... on the other hand, nobody noticed since now :->

[tritical]

It is definitely a bug if it is telling you that. Could you post the exact script you are using to get that error and the exact error message? I can't reproduce the problem here (tfm(mode=7) works fine) and none of the code changes between beta 2 and beta 4 had anything even remotely to do with that functionality. Also could you use tfm(display=true) and make note of the version number for tfm that it shows? :thanks:

[Marsu42]

Damn (i.e. somehow embarrassing to me, good for you) - I cannot reproduce the problem either and I have upgraded avisynth and the script since then. While I am quite sure that I got this error message, it might be that avisynth somehow loaded an outdated dll w/o mode 7... I'll try to be more thorough next time :-o

And on the more productive side, I've got a question: Does it help TFM to process frames and/or speed up the process noticeably if any black borders are cropped before, or shouldn't I do it because it could break field matching? In other words, is crop().tfm().resize() faster or is tfm().crop().resize() safer?

[tritical]

As far as speed is concerned it depends on how much you are cropping and what the values are. If you are only doing something like (4,0,-4,0) then it would probably be faster to crop after tfm cause the 4 off the left side will throw off the alignment which makes a difference for simd routines. However, if your crop values for the sides are multiples of 8 then that wont be a problem, or you could use crop(,,,,align=true) which would also keep alignment for the rest of the script (but will cost a few bitblt's). You'd have to be cropping quite a bit off the frame for it to make a noticeable difference in tfm's processing time I think.

For how it effects field matching, cropping off black borders can't hurt, but how much it would help depends on how much noise the borders have and how big they are. tfm's field matching routine already ignores the first 8 pixels on the left/right sides and the first 2 lines off the top and bottom because it usually tends to be junk. And if the borders have no temporal noise then they wont be having any effect in the field matching anyway.

[Mug Funky]

i find cropping off hard subs is a big help :)

[manono]

Hi-

I was creating a Lagarith uncompressed AVI before feeding it to CCE. VDubMod got almost to the end before aborting with the error:

AviSynth read error: TDecimate: internal error during prebuffering (131727, 131728, 20)!

And the AVI came up 60 frames short of being complete. I had done a similar encode earlier with a larger Cycle, and it had come up 100 frames short. Here's the script:

LoadPlugin("F:\DivX Stuff\GKnot\DGDecode.dll")
LoadPlugin("F:\DivX Stuff\GKnot\RemoveGrain.dll")
LoadPlugin("F:\DivX Stuff\GKnot\TIVTC.dll")
LoadPlugin("F:\DivX Stuff\GKnot\Undot.dll")
MPEG2Source("K:\Harold Lloyd Comedy Collection-Vol1-Disc1\1-1\1-1.d2v")
Trim(550,0)
TFM()
TDecimate(Mode=0,CycleR=10,Cycle=38,Chroma=False)
Undot()
RemoveGrain()

Using TIVTC v1.0 Beta 3 for AviSynth 2.5.x

Also, when using the Display parameter, it doesn't show all the frames in the Cycle before starting fresh again, so I can't see the information for most of the frames on the VDubMod screen. Did that make any sense? So I'm unable to see all the frames in the cycle to see what was kept and what was removed, and why. I don't really feel like running DebugView. I don't guess there's any workaround is there, when the cycles are large like that?

[tritical]

Try beta 4 and see if it works ok. Beta 4 fixed the problem prebuffering was having processing the last two cycles of a clip. That's why beta 3 only managed to last one day.

Yeah, atm the display info for large cycles doesn't really work cause any text that would be written off the frame (either vertically or horizontally) is just skipped. I'll try to change that in the next release and have it wrap around to multiple columns or something.

[manono]

Darn, and here I thought I was up-to-date. Thanks tritical.

[tritical]

[link removed], changes:
+ Added emask parameter
- Fixed edeint not working correctly with mode=1
- Changed field=-1 operation when hints=false
This version adds the motion mask from external clip ability and fixes the edeint problem with mode=1. I also decided to change the default behavoir for field=-1 so that it operates in the same way as tfm by setting field equal to order (except if hints=true, then field is still set to 0). This will hopefully be the last version before 1.0 final.

The configs for using eedi2 with tdeint's mode=1 are:

1.) tff

interp = separatefields().eedi2(field=3)
tdeint(mode=1,order=1,edeint=interp)

2.) bff

interp = separatefields().eedi2(field=2)
tdeint(mode=1,order=0,edeint=interp)

[tritical]

[link removed], changes:TDecimate:
+ wrap display=true text to multiple columns and mark frames that will be
dropped with '**'
+ round to nanosecond accuracy for all v1/v2 timecode output files
+ slightly more efficient metric cycle calculation I have been working on adding internal caching to TIVTC... mainly for tdecimate because avisynth's internal cache doesn't work very well with TDecimate's access pattern. It isn't a problem for the typical cycle=5, cycleR=1 setup, but for large (>10) cycle sizes with cycleR > 1 it can be quite noticeable. I would probably recommend doing a metrics output pass in mode 4 first, and then using mode=0 with the metrics input file when using cycle sizes > 10 or 15.

Anyways, since the caching requires a good bit of internal tinkering I decided to go ahead and release a version with only the above changes first. Aside from the internal caching, the only things still to do are to port optimizations from tfm/tdecimate into fielddiff/framediff... though I may skip that since those functions are not used much.

[tritical]

Here is [See this post: http://forum.doom9.org/showthread.php?p=759253#post759253]. This is the full incarnation of aviparse.exe which I put up last week, a readme is included. The new syntax is:

tc2cfr out_fps input_avi_file timecode_file output_avi_file

out_fps specifies the framerate of the destination file, whose name/path are specified by the output_avi_file parameter. The timecode file can have any framerate corresponding to the following equation:

framerate = out_fps/n for n=1 to a large number step=1

The maximum value for n depends on what out_fps is, but for an out_fps of 119.880120 the max value for n is 340 at which point it can no longer adaquetly differentiate between fps values. The program will tell you what the maximum value is for the specified out_fps value when it starts. In the end I've made this about as general a tool as possible as it can handle just about any combination of framerates. It also allows one to go from a vfr mkv to a 120fps avi file simply by using haali's mkv2vfr tool (for v1 tc) or mkvextract (for v2 tc) to get an avi+tc file and the using those as input into tc2cfr. Or you can use tdecimate in a timecode file output mode and instead of going to mkv go to an avi file.

[foxyshadis]

Here is tc2cfr v1.0 (http://bengal.missouri.edu/~kes25c/tc2cfr.zip).
Awesome. I'm going to start playing with this now. Thank you so much, tritical!

Does the "no audio" rule still apply?

[tritical]

It does. More specifically, the avi can only have one stream in it and it must be a video stream. Also, it must use an "idx1" index (i.e. not open dml). The reason is I just didn't want to deal with the added complexity of handling multiple streams, and I can't think of why someone would use this on an avi with an audio stream already in it since such a file would be desynced. It seems to me that any avi you would be using tc2cfr on would be originally created with a single video stream in it.

[tritical]

[link removed], changes:TDecimate:
+ internal caching for modes 0, 1, and 3
+ show cycleR value in display output
- Fixed a problem with display output wrap around for the drop frame #'s listing
- Fixed metric prebuffering computing metrics for the incorrect frames within
a cycle after a seek to a random frame, causing a minor slow down when
seeking.
- Fixed requesting frames an extra time in modes 0, 1, and 3 when no field
matching hints from tfm were present in the stream.
The internal caching was actually much easier to add then I originally thought. It makes quite a difference for some cases, but not for others... your mileage may very. Found and fixed a couple other small bugs while testing it out as well.

EDIT:
Alright, I'm just gonna act as if Beta 6 was never released because it sucks horribly :devil:. The internal caching helped for rare scenerios, but it cut the speed by up to 4-5x for the typical cycle=5/cycleR=1 usage. The caching is needlessly doing a ton of bitblts when it doesn't have too (which is what is slowing things down). I'll put up a new version to fix the problems tommorrow.

[tritical]

[link removed]. Only change was fixing the slowness problem of tdecimate's internal caching. It is now at least as fast as it previously was for all cases, and much faster for those rare cases. That should be the last new version for at least a few days :p.

[Malow]

hi folks, i have one question: is bad to use tivtc in the "hybrid mode" for mkv vfr (using the examples avs in tivtc package) if the movie is pure film or pure ntsc interlaced video? (the speed is no problem..)

i really would like a "all purpose" script, for almost all kind of movie.. (not including anime)

or im asking too much? :)

[tritical]

If it's pure interlaced then you would definitely be better off processing it with a true deinterlacer. If it's pure film then it should be fine to use mkv vfr mode. I personally use the vfr mode for all material that isn't pure interlaced.

TIVTC's hybrid detection is pretty accurate/robust and by default it will favor film detection since vidDetect=3 requires no duplicates via matches and all differences above vidThresh. The major place it will fail is not detecting really slow 30fps pans due to the frame differences being below vidThresh, though usually those pans are so slow that it isn't noticeable. That problem can be corrected by lowering vidThresh or simply using vidDetect=0 (only require no match duplicates), but that can lead to some false detections on anime and cartoons. The reason being that when there are repeated frames there will be no difference between possible matches and you can get things like "ccccc" instead of the usual "cccpp" pattern out of tfm. That scenerio cannot occur in a scene with continous motion, such as a panning scene, but in a static scene or a scene where there is motion only every 2, 3, or 4 frames it could happen leading to such a scene being detected as video. A false detection of such a scene as video is usually unnoticeable though (it would only cause a negative effect for scenes that have motion every 2, 3, 4, etc.. frames, not on a completely static scene). The problem with vidDetect=0 shouldn't occur with real life material where there aren't repeated frames. Also, if you use the d2v option then the rff flags will be taken into account when detecting film/video sections.

[Malow]

wow, tnks for explanation and for your great work!!!! :thanks:

[tritical]

I put up a new version (v1.2) of tc2cfr on my website, it was mainly just internal changes and code cleanup. I also made a utility called cfr2tc that does the opposite operation... it takes an avi file with null frames and outputs a file with null frames removed and a v1 or v2 timecode file. cfr2tc can handle avi files containing multiple streams (2 audio streams + 1 video stream for example), but only one of the streams can be a video stream. The avi file it creates will contain only the video stream from the input file with null frames removed. It also has the ability to do timecode output only without creating a new avi file. Syntax for cfr2tc is:

cfr2tc input_avi_file output_avi_file timecode_file mode

Set mode to 1 for v1 timecode file ouput and to 2 for v2 timecode file output. Set it to 3/4 for v1 or v2 timecode output only. There are already a few programs to do this type of operation, but I decided to make this just to have something to do. I might make a little gui for them as well, haven't decided yet.

Links:

See this post: http://forum.doom9.org/showthread.php?p=759253#post759253.

[Egh]

I also made a utility called cfr2tc that does the opposite operation... it takes an avi file with null frames and outputs a file with null frames removed and a v1 or v2 timecode file. cfr2tc can handle avi files containing multiple streams (2 audio streams + 1 video stream for example), but only one of the streams can be a video stream. The avi file it creates will contain only the video stream from the input file with null frames removed. It also has the ability to do timecode output only without creating a new avi file.


Well, at least someone finally outside japan realized the convinience of such conversion :) Though more than one year after...

Good thing it's in one tool, not in two (aka avidelnul.exe and avi2timecode.exe tools).

I personally use that kind of conversion for hdtv raws, it's convinient to have no-null-frames avi in order to be able to use AVISource() on it (cause other way is to mux into mkv, therefore no AVISource() then).

[tritical]

I ended up making a gui for cfr2tc/tc2cfr called tc-GUI. I hadn't coded anything in c# before and decided this would be a good time to give it a try... it definitely beats doing win32 api programming for gui development :). It does require the .NET framework (1.1) to run. cfr2tc and tc2cfr have also both been updated to v1.3, mainly cosmetic fixes, plus the ability to specify the output fps as a fraction in tc2cfr.

cfr2tc, tc2cfr, and tc-GUI are now merged into one zip package for download. Link: avi_tc_package (http://bengal.missouri.edu/~kes25c/avi_tc_package.zip)

[Revgen]

Note to all EEDI2 users:

If you're going to use a EEDI2/TDeint combo in any form on any film-based TFF material, make sure to use AssumeTFF() before you deinterlace. I was trying to use it as a postprocessor for TFM but it wouldn't deinterlace properly unless AssumeTFF() was enabled. The progressive frames may have confused it.

I've not yet experienced this problem with pure interlaced content though.

[MOmonster]

@tritical
I don´t if it is a bug or just different internal settings, but for this (http://forum.doom9.org/showpost.php?p=762834&postcount=36) case, the newest tfm seems not to work correctly. The older version works just fine. I could not find something in the changelist that will cause this problem, maybe you know the reason for. Can it be because of the changed sco default or because of some optimizations?

Edit: Sorry my mistake. Indeed it was the different sco value that cause the different behaviour.

[tritical]

[link removed], changes:
TFM:
+ Support d2v format version 13
+ Write MIC values into output file
+ A few new mmx/isse optimizations
+ sse2 optimizations (only used on P4)
- Fixed clip2 crash caused by writing to a referenced frame [thanks tsp]
- Fixed rare crash on d2v input parsing caused by sometimes writing one entry past
the end of an array

TDecimate:
- sse2 optimizations (only used on P4)


Writing the MIC values into the tfm output file is in preparation for some two pass stuff in v1.1 of TIVTC. Main ideas atm are identifying combed frames by looking at relative MIC differences between neighboring frames and for identifying problem frames by running field matching twice using field=0 one time and field=1 the other time. Any places where the matches don't correlate between the two outputs would be identified as problem areas. Not sure it will allow much automatic correction, but would be a way to automatically identify areas where manual overrides may be needed.

I have had a new version of EEDI2 ready for a while now, but haven't released it due to various bug reports that I haven't been able to reproduce or nail down. The new version has interpolation direction consistency post-processing and corner/junction detection post-processing. Mainly trying to identify problem areas and then drop back to linear interpolation. I'll probably release that in the next couple days.

Finally, there is still the external motion-masking that was gonna go into TDeint2 but which I decided simply to make available in TDeint via the emask parameter. I haven't really touched it for the last month or more, but it will be done eventually. I haven't given it much priority since scharfis_brain has a similar script function and I rarely need to process true interlaced material.

TDeint v1.0 final will be out as soon as I have time to write the new doc (I've been saying that for too long).

[foxyshadis]

Oh, thanks for reminding me. Minor feature request: Would it be possible in the future to make tfm optional in TDecimate mode 5 (would default to all "c")? If not, perhaps removing the check on too many frames? I only ask because I use it often on progressive sources, where tfm's not useful. It'd be nice to have one huge file for those tasks.

Now, if that's exactly what the batch=true option does, that'd be cool, I just don't want to break it messily what with the warning.

EEDI2 is liquid luv. ;D

[tritical]

Yep, I'll make the tfmIn file optional for mode 5 (set to all c matches if not given) in the next version. batch does roughly the same thing but still requires a file to be specified.

Currently, if batch=true, then when the metrics input array is allocated all values are initially set to 0 (instead of ULLONG_MAX which is what I use for not set) and the check for having real values for each entry is disabled. Any valid difference metric values that are present in the specified input file are still read in as usual. Also, when the tfm info array is allocated all values are initially set to 'p' (instead of the not set value) and the check for having real values for each entry is disabled. Any valid match info that is present in the specified tfmIn file is still read in as usual.

[puffpio]

First time using progressive content..time to try out TDecimate!

I think I'm using it right (output looks right) but just want to verify

I have a 60fps (not exactly) 720p HDTV stream
the frames repeat like 'aaabbaaabbaaabb' etc

So I apply TDecimate(cycleR=3)

and that should do it right? (remove 3 out of 5 frames)
it appears to work and gets me to 23.97 film framerate, but I just want to be sure