TDeint and TIVTC

[Leak]

all of it except the last 2 episodes. don't know why it went that way (deadlines in production?).
What do you mean? That the last two episodes were 30p? If so, I didn't really notice it... gotta check my R1 DVDs once I'm home...

np: Deadbeat - Let It Rain (Wild Life Documentaries)

[tritical]

[link removed], changes:
TFM:
+ Added v11 d2v format support
+ Field matching improvements (improved the code that decided which metrics to
use for picking the final match)
+ Added micout parameter

TDecimate:
+ Added fast isse routine for 50/50 blending case
- Fixed a bug in hybrid=3 operation that caused it to handle certain scenechange
situations incorrectly.

Added FrameDiff filter and CFrameDiff conditional function

Took a little longer than I thought it would to get this version out, but finally got it done. And a thank you to MaXi_TK96 for providing test clips where field matching failed.

[TheBashar]

Oops, I should have searched this forum first, but I posted a probelm I've been having with TDecimate over in the avs usage forum: http://forum.doom9.org/showthread.php?t=99414

Basically, for the last couple releases I've been getting unhandled exceptions from TDecimate in mode=5 when vfrDec=0. When in mode=4, this doesn't happen, and mode=5 with vfrDec=1 also doesn't cause a problem.

Would it be beneficial to give a truckload of other specifics or does that narrow it down enough?

[tritical]

Gonna need a little more info cause I can't manage to duplicate the problem here. Could you post the following:

full avisynth scripts used
TIVTC version (please test with v0.9.10.0)
avisynth version

It would also help if you could send me the tfm/tdecimate output files... you can upload them here:

68.119.245.113
port 17252
upload/upload

Finally, when do the exceptions occur? During script loading or does it actually manage to process a few frames?

[TheBashar]

AviSynth: 2.5.5
TIVTC: 0.9.10.0

ep.avs
LoadPlugin("C:\Multimedia\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Multimedia\AviSynth 2.5\plugins\TIVTC.dll")
LoadPlugin("C:\Multimedia\AviSynth 2.5\plugins\TDeInt.dll")
LoadPlugin("C:\Multimedia\AviSynth 2.5\plugins\DeDup.dll")
LoadPlugin("C:\Multimedia\AviSynth 2.5\plugins\fft3dfilter.dll")
LoadPlugin("C:\Multimedia\AviSynth 2.5\plugins\TTempSmooth.dll")
LoadPlugin("C:\Multimedia\AviSynth 2.5\plugins\WarpSharp.dll")

function FirstPass(clip c)
{
orig = c
c = tfm(c,d2v="ep.d2v",mode=6,slow=2,PP=6,cthresh=8,chroma=true,MI=20,output="ep.tfm")
c = TDecimate(c,mode=4,hybrid=2,vfrDec=0,output="ep.dec")
return c
}

function SecondPass(clip c)
{
orig = c
c = tfm(c,d2v="ep.d2v",mode=6,slow=2,PP=6,cthresh=8,chroma=true,MI=20,input="ep.tfm")
c = TDecimate(c,mode=5,hybrid=2,vfrDec=0,tcfv1=false,input="ep.dec",tfmIn="ep.tfm",mkvOut="ep.dtc")
c = DupMC(c,log="ep.dup")
return c
}

function ThirdPass(clip c, int begin, int end)
{
orig = c
c = tfm(c,d2v="ep.d2v",mode=6,slow=2,PP=6,cthresh=8,chroma=true,MI=20,input="ep.tfm")
c = TDecimate(c,mode=5,hybrid=2,vfrDec=1,tcfv1=false,input="ep.dec",tfmIn="ep.tfm",mkvOut="ep.dtc2")
c = DeDup(c,threshold=0.3,threshold2=1.0,range2=2,trigger2=2.5,maxcopies=20,maxdrops=20,log="ep.dup",timesin="ep.dtc",times="ep.tmc")
c = Trim(c, begin, end)
c = FFT3DFilter(c,sigma=2.5,plane=0,ow=24,oh=24,kratio=1.2)
c = FFT3DFilter(c,sigma=2.5,plane=1,ow=24,oh=24,kratio=1.2)
c = FFT3DFilter(c,sigma=2.5,plane=2,ow=24,oh=24,kratio=1.2)
c = TTempSmoothF(c,maxr=7,scthresh=7.1)
c = Crop(c, 0, 0, -2, -2)
c = LanczosResize(c, 880,480)
c = XSharpen(c, 15)
return c
}

clip = MPEG2source("ep.d2v")

start1 = 92
end1 = 5476
end2 = 7063
end3 = 61874

start2 = end1 + 1
start3 = end2 + 1

ep-a.avs
Import("ep.avs")
clip = FirstPass(clip)
return clip

ep-b.avs
Import("ep.avs")
clip = SecondPass(clip)
return clip

I open ep-a.avs in VDM and play through to the end. Then I close it and open ep-b.avs. Before any frame displays, AVS catches the exception.

It would also help if you could send me the tfm/tdecimate output files... you can upload them here:

Sorry, I couldn't seem to get the transfer to work. You can download the files from here: http://s33.yousendit.com/d.aspx?id=2N2B43EG7YC261R5KUA5V4LQP6

Anything else I can provide?

[tritical]

When you say "catches the exception" does vdub crash/terminate or does it give you an error/exception message like "unrecognized exception" or a read or write error?

[TheBashar]

It pops up the "unrecognized exception" error box.

[Chainmax]

tritical, in a couple of days I might be able to have a problem sample for you. It's a sample capture which only manual TMPG IVTC has been able to handle as all other methods (Decomb, TIVTC, SmartDecimate) make lines alternate between noticeably aliased and normal.

[tritical]

@TheBashar
I was finally able to track down the problem. One of the routines that is called when vfrdec=0 wasn't checking the bounds of a loop that alters an array and it would occasionally cause a little heap corruption. That would then occasionally cause an access violation during a malloc() call later on. I was only able to trigger it using the output files you posted.

@Chainmax
Any samples are welcome.

[TheBashar]

I was finally able to track down the problem. One of the routines that is called when vfrdec=0 wasn't checking the bounds of a loop that alters an array and it would occasionally cause a little heap corruption. That would then occasionally cause an access violation during a malloc() call later on. I was only able to trigger it using the output files you posted.

Great! I'm glad to hear you were able to find the problem. Thanks for running it down!

[TheBashar]

I ran into another problem. Is this the best place to brin it up or should I start a separate thread?

I have been testing using TDeint to decomb frames that come out combed from TFM. Oddly, I've found a scene cut that is combed but TFM doesn't think it's combed. TDeint has no trouble detecting it's combed, but I'd prefer to use TFM hinting so I wonder why TFM doesn't see it.

orig = clip
deint = tfm(c,d2v="ep.d2v",mode=6,slow=2,PP=1,cthresh=6,MI=64,output="epd.tfm")
deint = TDeInt(deint,type=2,clip2=orig,full=false,debug=true)
return deint

I believe with the parameters I've passed to TFM, it should be using the exact same combind detection as TDeint defaults to using. Unfortunately, I can see in DebugView that TFM hints not combed. If I change TDeint to hints=false, it detects and correctly deinterlaces the scene cut.

Any ideas?

[tritical]

Most likely, the original frame (this is what tdeint would see when using clip2) does create a value greater than 64, but the frame that comes out of field matching by tfm, while still combed, doesn't create one as big. You could see for sure by using "debug=true" and "micout=2" in tfm. That would show you what match is delivered by tfm and the MIC values (what is compared against 64) for each of the 5 possible matches. In the next version of tivtc I am adding an option to consider mic values during field matching that should make the handling of scenechanges that need to have a u or b match used much better, but it will also slow things down (it will be like using micout=2 is now). I am also planning on using the mic values in the field matching process in a more complicated manner than what will be in the next version, but that wont be added for a little while due to lack of time.

Finally, bringing up problems with tdeint or tivtc in this thread would be easiest for me in terms of making sure I see it.

[TheBashar]

Thanks for the explanation tritical, but I think there's some other problem here. I turned on the tfm debugging like you suggested. Below is the debug output from tfm and tdeint (ignoring hints). I don't understand how it all works, but I think some field match something is short-cutting the combed decision.


TFM: frame 320 - final match = p (D2V) MIC = N/A (OVR)
TFM: mode = 6 field = 1 order = 1 d2vfilm = F
TFM: CLEAN FRAME
TDeint: frame 320: accumP = 67248461 accumN = 117154
TDeint: frame 320: field = bottom order = ttf
TDeint: frame 320: mthreshL = 6 mthreshC = 6 type = 2


I should have noted earlier, TFM was still not detecting combing even with MI=20. And that frame is heavily combed.

[Leak]

TFM: frame 320 - final match = p (D2V) MIC = N/A (OVR)
There you go - you're using a D2V file, and the flags in it are b0rked. Try setting flags=1 or flags=0 in TFM and see if that fixes it.

[TheBashar]

There you go - you're using a D2V file, and the flags in it are b0rked. Try setting flags=1 or flags=0 in TFM and see if that fixes it.

By default it's already using flags=1 which does a superset of flags=0. But, just to be sure, I tried flags=0 anyway. There was no change in the results. In both cases TFM reports (via debug) "no errors found in d2v." Any other ideas?

[tritical]

Leak is right, the debug output you posted shows that it is doing the matching off the flags for that frame and isn't doing any analyzing of its own. With flags=0 or flags=3 it shouldn't do any matching off the flags.

By default it's already using flags=1 which does a superset of flags=0. But, just to be sure, I tried flags=0 anyway. There was no change in the results. In both cases TFM reports (via debug) "no errors found in d2v." Any other ideas?
The "no errors found in d2v" message simply means that there are no illegal field order transitions in the d2v file, it can't tell you anything about whether or not the flags are reliable to match from. Could you run tfm with "debug=true" and the default value for flags, and then run it with "debug=true,flags=0" and post the two logs somewhere so that I could look at them? Just run it for the first 400 frames or so.

[TheBashar]

Looks like I need to put my dumbass hat back on and apologize to Leak. I really did try it with flags=0, ahh... I thought. But now, flags=0 shows correct output with no combing. I dunno if I forgot to save the AVS after I made the change or what. Damn. Sorry Leak.

In case it still helps, I've attached the logs you asked for tritical. In the file name hint0/hint1 signifies the true/false setting of hints for TDeint. The problem frame I've been testing with is 320 in the logs.

I don't really understand what this implies. Does this mean that DGIndex is outputting incorrect flag information or does it mean the wrong flag information has been encoded into the source?

[tritical]

[link removed], changes:
TFM:
+ Added micmatching parameter
+ micout now works with display output
- stricter checking for 0123 sections with flags=1
- fixed a few debug output format inconsistencies

TDecimate:
- Fixed vfrDec=0 crash due to heap corruption

I don't really understand what this implies. Does this mean that DGIndex is outputting incorrect flag information or does it mean the wrong flag information has been encoded into the source?
It doesn't have anything to do with dgindex, it simply means that the trf flags in the mpeg2 stream can't be used for field matching purposes... at least for that frame. What that really means is that the encoded frames in the mpeg2 stream are not progressive, because if they were the flags could be used to tell which fields are part of the same frame (i.e. what you need to know for field matching).

Out of curiousity, could you try TIVTC v0.9.10.1 with flags=1 and see if that frame is still combed? Also, see how much of it is reported as film from the d2v? One problem was that when tfm checked for the 0123 pattern it only checked if the current flag was part of a pattern, but that could lead to problems. For example if you had:

123012230123

There is a pattern break with the two consecutive 2s, but both are part of a 0123 pattern if you allow for only looking completely ahead or completely behind (the first would be "3012" and the second would be "2301"). tfm was considering that type of pattern to be all ok to use for matching. I've now changed it so that it requires the current flag to be part of a 0123 pattern that extends on both sides, so for the above example it would do its own matching for frames that included any fields from the "1223" part.

[TheBashar]

Out of curiousity, could you try TIVTC v0.9.10.1 with flags=1 and see if that frame is still combed?

Using the new build, my "320" frame is not combed anymore. 320 and many of the frames before it are no longer marked with the (D2V) flag.

Unfortunately, some of the other scenechange interlaces still exist. Frame 1307 used to be just like 320, but it has not been fixed by the new build. It's debug output shows it is still using the (D2V) flags. :(

Interestingly, with the new micmatching turned on:
320 - match c: Detected As NOT Combed! (29 <= 64)
320 - micmatching override: c to u
320 - final match = u MIC = 29
320 - mics: p = 256 c = 29 n = 29 b = 256 u = 0

But, micmatching does not save frame 1307. It appears that micmatching is not used when TFM is taking the field flag information for that frame from the d2v. In order to be helpful with needed u/b matches around scenechanges, I would think you'd need it to be always on, not just when out of pattern.

Also, see how much of it is reported as film from the d2v?

In the last relese of IVTC, it reported 93.8% film. This release (with stricter pattern matching) reports 89.7% film.

Is there anything in the way of logs that would help in tracking down what's happening at 1307?

Thanks for all your help!

[TheBashar]

+ Added micmatching parameter

I forgot to mention one thing. I know that micmatching incurs a performance hit and from the help file you only override the field match when an mic is "significantly lower" (lets say by at least X amount) than the current match.

A quick shortcut would be not to compute the extra mic values when the current match has an mic of <= X. I have lots of frames with regular match mic values of 0 or 1 and it seems a shame to calculate all the other mic matches anyway.

Cheers!

[tritical]

Is there anything in the way of logs that would help in tracking down what's happening at 1307?
If you could post the d2v file and a list of which frames show up combed when matched from the d2v flags with flags=1 that would help the most.

It appears that micmatching is not used when TFM is taking the field flag information for that frame from the d2v. In order to be helpful with needed u/b matches around scenechanges, I would think you'd need it to be always on, not just when out of pattern.
It then somewhat defeats the purpose of matching off the flags... which is to avoid needing to compare matches and check if frames are combed, but I might consider changing it in the future. There hasn't really been enough testing with flags=1 on sources that have film percentages in the 10 to 90 range (I don't have any atm) to determine exactly how everything should work.

[TheBashar]

If you could post the d2v file and a list of which frames show up combed when matched from the d2v flags with flags=1 that would help the most.
Sure thing! The following 1.3MB linked 7z archive contains:

ep.d2v - Self Explanatory
ept.avs - Contains FirstPass() function used for output
test.log - 33MB log of avs processing
scenex.txt - List of scene change interlaces


http://nerdswap.com/files/9ae51aaa8b3e602101767adbfbbbd519thebashar.7z

There are many frames with interlacing, but the scenex.txt file lists only the frames with blatant scene change whole frame interlacing. I verified these by hand with the ept.avs script after commenting out the tdeint call. I included the debug log in case it would be informative. I used it (comparing tdeint to tfm) to locate the interlacing frames to be verified.


It then somewhat defeats the purpose of matching off the flags... which is to avoid needing to compare matches and check if frames are combed, but I might consider changing it in the future.
That makes sense. Unfortunately there are some frames that have been encoded with with some interlacing/combing in them. I would really like some way to detect those frames so that they can be subject to TFM's decombing options.

I humbly suggest changing the micmatching option to:

[EDIT: Removed suggestion due to better idea in next post.]

[TheBashar]

Tritical,

I was thinking some more about my suggestion that I posted last night, and I think there's a better suggestion to be had. My previous suggestion had the problem that it mixed two concepts (trusting d2v flags & micmatching method) in one parameter.

I would like to humbly suggest adding an additional parameter to d2v flag controls and modifying the micmatching. What would be most beneficial for me would be something like:

d2vsuggest (bool):

false - [default] When d2vsuggest is false, flags taken from the d2v according to the "flags" parameter are implictly trusted as long as they are in-pattern.
true - When d2vsuggest is true, flags taken from the d2v according to the "flags" parameter are treated as suggestions even if they are in-pattern.

Explanation of "treated as suggestions": There are several matching modes (6 right now) in TFM. When a d2v file would be provided, and the flags used, and they are in-pattern, and d2vsuggest=true, the field match suggested by the d2v flags would be moved from its place in the selected mode's matching sequence to front of the line. For example, in mode=3 with a d2v suggested field match of "n", the match sequence would become (n + p/c + u/b). It should be noted that with d2vsuggest=true, the current match would always checked for combing.

With that concept broken off, my suggestion of micmatching simplifies to just whether or not the MIC has to rise above some small threshold X or the MI. The small threshold that I am referring to is what you refer to as "significantly lower" in the current micmatching description (45 I think). So it would become something like:

micmatching (int):

0 - [default] micmatching is not used.
1 - Frames that are checked for combing and have MIC > X are subjected to micmatching. MIC are calculated for all other matches and match is overriden if new match MIC < old match MIC - X.
2 - Frames that are checked for combing and have MIC > MI are subjected to micmatching. MIC are calculated for all other matches and match is overriden if new match MIC < old match MIC - X.

Personally, for the sources I'm presently working with d2vsuggest=true and micmatching=2 would offer significant benefits. While they do add a speed penalty, it comes in the form of only checking each frame for combing in the majority of cases. Only a minorty of matches would be subjected to the full micmatching every match possible combing check.

Does any of this sound valuable to you? If so, I'd be willing to take a crack at the source code. I'm not beneath trying to help code solutions that would help me.

[tritical]

When I click that link it brings up a page that says "you have clicked on a stale link".

Adding an option like d2vsuggest would be possible.

With that concept broken off, my suggestion of micmatching simplifies to just whether or not the MIC has to rise above some small threshold X or the MI. The small threshold that I am referring to is what you refer to as "significantly lower" in the current micmatching description (45 I think). So it would become something like:

micmatching (int):
0 - [default] micmatching is not used.
1 - Frames that are checked for combing and have MIC > X are subjected to micmatching. MIC are calculated for all other matches and match is overriden if new match MIC < old match MIC - X.
2 - Frames that are checked for combing and have MIC > MI are subjected to micmatching. MIC are calculated for all other matches and match is overriden if new match MIC < old match MIC
The significantly lower threshold is a relative percentage threshold and not exclusively a hard threshold. Currently, it is 1/3 of the next lowest match, but to prevent weird things with low values (like 1 vs 5) there is a hard threshold minimum of 15. When I first added it I hijacked micout for micmatching so it calculated all of them regardless. I went ahead and separated the two so that when micmatching=true it only calculates the other mics when the final match has a value > 15, most frames don't produce a value that large. Adding an option to only do it when the final match value > MI is possible as well though.

[TheBashar]

When I click that link it brings up a page that says "you have clicked on a stale link".

Damn, sorry about that. Try this:
http://nerdswap.com/files/9ae51aaa8b3e602101767adbfbbbd519thebashar.7z

Thanks!

[TheBashar]

I went ahead and separated the two so that when micmatching=true it only calculates the other mics when the final match has a value > 15, most frames don't produce a value that large. Adding an option to only do it when the final match value > MI is possible as well though.

I guess it's pretty arbitray, but I think the last released version had a threshold of 45 not 15. The absolute difference has to be 15 and it has to be 3 times less... so combined I think that means it should be value > 45.

[tritical]

The 15 was the minimum value between the lowest match and the next lowest match, the 1/3 was also between the lowest match and the next lowest match. That is if you had 5 21 30 25 25 it would override because abs(5-21)>15 and 5*3 < 21, but if you had 0 5 5 5 5 or 10 28 28 28 28 it wouldn't.

[TheBashar]

Oops, I multiplied by 3 on the wrong side. Thanks for clearing that up.

Sorry to be such a PITA with all the d2v issues. I'm just spoiled. I really like the speed up from the d2v info but I really like the quality I can get from using tfm's motion adaptive masking in conjunction with tdeint (PP=6 with clip2).

[tritical]

Its not a pain, any suggestions are welcome I just don't always have the time to implement/test things right away.

Anyways, I looked at the d2v and scenex.txt files and, unfortunately, all the frames that are listed are right in the middle of long 3:2 pattern runs and also have the progressive frame flag set on top of that. Is that stream off a dvd or is it some type of capture? I've seen a similar situation before on an hdtv capture, just wanted to know where yours came from.

Any way that I would be able to get part of it? Say the first 5000 to 10000 frames? If so, you could just upload it to the ftp address I posted before:

ftp://upload:upload@68.119.245.113:17252/

[TheBashar]

Anyways, I looked at the d2v and scenex.txt files and, unfortunately, all the frames that are listed are right in the middle of long 3:2 pattern runs and also have the progressive frame flag set on top of that. Is that stream off a dvd or is it some type of capture?

That's what I was afraid of. I thought perhaps the problem was some of the interlaced scenechanges were encoded into a progressive frame. I think if the d2v says its interlaced (from telecine) it's probably trustable, but it would have no idea if a regular frame would be interlaced from the source.

It's a DVD. I'm trying to get my partial SG-1 collection cleaned up (some have tons of grain) and made available to the HTPC box. From what I gather, these TV show DVD collections are usually pretty challenging. There's the telecine, then (as seems to be the case here) there's some natively interlaced stuff, there's pattern changes in the CGI sections, different patterns in slow motion sections (surprising how often), and often less than ideal cuts.


Any way that I would be able to get part of it? Say the first 5000 to 10000 frames?

I suppose a small portion of demuxed m2v would be a reasonable fair use for analysis purposes.

Thanks for all your help!

[levi]

I really like tivtc. Use it on occassions where decomb fails. I haven't used it in a while, tho. Right now I'm trying to figure out which dgmpgdec version I should use with TIVTCv09101.zip. Keep getting d2v 'unsupported format' errors. Thanks for the excellent tool & keeping it up to date!

[tritical]

v0.9.10.1 supports every dgindex project file type ever used up to v11 (plus dvd2avi 1.76, dvd2avi 1.77, and dvd2avidg) which is the latest unless it was changed again in 1.4.4. What version are you getting the unsupported format error with?

[levi]

hmm

ok, i tried several versions. I'm running on 64bit (intel) Windows. Could that cause a problem?

[tritical]

It could be, but I've never used a 64bit version of windows and don't know much of anything about the issues related to running 32bit programs with it.

I tested tfm with the d2v you posted (it is v11) and it worked fine here. The debug output from tfm shows:

TFM: v0.9.9.4 by tritical
TFM: successfully opened specified d2v file.
TFM: newest style (dgindex 1.2+) d2v detected.
TFM: auto detected field order from d2v is TFF.
TFM: no errors found in d2v.
TFM: d2vflags = 131242 out_of_pattern = 38595 (70.6% FILM)

[Guest]

Yeah, but did you notice his AVS script refers to a different D2V file???

[foxyshadis]

Is tdec mode=5 reading of tfm output file broken? Or am I just giving something the wrong inputs?

1st:
x=x.tfm(mode=0,ovr="sisters.tfmovr.txt",output="sisters.tfm.txt")
x=x.tdecimate(mode=4,output="sisters.tdec.txt")
return x.crop(0,0,4,4)

2nd:
x=x.tfm(ovr="sisters.tfmovr.txt",input="sisters.tfm.txt")
x=x.tdecimate(mode=5,hybrid=2,vfrDec=0,input="sisters.tdec.txt",tfmin="sisters.tfm.txt",mkvout="sisters.mkvtmp.txt")

2nd pass load always generates an exception "TDecimate: tfmIn file error (invalid specifier)!" I looked in the source and it's coming from TDecimate.c line 2576, apparently expecting a + or - after every framenum and match. I inserted a - on each line and it loaded like a charm. Otherwise they're unedited.

sisters.tfm.ovr is just "0,11853 c" since it's not telecined or combed, a few parts of the video just have dupe frames inserted every 5th and it's crazy jerky.

[tritical]

Yep, it is broken for the case when there aren't +/- markings (combed/not-combed) in the tfm output file because I forgot about the new line feed character. However, the only way to generate a tfm output file that doesn't have +/- markings is by setting PP=0, and I don't see that in your script. Anyways, it will be fixed in the next release.

[foxyshadis]

Sorry, I guess I shouldn't have edited the calls at all, I assumed they were equivalent. My bad.

[tritical]

[link removed]. This version fixes the tdecimate error when reading in tfmIn files w/o +/- markings, micmatching=true only calculates the extra mic values when it is possible the match could be overridden, and adds a flags=4 option. The flags=4 option is equivalent to flags=1, except that d2v matches are checked for being combed, and if a d2v match is detected as combed tfm uses its own matching instead.

[TheBashar]

Marvelous! Thank you. I can't wait to give flags=4 a try!

[TheBashar]

Thanks so much tritical! The new release works wonders on my problem clips! Flags=4 really takes care of my problems. And I'm not sure I mentioned it before, but TFM's motion adaptive masking with (clip2=tdeint) tdeint decombing is really great! Because of the motion masking I'm more comfortable reducing the combing threshold to clean up even more of those combed trouble spots. Thanks so much for the great tools!

[foxyshadis]

It works! Loaded it for the other video and it worked fine.

I hate being annoying, but I'm wondering if M-in-N vfr is to be supported in the near future? I'm going to have to put a couple new clips aside until then, it has crazy 4-in-7, 3-in-8, 5-in-8, 17-in-26, and more sections. If I had to specify cycle/cycleR in an override file, that would be fine, I know it's a corner case. I'd try right now by iterating the filters, except it doesn't have an mkvin parameter, and I guess it might be messy to support that.

[tritical]

I hate being annoying, but I'm wondering if M-in-N vfr is to be supported in the near future? I'm going to have to put a couple new clips aside until then, it has crazy 4-in-7, 3-in-8, 5-in-8, 17-in-26, and more sections. If I had to specify cycle/cycleR in an override file, that would be fine, I know it's a corner case. I'd try right now by iterating the filters, except it doesn't have an mkvin parameter, and I guess it might be messy to support that.
Could you expand on what the source is (framerate, origin) and what 4-in-7, 3-in-8 are? Are you talking dedup type functionality or actual detection of differing framerate sections? If it is the first, I don't plan to add that into tdecimate since it already exists. If it is the second then all of the framerates present must be integer multiples of some base framerate.

M-in-N vfr for tdecimate that worked in the same way as mode 3/5 now would probably not be useful for any sources, at least I have never seen any... it would go something like this: say you have cycle=5 and cycleR=2. TDecimate would try to decimate 2 duplicates from each cycle and the framerate for the remaining frames from that cycle would be set to 3/5*input_fps. However, if it detects only 1 or 0 duplicates then it would only remove those many and the framerate for the remaining frames from that cycle would be set to 4/5*input_fps for the 1 case or 5/5*input_fps for the 0 case. At this point though, it wouldn't be hard to add that functionality to tdecimate (for hybrid=2) so I'll go ahead and do it in the next release.

On another note, I am planning in the next version to add the capability to allow the use of trim() before tfm() when using the d2v option with flags=0,1,2,4. To allow for multiple trim() calls, it will probably be made so that tfm can read an input file that lists all the cut frame ranges or all the included frame ranges. If anyone has other ideas please speak up.

[foxyshadis]

I was just hoping to get it working cleanly for 24 fps animation with odd initial framerates, like 10 fps (causing the 3,3,2 pattern). Repositioning the frames' times, ie adjusting framerates, not just dropping like dedup.

To be honest, I'm just a perfectionist, I guess; maybe it wouldn't be that useful after all. It'd be (marginally) smoother, and maybe give slightly better telecines to video.

If you do change it, I'll play with it and see if my theory pans out or not.

[tritical]

[link removed]. Only reason for this release is to fix a dumb little bug with the micmatching parameter not being initialized correctly (was missing a '_' in tfm's constructor). It would usually result in it being on when it shouldn't be causing slower than normal operation. Haven't had any free time to work on new stuff.

[MOmonster]

First of all, I think there is a small mistake in your tfm readme. Shouldn´t n and u be changed (I mean p and n and on the other site b and u have the same parity, so also the same direction)? Maybe ones more there is something I don´t understand right, but you will tell me ;)
But to come to my request for tfm. What about a new mode?
Here (http://forum.doom9.org/showthread.php?t=100137) ones more there is a problem fieldblended source. It shows a simple pattern: c d b c b c d b c b... (c-clear,d-double,b-blend).
Maybe you allready get my request. I think about a mode similar to mode=0, but that will change the field what has to be matched. So if the last match we get was a c-match, tfm will try to match the second field of the next frames, so look for a c- or a n-match. If there is no match, tfm would use the second fields for deinterlacing. If there is a c-match, the next frames tfm would still take the second field, but if there is a n-match, tfm would try to match the first field with p- or c-matches and would also use the first field for deinterlacing, if there is no match. Also here p-match means no change (keep working on the first field) and c- match means working on the second field for the next frames.
This would help for many blendconversations and also for some strange pattern mixes.
I just offer this request because it could be useful for many sources and shouldn´t be so much work to implement it (compared to my last request :o ).

Edit: For this mode it is of course necessary not just to have a look at the last choosen match, but on the last possible match.

[tritical]

First of all, I think there is a small mistake in your tfm readme. Shouldn´t n and u be changed (I mean p and n and on the other site b and u have the same parity, so also the same direction)? Maybe ones more there is something I don´t understand right, but you will tell me There probably is, you're talking about the wording somewhere or about the diagrams down towards the bottom under the overrides section?


On the other thing, let me see if I got this all correct (assuming top field first clip). Start out matching from top w/ c/p (field=1,order=1), when a c match comes up then on the next frame start matching from bottom with c/n (field=0,order=1), keep matching from bottom until an n match comes up, then on the next frame switch back to matching from top w/ c/p and then just keep repeating that process?

That would work for the pattern you describe, but it seems like it would 1.) only work in areas with continuous motion and a constant pattern and 2.) the user would have to manually set whether or not to start matching on the top or bottom field depending on when the first blend comes.

[MOmonster]

Yes, this mistake I see, was in the override section (line 560 and follows).

And yes, you get it correct. Of course, this won´t work on the first pattern and also not for Edits and also not for every blendconversion, but for many cases it would work and the problems of edits and so on would be only one pattern length. It just use the principle that if we have a match, we now that both fields are clear and the next not matching field than has to be the blend, so tfm would avoid this fields until the next matches. It would be just a quick easy solution against the most blends.

Edit:
So your second point is right, but your first not that correct. If we have no motion we have no blends and the first motion we get will be the blend after the last possible match. So it will also work for no-motion scenes (strange match-jumping, but working fine) and scenechanges. And the other point, the patternlength is also not that important for this principle, only edits and the first frames (if we don´t set them manual) would make problems, but this parts aren´t that big. Perfectionist should still use restore24, but for the others this could be a nice solution. ;)
(The wrong decission, because of edits we could override)

[tritical]

So your second point is right, but your first not that correct. If we have no motion we have no blends and the first motion we get will be the blend after the last possible match. So it will also work for no-motion scenes (strange match-jumping, but working fine) and scenechanges. I think I am missing what you mean by "last possible match".

[MOmonster]

What the last possible match mean.
I mean the last match of a static scene, before there is motion. The first different field is a blend. For the most blendconversions the fields after a match are the blends. The same fields (botton or top) are the blends of the next frames till we have other matches.
That´s why, if we work for example on the top field, also if we have a better matching p-match, if there is a c-match for the next frame we look for the botton fields. Here it is the same, also if the c.match is the better match, if there is a n-match we change the field. We just try to avoid the field after the "last" detected match (c for top or n for botton). This work for many blendconversions. The next not matching field after a matching field is mostly a blend.

Edit: This evening I created a function, that realize this idea. See here (http://forum.doom9.org/showthread.php?p=713709#post713709) . And yes, it works like it should and catch the most blends. But of course it would be better to realize this in a Filter, because of speed, compatibility and stability reasons.