TDeint and TIVTC

[tritical]

After looking at your function it seems that what it should be doing isn't totally dependent on the matches alone but on if the final output frame is combed or not. For example... start matching on the top field, keep matching on the top field until you hit a frame with no good c/p match (i.e. both c/p have mic values > MI) and where the previous frame had a clean c match... if that occurs switch the field value (before deinterlacing of the current frame) and then start matching off the bottom field. Keep matching off the bottom until you hit a frame with no good match and where the previous frame had a clean n match... at that point switch the field value and repeat.

Now I would agree that that type of scheme would work for areas w/o continous motion.

[MOmonster]

This is exactly what I mean. I should learn to describe my ideas better. :D

[tritical]

@MOmonster
k, I will add a mode for that here soon. Also I read back through the overrides section of the tfm readme and didn't spot any mistakes. What exactly was it that needed to be corrected?

@All
[link removed], changes:
TFM:
+ Field matching improvements
+ micmatching changed from bool to int, added micmatching 2/3 options
- flags parameter now defaults to 4
- Fixed IsCombedTIVTC always resulting in an error being thrown (was
introduced when micout was added)
Adding hybrid=2 operation for when cycleR isn't equal to 1 is actually a lot more complicated then I thought it would be, so that might be a while. The option for trim() before tfm with the d2v option should be ready sometime soon though.

EDIT: Seems I made a mistake in the tfm docs. The micmatching 1/2 option descriptions are switched... the description for option 1 is actually describing option 2 and vice versa. Will fix it next time around.

[Didée]

tritical, I'm experiencing an annoying problem with TDecimate(mode=2) within Restore24. Too often, TDecimate is producing a noticeable jerk by putting out a dup frame where it definetly should not do so.

The pattern of frames produced by R24 in the meantime is very regular. Still TDecimate sometimes insists on putting out dups, even in scenes with obvious motion, or even panning.
An interesting point is, that *if* these dups occur (it's not always, often the output is fine), then usually they are roughly ~1000 frames apart from each other, sometimes ~500. (Hear the NTSC 1000/1001 bells ringing?)

SmartDecimate, which was planned to be dropped because of all the headaches and instabilities it gives with recent Avisynth versions :devil: , does always produce a result free of drops, in comparison.

My assumption is that TDecimate uses a slightly too narrow margin of "how far away" a frame may be allowed to be from the "theoretical" position to get chosen:

... 1 2 3 4 5 6 7 8 9 101112131415
... c c d d e e f f f g g h h i i

Say, TDecimate has chosen frames 1,3,5,7 ... next choose should be 10, but there seem to be circumstances where 10 is calculated to be just "out of reach", and TDecimate forcibly uses 9 instead, even if 7/8/9 are bit-identical.
Tried using mxndl=2 instead of 1, but this made things even worse (much worse).

I've no clue how you are calculating these things internally, so my question is: is it possible to slightly widen the search window for frames to use?

If you want to have the script and/or sources to test, just tell me.


Addendum: TDecimate seems to have problems at the very start of any source - the sequence of the first few frames is almost always jerky. Here again, SmartDecimate has much less problems.

[tritical]

Having the clip/scripts would help a lot since without it I can't really tell much. What results do you get if you set m2pa=true in the tdecimate line? Or what results do you get if you run it one time with tdecimate(mode=4,output="out.txt") and then use tdecimate(mode=2,input="out.txt")? Needless to say mode 2 doesn't get much attention, I haven't used it since the time it was added and it has only been tested with a single clip that scharfis sent.

What instabilities do you experience with smartdecimate? Considering the source is available and it isn't that large, I could probably track them down or port it to use the normal avisynth plugin interface instead of the c-interface.

[Didée]

Well, with m2pa=true the PC stalls for several minutes (feeding a 3000 frames source...), and then the output is hoplessly b0rked. This probably has to do with the conditional environment, since in this case the output contains almost exclusively blends.
Admittedly, the 2-pass mechanism I didn't even try. It would never be used in R24 ... in case you didn't note already: the public acceptance of 2-pass mechanisms for Avisynth scripts is rather poor ;)

Regarding SmartDecimate:
It may give all kinds of grief. Sometimes it's not loading at all, often it is impossible to use <refresh> in Vdub, and sometimes it just doesn't work or makes scripts (complex ones like R24) generally unstable.
It works fairly OK with AviSynth 2.55, or very early 2.56 versions (up to Dec.19,2004). With all AviSynth versions of 2005, it is practically unusable - at least in scripts that use the conditional environment. From the guts, I suspect these issues have to do with the way all the caching stuff is handled in Avisynth - that's where a lot of changes have been done this year.

However ... having a de-bugged or even ported version of SmartDecimate would be a nice option for sure. After all, the approach of SmartDecimate is unique: where all other solutions do field matching and decimating in two separate steps, SD is the only one that's doing it all in one step.

PM'ed you about the sources. Thanks. :)

[tritical]

Admittedly, the 2-pass mechanism I didn't even try. It would never be used in R24 ... in case you didn't note already: the public acceptance of 2-pass mechanisms for Avisynth scripts is rather poor Guess it is a good thing I am very accepting of it :p. Then again, every encode I do anymore is with mode 5, hybrid=2 in tdecimate which requires a previous pass so maybe I am just use to it.

However ... having a de-bugged or even ported version of SmartDecimate would be a nice option for sure. After all, the approach of SmartDecimate is unique: where all other solutions do field matching and decimating in two separate steps, SD is the only one that's doing it all in one step. You don't remember old ivtc 2.2 :D? It works that way, matching and decimation in one step, though smartdecimate is quite a bit more complex internally. Actually one of my very first avisynth filters was an ivtc filter that worked in that manner, back around avisynth 2.07 or so. What is now TIVTC started out back then as well, about 5 generations previous to what it is now, though none of that was ever released to more than a couple people. I wasn't even a registered member of this forum at that time.

[tritical]

[link removed], changes:
TFM:
+ Allow MI to be changed via overrides file (same as field/mode/order/etc...)
+ Added mode 7 matching routine (specifically for material with blended fields)
- changed default MI value to 80
- changed default cthresh value to 9
- changed default micmatching value to 1
- fixed description of micmatching in help file

TDecimate:
+ Added "noblend" parameter (disable 2 drop scenario blending when hybrid=0)
+ Added mode 7, an arbitrary framerate decimation mode (like mode 2), but based on some
ideas from smartdecimate
+ Added denoising parameter, enables spatial denoising of frames prior to calculating
difference values (good for noisy sources or sources with lots of dot-crawl)

+ ShowCombedTIVTC filter added, for easy visualization of how cthresh/mi/chroma/blockx/blocky
actually effect combed frame detection

@MOmonster
I didn't write much of anything about how mode 7 of tfm actually works. In the end I just made it work the same way as the script function you posted in the other thread as opposed to how I described it earlier in this thread. I tested it on the clip posted in the other thread and the result was the same as your function.

[MOmonster]

Wow, a nice changelist :thanks:
I will test the new tfm and the new tdecimate mode soon.

[Egh]

That's really good. If only TDeint was updated so often :)

Since i'm quite sure it has several things which need more tweakining/fixing.

[tritical]

Yeah, TDeint could be improved quite a bit. I actually have a lot of ideas stored away for improving static area detection, interpolation, and temporal stability, but it would really need to be a complete rewrite starting from scratch at this point. Also, I spend a lot more time on TIVTC because I actually use it for 99.9% of what I encode whereas I almost never deal with interlaced material. Therefore, I don't have as much motivation to work on TDeint. In fact, I don't think I've ever actually encoded a clip longer than a minute or two with TDeint :p.

[Didée]

Sorry for being sluggish again, tritical. Report will come.

For now, TDecimate(mode=7) works pretty good. At the finish line, it's stepping right into SmartDecimate's hoes ;)

But I think there's a bug in it:

Used within Restore24, converting 25->23.976, it works fine up to frame ~19000. (output frame. Sorce frame ~19800, bobbed frame ~39600). From there on, duplicates are created every second. It seems that some miscounting (or rounding error?) is happening.
I confirmed that both SmartDecimate and TDecimate(mode=2) work correctly with the same setup. The error can also be reproduced by loading the script, jumping directly to 19000, and playing from there (so it's not R24's blend replacement doing something wrong after a certain amount of frames). And with trim(19000,0) the formerly b0rked section comes out fine, but again the errors start at ~19000. So it's not the source, either.

[Egh]

Yeah, TDeint could be improved quite a bit. I actually have a lot of ideas stored away for improving static area detection, interpolation, and temporal stability, but it would really need to be a complete rewrite starting from scratch at this point. Also, I spend a lot more time on TIVTC because I actually use it for 99.9% of what I encode whereas I almost never deal with interlaced material. Therefore, I don't have as much motivation to work on TDeint. In fact, I don't think I've ever actually encoded a clip longer than a minute or two with TDeint :p.

Tritical:

Thanks for you job :) But the thing is that TDeint is needed when DVD producers created something new and strange again :) Like now i'm dealing with source where there're fadings added to 24fps animated source. Ofc, the fadings are 30fps. And applying of TDeint ofc removed combing... Creating *very* powerful noise. And it's so powerful that PixieDust doesn't work on that scene (creates blocks). Amongst casual filters you need something like hqdn3d(5) at the very least to reduce the noise (but the picture looks not good in terms in quality after that). Sangnom doesn't produce noise (although the picture quality is not good either).

Luckily in this case I used one of my pwn "sledgehammer" filtering presets and managed to remove that heavy noise without any serious detail loss. But it would be great if TDeint didn't make that noise in the first place.

Also, not so recently, while working on hellsing ultimate ova trailer, i noticed strange artefacts in one of the interlaced flash scenes. TDeint created big ugly grey spots in areas where there's nothing similar to that at all. Sangnom didn't have any artefacts there. If you need, I can dig that project up and show you the screenshots. (I didn't release my version cause in the end it was only marginally better than Fluffy's one. Btw he also noticed that artefacts and had to use a workaround with different deinterlacing for that scenes).

And while working on saikano ova (yes, i'm the one responsible for #PSNR release ^^), more precisely on the "Second Mission preview" (note that trailers, promos, previews and so on usually have lots of interlaced content, more than actual episode does, so deinterlacer is highly needed while encoding), in one scene i noticed strange thing -- despite quite heavy combing all other the picture several frames in a row, TDeint *ignored* that combing at all. Some advised me to tweak the settings for TDeint, but I just switched to sangnom for that preview (and sangnom picked that combing w/o any problems).

And note that I do think TDeint is better than sangnom in general, the quality of picture is better and more sharp (in the way I like it). But using TDeint still has some issues atm, you have to treat it carefully and look for possible artefacts.

Also, if possible, I'd like more explanation/recommendation on different modes of TDeint. Especially the difference between 1<->3 and Kernel modes.

[tritical]

@Didée
Any possibility I could get a sample? Cause I honestly can't think of anything that would cause that to happen... I will try testing a longer sequence when I get home and see if it behaves strangely.

@Egh
All your descriptions sound like typical motion adaptive artifacts to me, and since sangnom discards one whole field it would not show any of that type. If you set the motion thresholds in tdeint to -1 do the artifacts go away? or you could use AP=0,APType=0, which would eliminate those type of artifacts as well. Of course doing that means you're giving up motion adaptation. I have used TDeint to deinterlace some bad sections of anime episodes and often end up setting the motion thresholds to -1 because the temporal order of the fields is all screwed up (separatefields() is jerky with both assumetff() and assumebff()) which will make TDeint create some interesting effects that would fit your description:
TDeint created big ugly grey spots in areas where there's nothing similar to that at all.
Also, motion-detection in tdeint by default is 5 fields and only pixels from the same field are compared, but motion of the top and bottom neighbor pixels in the other field are considered. That 5 field window is behind/ahead/across the current field. So often you get parts where it clearly looks like an area is in motion to a person, but if you look at it on the single pixel level the way TDeint does then it appears stationary. That is what often leads to the type of artifacts you describe. Especially, flashing scenes will be a problem.

I've got to go now, but I'll try to explain the modes in more detail later tonight sometime.

[Didée]

tritical - no additional sample is needed. Just pad the sample's start by X frames with Loop() or BlankClip(). Load the script, jump to frame X, and play from there on.

Interestingly, now on the other PC the effect does not start around output frame 19000, but around 40000 instead. :confused:

Dunno what it is, but it's not R24 itself. When jumping far into the source directly after loading the script, then R24 is making a "cold" start - it uses absolutely no reference to the actual position in the stream. Whereas I suppose the decimator to use such a reference, in some way.

[tritical]

Could you describe some exact steps to follow to reproduce the problem using the clip you sent before? I tried appending 50000 and 100000 frames using blankclip to the clip before sending it into restore24 and saving the clip that gets fed into tdecimate inside of restore24 to a lossless file then opening that avi file back in avisynth and adding 50000 and 100000 frames to the front of it. None of those scenarios produced problems.

[Didée]

I just use
padding = 50000

src=mpeg2source("ENT_Marodeure_snip2.d2v")
src.trim(1,1).loop(int(padding/23.976*25.0)) + src
Restore24( blabla )
Results (http://home.arcor.de/dhanselmann/padding.rar) (updated 20:45 - had put the wrong snapshots in it).
This little padding trick reproduces exactly what I am getting in normal full-lenght encodings.

Could it somehow be related to the used Avisynth version? On my 2 PCs I am, still, using AviSynth 2.55 from Dec.19.2004 resp. Jul.29.2004.
(I stuck with those because having SmartDecimate working reliably was always a must.)


[edit] Update:

I tried with the Avisynth RCs from your website (avisynth 2.5.6 - 9.18.2005 / - 16):

- No change for TDecimate - mode=7 still produces dupes on high frame numbers

- Now, if I try to load SmartDecimate.dll, VirtualDub vanishes. (That's why I still sit on those old Avisynth versions.)

[tritical]

I'll try your script and see what happens.

Now, if I try to load SmartDecimate.dll, VirtualDub vanishes. (That's why I still sit on those old Avisynth versions.) If you use mine make sure you use avisynth_c_loadcplugin() after loading avisynth_c.dll. In my builds I made it so external plugins such as loadpluginex/avisynth_c cannot replace avisynth's internal loadplugin() and loadcplugin() functions, but you can still access the functions of those dll's using dllname_loadplugin(). Unfortunately, I didn't test it enough... it should have given you an error, but avisynth's internal loadcplugin() routine now assumes __stdcall and smartdecimate exports avisynth_c_plugin_init as _cdecl so it finds the function, but when it attempts to call it it crashes becauses of the mismatched calling conventions. Will have to fix that on the next build.

EDIT: I put up new builds to fix the above crash, there had been a few other memory leak fixes since the last builds anyways.

[Egh]

Tritical:

I've seen yet another crazy bug in TDeint :) It's crazy one, but true :)

So i have again real source, not some test, and in some places applied text is interlaced. So naturally one needs deinterlace for it.

Here TFM+yatta made P-match on that frame.

Here's source frame: http://xs49.xs.to/pics/05402/sourc1.png

Here's what fielddeinterlace does, not good, but at least does the job :) http://xs49.xs.to/pics/05402/sourc1_blended.png

Here's what TDeint does :P
http://xs49.xs.to/pics/05402/sourc1_tdeint_frame1.png
http://xs49.xs.to/pics/05402/sourc1_tdeint_frame2.png

Not one but *two* frames :P And this behaviour seems independent on the mode chosen in TDeint. Also that's not the single frame which causes that frame doubling, in all similar cases in both episodes the result was same.

[TheBashar]

Here's what TDeint does :P


What's your exact TDeint command call look like?

[Revgen]

What's your exact TDeint command call look like?

I'd like to know too.

Perhaps we can start using TDeint to remove logos! :)

[tritical]

Change the field that is interpolated if you want to keep the text on that frame because the writing on the source frame you posted is only in the bottom field, but the top field is the one being kept.

[Egh]

Change the field that is interpolated if you want to keep the text on that frame because the writing on the source frame you posted is only in the bottom field, but the top field is the one being kept.

Well, that's NOT a problem there. Two frames instead of one -- that's a problem. Meaning on those scrolling scenes that means broking smoothness of animation because of additional duplicate.

And it can't be decimated -- cause original animation is 23.976, it has duplicates to decimate already.

So do you always use top field only? How to change that only for some particular frames?

For others: the command is exactly standard thing which yatta uses.

YattaPreMatchClip = last
FieldHint(ovr="G:\1\himm01.pet\himm01.d2v.fh.txt")
TDeint(clip2=YattaPreMatchClip,hints=true,order=1,type=3,sharp=false)
That's copied from my upcoming HiMM R2 project, of course :) Since I had to use FieldDeinterlace for Ichigo Mashimaru. Retaining text on those frames is not particulary important, it's only one frame at a time. But getting additional duplicates is out of question for panning/scrolling scenes :P

[tritical]

Which field are you doing frame matching off of? You should set tdeint to keep the same field as you are matching from (if you are matching off top set field=1, if you are matching off bottom set field =0).. by default field is set to the same value as order. You can override the value of field for one frame or a range of frames using an overrides file. If you could upload the clip to my ftp I will take a look at it.

68.119.245.113:17252
upload/upload

[Egh]

Which field are you doing frame matching off of? You should set tdeint to keep the same field as you are matching from (if you are matching off top set field=1, if you are matching off bottom set field =0).. by default field is set to the same value as order. You can override the value of field for one frame or a range of frames using an overrides file. If you could upload the clip to my ftp I will take a look at it.


Thanks, i will cut some part from vob with those frames and upload it.

Remember that i do stuff from yatta. So if it's yatta's fault somehow, tell me and I will ask Myrsloik to fix/improve it. How to assign frame/range field override in yatta?

=Update=
Sending the file to ftp atm. I used TFM (older than current version) and yatta on it.

[tritical]

@Egh
Thanks, I'll take a look at it tommorrow.

@Didée
I finally figured out what was happening using the script you posted. It wasn't actually related to the frame number at all. I had modified the decision code to specially handle a common case for restore24 processed video, but didn't make it exclusive enough and it was being triggered for other cases. When that happened it selected the next frame instead of the current frame, and if the stream was in just the right place it would throw things off such that eventually it would end up in a situation trying to select between two frames that were both duplicates of the previously delievered frame. Anyways, I will put up a new version tommorrow, I've also expanded the debug/display output for mode 7 to include more info.

[Didée]

That's good news :)
Since, apart from this issue, mode 7 seems to play very nice ... didn't find anything else I could complain about ;)

[tritical]

@Egh
I looked at your clip real quick and it is fine as long as the deinterlacer is keeping the same field as is being matched from.. i.e. the following produces a smooth result:

mpeg2source("C:\test.d2v")
deinted = last.tdeint(order=1,type=3)
tfm(d2v="C:\test.d2v",micmatching=0,clip2=deinted)
tdecimate(mode=1)

in the above script both tfm/tdeint will use field=1 by default since order=1. It will also work if you set field=0 in both. Also note the micmatching=0, in the last version I made it default to 1, but it will need to be zero for this clip else it will use a u on the combed frames creating an extra duplicate. However, if you are using a version prior to v0.9.11.0 it defaults to 0 or isn't implemented so don't have to worry about it.

You could replace tdeint in the script above with any deinterlacer you like, just make sure it is keeping the correct field.

If I remember correctly yatta makes the field parameter selectable in the tfm configuration window. So just make sure it is set to the same value that TDeint is using in your script or vice versa.

[Egh]

@Egh
Also note the micmatching=0, in the last version I made it default to 1, but it will need to be zero for this clip else it will use a u on the combed frames creating an extra duplicate. However, if you are using a version prior to v0.9.11.0 it defaults to 0 or isn't implemented so don't have to worry about it.

You see, I used TFM version prior to 0.9.11.0 and i got those duplicates. So how to make sure in yatta that those duplicates would NOT appear? Should I always use micmatching=0 in ymc on precalculating metrics? Does disabling micmatching provide worse results? BTW, it could be possible that micmatching was on by default in *ymc* while importing that project.

Also, since I use yatta, TFM is not used in script at all. Only TDeint used (and FieldHints, of course). And really strange thing is that only TDeint was producing duplicates. So if you change TDeint into something else in the following script, duplicates are not produced [e.g. FieldDeinterlace(blend=true, dthreshold=0), which was eventually used in Ichigo project, see manhole-fansubs release of it]

the script:

dgdecode_Mpeg2Source("G:\1\shan\shana.d2v")
YattaPreMatchClip = last
FieldHint(ovr="G:\1\shan\shana.d2v.fh.txt")
TDeint(clip2=YattaPreMatchClip,hints=true,order=1,type=3,sharp=false)
It's different project though, since it's my current one and I use 9.11 TFM metrics in it.

[tritical]

Alright, I went ahead and downloaded the latest ymc/yatta and know what is happening.
You see, I used TFM version prior to 0.9.11.0 and i got those duplicates. Yeah... but you got them because TDeint wasn't keeping the same field as was being matched from, not because of micmatching.
Should I always use micmatching=0 in ymc on precalculating metrics? Does disabling micmatching provide worse results? BTW, it could be possible that micmatching was on by default in *ymc* while importing that project. The latest version of ymc has micmatching options 0 and 2 and not 1 because yatta doesn't support tfm's u/b matches. Setting micmatching to 2 wont change anything in your clip because 2 can only change the matches to matches that could be used within the current matching mode. Using micmatching should improve things vs not using it, but micmatching=2 wont usually do much at all especially if the matching mode is limited to 0, 1, 2, or 4. All versions of tfm prior to v0.9.11.0, which you said you used one, either didn't have micmatching at all or it defaulted to 0 so there is nothing to worry about. Obviously, older ymc/yatta versions that don't set the micmatching parameter wont work with v0.9.11.0+ of tivtc because micmatching defaults to 1.

Also, since I use yatta, TFM is not used in script at all. Only TDeint used (and FieldHints, of course). And really strange thing is that only TDeint was producing duplicates. So if you change TDeint into something else in the following script, duplicates are not produced [e.g. FieldDeinterlace(blend=true, dthreshold=0), which was eventually used in Ichigo project, see manhole-fansubs release of it] The problem is the mismatch between what field tfm is matching from and what field TDeint is keeping. After looking into fieldhint, and remembering one of the TDeint's workarounds, I know the reason. It is because fieldhint uses the same hinting as decomb... when TDeint sees decomb hints and field is set to -1 (Auto), instead of setting field equal to order, TDeint sets field to 0 because decomb matches off the bottom field. Seeing those hints, TDeint thinks telecide came before it and that it should keep the bottom field. However, fieldhint assembles the matches according to how tfm matches the frames (assuming you used tfm in ymc)... and what field tfm matches from depends on how "field" is set in ymc. If field is set to "Auto" (-1) then it will use the same value as order... BFF=0 TFF=1.

So, what you need to do is in the TDeint() line of your script, add "field=1" w/o the quotes if your tfm configuration in ymc matches any of the following:

1.) order = TFF and field = AUTO
2.) order = TFF and field = 1
3.) order = BFF and field = 1

I am guessing you had field = AUTO in ymc, so if you set "field=1" in the TDeint line of your script the duplicates should go away.

[tritical]

[link removed], changes:

TDecimate:
+ Expanded mode 7 display/debug output
- Fixed a bug in mode 7's frame decision code

Didée, when you get a chance could you test it out and make sure it fixes your problem? It should, but there is always the possibility I screwed something up :D.

[Egh]

Good :) Now i finally understand this scheme :)

So it was essentially fieldhint+TDeint combo's fault. Is it possible to implement some kind of workaround in future versions of TDeint? Also, i now somehow suspect that some other artefacts I had with TDeint could be caused by FieldHint... Is it possible?

[Didée]

Your changes to TDec's mode 7 improved on the problem, but didn't solve it. Formerly, the dupes appeared up from a certain frame number & then were produced until the end. Now the effect still starts up from any certain frame number, but from there on there are ranges with regular dupes, alternating with clean ranges.

I've no clue what the point is. If there's anything you'd like me to test or try, tell me.

[tritical]

Send me a clip that fails to test with, or if you can't upload the clip then run with debug=true in tdecimate and send the log file plus a list of the frames that are duplicates. Gonna need one of those two things cause I have nothing to go on to try and identify the problem otherwise.

[Egh]

Tritical:

Could you please explain me in detail the differences between different TDeint modes? Or at least give a link to such explanation.

[Revgen]

There is a text file included with TDeint .zip file that explains the options.

[tritical]

[link removed], only change is d2v format v12 support in tfm.

@Egh
Like Revgen said the help file should explain everything pretty well. The possible mode settings should be pretty clear... there is only same frame rate output, double rate output, plus the resizing modes. If it is another option that you want a more indepth explanation just say what it is. I never did post an explanation about the different modes because there isn't really anything else to say other than what is in the help file.

[tritical]

[link removed], changes:
TFM:
+ micmatching can now be limited to scenechanges only, via the "sco" parameter
+ slow=1/2 matching mode optimizations (~10% speed increase)
+ display output now shows when scenechanges are detected when sco > 0
- changed sco default to 12.0
A version 1.0 should be along shortly, only actual feature I have left to add is the input file for trim() before tfm support w/ d2v input. I also need to get all the docs switched over to html files. Even though I don't have anymore planned atm, I am still gonna make updates/changes to tivtc as I get new ideas for improvements or bugs need to be fixed, etc...

When I release the 1.0 version of TIVTC, I'm also gonna release the final v1.0 for TDeint as well. I haven't worked on it for a long time and don't plan to do anything more with it. Since no bugs have been reported in the last release I assume it is good to go.

For the past week or so I've been working on a new motion-adaptive deinterlacer that's gonna become TDeint2. It addresses the major problems with TDeint: poor static area detection, temporal/spatial stability, and noise resilience. It will also contain a new edge directed interpolation mode that surpasses TDeint's current type=1/3 by a lot. I am hoping to have a first release out in a couple weeks, expect it to be quite slow :p.

[MOmonster]

Yeah, great.
A new tdeint version with a new type, poor static area detection, temporal/spatial stability, and noise resilience.
Sounds amazing.
Keep up this good work, tritical.

[Egh]

YES!!!

New TDeint announced!!! That's the best news in last days, and those last days included final version of AVS 2.5.6 and new version of mkvtoolnix 1.6.0 :)

The fact that it is slow is not important at all. Just make it better quality and less artefacts, and better detection. Since in dvdripping it's used only on few frames (better when it's not needed to be used at all :), speed doesnt' really matter much. But better detection is really needed.

[foxyshadis]

Any timeframe on when we can expect TIVTC2, then? :p j/k

[manono]

tritical-

That "tritical is a dumbass" is real cute and everything, but do you have to print it on my video? I see the message all the time, but it's the first time I've seen it on my video in bright red letters on the last frame of a black and white movie.

LoadPlugin("F:\DivX Stuff\GKnot\DGDecode.dll")
LoadPlugin("F:\DivX Stuff\GKnot\Undot.dll")
LoadPlugin("F:\DivX Stuff\GKnot\TIVTC.dll")
MPEG2Source("K:\Treasures3\3-1Movie\3-1.d2v")
TDecimate(Mode=0,CycleR=12,Cycle=44)
Undot()
Tweak(Sat=0)

Serving into CCE. Used the most recent TIVTC, as well as the previous one. Can't figure out how to get rid of the damn thing.

[Revgen]

@Tritical

Could you try to make TDeint2 compatible with TSP's MT function?

You can read about it here. (http://forum.doom9.org/showthread.php?p=711362#post711362)

Thanks.

[tritical]

That "tritical is a dumbass" is real cute and everything, but do you have to print it on my video? I see the message all the time, but it's the first time I've seen it on my video in bright red letters on the last frame of a black and white movie.
You should have told me before when you first saw it :p... tdecimate throws that error when the number of decimated frames in a cycle + the number of remaining frames don't line up correctly. In this case, there is a bug in tdecimate's handling of the last cycle in a video when that cycle doesn't have a full cycle's worth of frames and cycleR > 1. It effects modes 0 and 1 with cycleR > 1. I'll fix it in the next version, there isn't any way to work around it except to add extra frames to the video so that the last cycle isn't short.

Could you try to make TDeint2 compatible with TSP's MT function?
Work with which mode? TDeint2 eats a lot of memory and to save tons of repeated calculations stores temporary frames in its own buffers which are defined as class member variables... so it probably wont be compatiable with all modes.

[manono]

Hi tritical-

You should have told me before when you first saw it

Gee, I just figured that everyone got that message. It's never been a problem before, because the message wasn't imprinted on the video. Then thinking about it some more, I decided that previously I had been encoding AVI through VDubMod, and this was the first time that I had used the TDecimate Cycle Mode in CCE, as my usual silent film IVTC, SmartDecimate, wasn't working properly on this film. In this case, interestingly, the original DVD didn't go from the source framerate to 29.97fps via adding fields, but complete progressive frames. You hardly see that with silent films. So I just needed a good Decimator. Must be something about CCE that causes the message to be imprinted on the video. Although frustrated when I posted before (could you tell?), I've since solved the problem by first creating an uncompressed Lagarith AVI in VDubMod, and then frameserving that into CCE using AVISource. No more message. I'll remember your tip and Loop in some extra frames the next time the problem arises.

Anyway, thanks for the response, and I'll patiently await an updated version of TIVTC.

[tritical]

Having the error message reported (as in vdub) vs getting frames with the error message printed on them is dependent on the error handling of the external application... that is to say TIVTC has no control over it. In both cases TIVTC is simply calling env->ThrowError(). vdub/vdubmod are the only programs I know of that will present the actual error message instead of returning the error stream. If you open the script in a player such as mpc it will return the stream with the error message printed on the video.

I personally never have occasion to use cycleR>1 with modes 0 or 1, which is the reason the problem has managed to slip through. I have used it before on a few tests, but either missed the message or it happened that the length/cycle lined up correctly. The reason the message is worded the way it is (useless) is that I never thought anyone would get it (I originally used it for debugging purposes and it should never occur).

[manono]

Hi-

If you open the script in a player such as mpc it will return the stream with the error message printed on the video.

Yeah, I didn't actually see the message until testing the finished DVD in PowerDVD. It was definitely imprinted on the video. I opened the M2V in VDubMod to confirm. Lucky thing I checked, as I found it on another short silent film also. Strange, though, that CCE imprints it and VDubMod doesn't.

[Revgen]

Work with which mode? TDeint2 eats a lot of memory and to save tons of repeated calculations stores temporary frames in its own buffers which are defined as class member variables... so it probably wont be compatiable with all modes.

Preferably mode 1 or 2. They are the fastest. Modes 3 and 4 are slower. It doesn't really matter which one it works with as long as performance can improve. If TDint2 is as slow as you say it is, I would like to utilize any performance advantage that I can find.

TDeint for example gains about a 50% performance increase with Mode 2.

If you could explain how TDeint2 works to TSP he may have a better idea of what might work.

Thanks.

[Egh]

@tritical:

Just make it artifacts-free and I don't care about speed :) It can be 10 times slower than current mode 3, but what is really needed is artifact- and noise-free :)

In any way, during casual ripping process there are few frames (usually transitions, fadings and so on) which need to be deinterlaced, so actual speed here doesnt' matter much at all.

[tritical]

Still don't have the new docs done, and there were enough changes in this one that a beta before the final seems in order, so here is [link removed]. Changes:
TFM:
+ Added trimIn parameter (trim before tfm w/ d2v support)
+ Minor field matching improvement

TDecimate:
+ Added ssd parameter
+ Added vidDetect = 4 mode
+ Converted all metric storage/calculation to int64 to prevent possible overflow
on large frames
- Fixed mode 0/1 incorrect decimation of the last cycle of a clip when cycleR > 1
and the number of frames wasn't a multiple of the cycle length

FrameDiff:
+ Added ssd parameter (same as tdecimate)
I also reworded that error message slightly in case someone somehow gets it again :p.