TDeint and TIVTC

[tritical]

@puffpio
Yep, what you have should work. If you want to see exactly which frames it is dropping you can use: TDecimate(cycleR=3,display=true).


@All
[link removed], changes:
TFM:
+ assembly optimizations

TDecimate:
- don't require input and tfmIn files for mode 5

From the readme:

As of version 0.9.11.8 of TDecimate, an input and tfmIn file are no longer required for mode 5. If an input file is not specified then a value of ULLONG_MAX-1 (the maximum value tdecimate has) is assigned to all metrics. Likewise, if a tfmIn file is not specified then all matches are assigned as 'c', not combed, and not a d2v duplicate.

So be sure that vidDetect is set correctly when not using an input or tfmIn file.

The assembly optimizations to tfm are plain x86 assembly (not mmx/isse). I hand wrote the code for the only two remaining routines that use a significant amount of cpu time (actually they are the two costliest) in the default tfm/tdecimate code path and are not already using mmx or isse (they have too many branch and early exit conditions to make simd useful). On my systems the code runs about 2.5x faster than the compiler generated code and lead to an overall speed increase in tfm of 30-50% :).

Weird, I've been changing the title of my first post in this thread for a while, but today is the first time it actually changed what title is seen in the forum.

[Egh]

TIVTC v1.0 Beta 9[/URL], changes:
TFM:
+ assembly optimizations

TDecimate:
- don't require input and tfmIn files for mode 5




I was quite suprised but seems new TFM (compared to the one even 2 builds ago ^^) is seriously faster. The gain for metrics calculation in yatta is likely +20% or so. But I didn't test that extensively. Have you measured speed improvement for new build?

[anton_foy]

As I post for the first time in this great thread I give two THUMBS UP for tritical and everyone who has made this Deinterlacer possible! Beautiful work guys!
It is also very fast IMO.

Yet I have a slight problem. The footage I use is mpeg2(d2v) interlaced 1440x1088 that i first upsize to 1920x1080 this is my script:

mpeg2source("C:\...\snow.d2v")
LoadVirtualDubPlugin("C:\...\plugins\DeBlock.vdf","Deblock", 0)
Lanczos4resize(1920,1088)
Crop(0,0,0,-8)
interp = separatefields().eedi2(field=3)
tdeint(mode=1,order=1,edeint=interp)

The clip has very little movement in it, only a bit of wind blowing in the trees but on the branches it looks like the horizontal lines are jumping a bit up and down. I am guessing it is because the DeInterlacer interpolates the fields in a certain way. When there are some more movement it leaves small grains/pixels that are very distinct or contrasty.
This is not too bad but I just want to know if it is part of the DeInterlacer or just a bad scripting from my side.

[AS]

Best to deinterlace and then upsize.

[Chainmax]

tritical, you are awesome. I only wish there was something we could have done about those troublesome X-Men captures :(.

anton_foy: it is bad scripting on your part. Like AS said, if it's actually interlaced, either a) deinterlace and resize or b) separate fields, resize each of them and weave back. If it's telecined, IVTC and then resize. I don't know much about interlacing, but I heard many times here that directly resizing interlaced is a bad, bad idea.

[Didée]

Slowly, Chainmax & AS, slowly. What anton_foy does is fully OK. He is resizing only horizontally, thus leaving the interlaced structure completely untouched. Nothing wrong with that.

So far for protecting anton_foy ... now comes the bashing:

LoadVirtualDubPlugin("C:\...\plugins\DeBlock.vdf","Deblock", 0)

BOOO!! :D

[tritical]

@Egh
I was quite suprised but seems new TFM (compared to the one even 2 builds ago ^^) is seriously faster. The gain for metrics calculation in yatta is likely +20% or so. But I didn't test that extensively. Have you measured speed improvement for new build?
From beta 7 to beta 9 I got a speed up of about 30-40% for tfm on my laptop (pentium M). The sse2 stuff gave another slight improvement on my p4 system.

I would expect maybe even more of a gain from the last two builds with yatta than in normal operation since it uses micout=1. The new mmx/isse stuff in beta 8 was part of the checkcombed routine, which with default settings is usually only run once per frame, but with micout=1 is run 3 times per frame. The assembly optimizations in beta 9 effected the builddiffmapplane and comparefields functions which get used on every match comparison when slow=1 or 2 (builddiffmapplane isn't used with slow=0). Those 3 functions were also the ones taking up the most cpu time in tfm/tdecimate so speed improvements to them make quite a difference. I can probably still speed up slow=0 by a good bit, but not sure if anyone uses it.

@anton_foy
Your script looks ok to me, but even though you are only resizing horizontally it would still probably be better to deinterlace first and allow tdeint/eedi2 to work on the original image. And if nothing else, it will run faster (I don't even want to imagine how slow it runs on a 1920x1080 image).
The clip has very little movement in it, only a bit of wind blowing in the trees but on the branches it looks like the horizontal lines are jumping a bit up and down. I am guessing it is because the DeInterlacer interpolates the fields in a certain way. When there are some more movement it leaves small grains/pixels that are very distinct or contrasty.
This is not too bad but I just want to know if it is part of the DeInterlacer or just a bad scripting from my side.
Ususally bobbing does result in some jumpy horizontal lines since how a horizontal line gets interpolated will vary depending on which field is kept. It would probably be eliminated if you went for same framerate output (mode=0). The movement artifacts are most likely from imperfect motion-masking, if you want more resilient detection try using scharfis_brain's function that requires a longer static period.

@Chainmax
tritical, you are awesome. I only wish there was something we could have done about those troublesome X-Men captures . Me too, but I haven't had any insights about how better to deal with that particular problem. It's tough to come up with a reliable and automatic way for tfm to choose a match that every statistic it calculates says is worse than the other, and to only do it for the specific frames for which that happens. That's why I much prefer working with sane sources :D.

[foxyshadis]

Currently the state of hybrid->cfr decimation is pass most frames through, blend the recalcitrant segments. (Decimate mode 1/3, TDec's hybrid mode 1/3.) Is it possible to replace that with a motion-compensated clip (motion/mvtools 1.x) via the clip2 parameter in tfm? At worst they fall back on blending anyway. I'm not sure if I'm missing any special considerations or if it really is that simple. Also not sure how it would add a dup frame for tdecimate for every cycle in that case. Or would that have to be done to the clip2 clip beforehand?

I tried one clip with 60i and 24 telecine, and it seemed all right, but hard to tell. It has so little motion (and quality) in the 60i part, and the only dvd I knew was hybrid I sold. Maybe it's time to rip some more.

[DarkNite]

If you have any Excel Saga sitting around that would be a good start. ;)

Oh, the memories...

[Egh]

@Egh

From beta 7 to beta 9 I got a speed up of about 30-40% for tfm on my laptop (pentium M). The sse2 stuff gave another slight improvement on my p4 system.

I would expect maybe even more of a gain from the last two builds with yatta than in normal operation since it uses micout=1.

Well i didn't test that extensively, only ongoing projects. And as usuall I have background programs working, different priorities for metrics collector thread and most important -- there's also scene change detection filter working during ymc operation. So the gain from TFM metrics seems to be extraordinary, cause I was talking about gain in ymc step in general (I use ymc as default with TFM, slavc and TDecimate). Since metrics collection is only a fraction of that step then the speed of metrics collection just skyrocketed :P Overal time for one episode (720*480 24min) decreased from ~45min to approx. 32-33min. I guess that means metrics collection per se now is twice as fast as before :) And my cpu is Athlon XP 1800+, so SSE1 only. And even with only that, I'm close now to 1:1 ratio. So on something like 2200+ ymc should process one episode in real time then.

Btw, any suggestions for TFM setting as for ymc? I usually use TFF, mode=1, field=auto,slow=1,pp=7,MicM=0. That's pretty much default configuration in ymc. Still quite puzzled though with Chroma and mChroma parameters, seems doesn't have much effect...

[Chainmax]

Slowly, Chainmax & AS, slowly. What anton_foy does is fully OK. He is resizing only horizontally
...

Oh, I didn't notice that :o.

[anton_foy]

Thanks guys! Yes I dont think it would do any difference by rezising before or after TDeInt since deinterlacing is only a horizontal matter. I think I scripted it a bit better now...no jumpy or jittery lines and motion is stable but it is not using eedi2. New script:

mpeg2source("C:\...snow.d2v")
Crop(0,0,0,-8)
LoadVirtualDubPlugin("C:\...\plugins\DeBlock.vdf","Deblock", 0)
TDeint(mode=1,order=1,mtnmode=1)
Lanczos4resize(1920,1080)
LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=4,strength=150,soft=34)
ConvertToRGB32.Deblock(1, 1, 1, 1)

After treating the footage with this script it looks AMAZING! TDeInt is great and fast, 1,67 fps...hehehe it ten times faster than my previous deinterlacer. :)
BTW Didée, a bad choice of using DeBlock?

[Chainmax]

It's just that using VDub means converting to RGB and back, which unnecessary slows down the encode. Besides, there are many excellent deblockers for avisynth like DeBlock_QED or FunkyDeblock.

[tritical]

Currently the state of hybrid->cfr decimation is pass most frames through, blend the recalcitrant segments. (Decimate mode 1/3, TDec's hybrid mode 1/3.) Is it possible to replace that with a motion-compensated clip (motion/mvtools 1.x) via the clip2 parameter in tfm? At worst they fall back on blending anyway. I'm not sure if I'm missing any special considerations or if it really is that simple. Also not sure how it would add a dup frame for tdecimate for every cycle in that case. Or would that have to be done to the clip2 clip beforehand?
I think the better way might be to run the motion compensated conversion on the output from tfm (or you could leave out tfm or replace tfm with a deinterlacer), and then feed that to tdecimate. So a script for vfr->cfr with film and 30p would look like:

tfm()
newfps = mcChangedFPS() # 29.97 to 23.976
tdecimate(mode=1, hybrid=1, clip2=newfps)

I've thought about doing this before, but there are some problems. Mainly, most of the mc methods use a recursive approach that produces different results if you jump to random frames than if you run up to that frame linearly. In the above method, tdecimate wouldn't be requesting every frame but only random ranges of frames so it might cause problems. I am not up to date on all of the mc filters for avsiynth (the new mvtools and mg262's filter) and don't know how much that would really effect things. However, if it is a big problem tdecimate could easily be made to request every frame from clip2 in a linear fashion. I remember thinking about another potential problem with this before, but can't remember it now.

Btw, any suggestions for TFM setting as for ymc? I usually use TFF, mode=1, field=auto,slow=1,pp=7,MicM=0. That's pretty much default configuration in ymc. Still quite puzzled though with Chroma and mChroma parameters, seems doesn't have much effect...
Those settings seem fine to me. mChroma usually wont effect things much, especially in yv12 where there isn't much chroma information. Literally all it does is run the temporal/spatial differencing on the chroma planes and adds the results to the statistics gathered from the luma plane. The chroma results aren't multiplied up to adjust for the sub-sampling vs luma. If you want to see how much it actually changes things you can use debug=true with mChroma=true/false and check how much the match statistic numbers change. In the debug output it will also list the ratio between matches which indicates how well tfm could differentiate between the two matches (which is good and which is bad). The larger the number (>1) the better the recognition and the closer to 1 the worse.

chroma=true is pretty much only needed if there are frames that have chroma only combing. Otherwise, it usually just makes things worse due to false detections... a false detection in yv12 counts the same as finding 4 combed luma pixels since in the combed frame detection chroma is adjusted for the sub-sampling vs luma.

[foxyshadis]

I know motion can handle out-of-order access because of how its frame cache works, though it might have to recalculate an extra frame or two every time. It also plays havok with some conditional functions. Not sure if you have to access out-of-order (like TFM) but it would probably play a little nicer with them if you could do it sequentially.

Thanks for the pointers though, that helps me a lot. I didn't realize TDecimate had a clip2 as well, sicne it's not in the readme.

[tritical]

Thanks for the pointers though, that helps me a lot. I didn't realize TDecimate had a clip2 as well, sicne it's not in the readme.
It doesn't have such an option atm, that was just an idea. To me it seems like that would be the best way for things to work since the hybrid decision is made in tdecimate. Then when a hybrid section is detected it just requests the frames for that section from clip2 instead of doing the blend conversion on frames from the input clip.

For that matter, I had planned a while ago (I've forgotten exactly who asked for it) to allow decimation detection to be run on the input clip but to actually deliver frames from a second clip. I might tackle these two features after I finish optimizing slow=0 and write an mmx version of the isse comb checking functions. I think these new features would be suited to a v1.1 release.

[foxyshadis]

Another minor bug: "CFrameDiff does not have a named argument "ssd""

I look forward to a future version then, though I'm sure it'll be a little while until then.

[tritical]

Looks like I forgot to add ssd into the parameter string... it'll be fixed in the next version. I've completed all the optimizing stuff I wanted to do... so the next release will be v1.0 RC1 and I'll only do bug fixes on that version from now on. A v1.1 release will probably be out shortly after. Current things to add are:

1.) tdecimate clip2 for hybrid modes and to allow running decimation detection on one stream while delivering frames from another. These should be pretty quick/easy to add.

2.) ovrHelp file generation option for tfm. It would work such that you run field matching twice, once with field=0 and once with field=1, with output enabled. Then, tfm would take those two output files and generate a file that lists frame #'s where the matches don't correspond, where combed frame detections don't correspond, of possible missed combed frames or falsely detected combed frames (based on relative mic values of neighboring frames). It would also make a list of detected combed frames and frames where u/b matches were used.

The ovrHelp thing is mainly because I'm too lazy to go through and find things manually.

[tritical]

[link removed]. Only real changes were fixing the "ssd" parameter problem in CFrameDiff and fixing an issue with a few of the sse2 routines that were added to tfm/tdecimate in beta 8 (forgot that the count value for psrldq is # of bytes not bits). I also added an "opt" parameter to all the filters which allows forcing the use of c, mmx, isse, or sse2 routines (for testing purposes). The other changes were just optimizing slow=0 in tfm and porting assembly stuff from tfm/tdecimate into fielddiff/framediff.

[Chainmax]

tritical, would it be possible to make a faster version of EEDI2 for use as a standalone image enhancement filter like HQ2X (http://www.hiend3d.com/hq2x.html)?

[tritical]

A standalone version could be made, but it don't think that alone would really make it noticeably, if any, faster. Also, the nice thing about having it as an avisynth filter is I don't have to mess with opening/decoding sources, saving the result, etc... which is a big pain the ass :devil:. There are some potential areas for improving its speed, but only a couple of the routines in eedi2 lend themselves to simd implementations and they are the ones that use only a little of the current processing time. I could probably speed up the more intensive routines by writing the code in plain assembly myself, but that is a big job and I am still changing them some atm.

[Chainmax]

What about just making the standalone version without any optimizations? I'd love to see this implemented in emulators :).

[foxyshadis]

HQ4x looks a lot nicer (for sharp, low-res, high-contrast pixelly animation) and retains fine details, imho, and runs much faster than 2x2xeedi2. Eedi2 smooths edges somewhat, very similar to the much faster 2xsai or even cubic interpolation. It mostly outperforms others on curves on antialiased material.

HQ4x (http://foxyshadis.slightlydark.com/random/mmx_win_hq4x.png)
EEDI2 (http://foxyshadis.slightlydark.com/random/mmx_win_eedi2.png)

Not a representative sample, of course, and a bit of a matter of taste.

heh, I wonder if I could use isochroma's old neat image script to interface with photozoom pro, with its amazing (and amazingly slow) s-spline resize.

EEDI does have one significant advantage: You can use it as many times as you want, as far as I can tell. (Avisynth itself has problems with huge frames though; sucks since I'd love to use it as a poster-scan processor.) It's generally a very bad idea to chain HQXx, on the other hand, it causes bad haloing even on the second gen and just gets worse from there, along with uninterpolated areas standing out much more. It was created with one very specific purpose, and sucks for anything else, heh.

[Chainmax]

I don't like HQ4X much, in my opinion it makes the picture much too cartoonish. I prefer just using HQ2X and letting the video card scale up to monitor size (video cards use bilinear, right?). Therefore, for me it would be more interesting to see EEDI2 Vs HQ2X.

[tritical]

EEDI2 is really not that great for trying to enlarge a regular image. It takes too many risks, searches much farther than it needs to to find connections, and blurs too much. The same is also true of type=1/3 in tdeint. Interpolating a missing field in an image is a much different ballgame than normal resizing. I can think of a number of modifications that would improve how eedi2 performs when upsizing a normal pic. Also, I don't think EEDI2 could ever really compete with HQ2X or 4X on resizing those type of non-antialiased, blocky, high contrast video game images... that is specifically what HQ2X is designed for and it does a pretty good job. I am gonna try out a few modifications to improve how EEDI2 performs with normal images.

[foxyshadis]

Chainmax, I bet you would like something like 2xsai, eagle, or or even bicubic followed by a strong sharpen filter with few halos. I might ask at the zsnes board if that would be possible. HQXx are custom made for perfect sharpness, everything else fuzzy and unsharp, and you're right that it's quite annoying to have no middle.

Tritical, that would be awesome, I love it. =D I'm a little curious on how it works, since I get fuzzy on C algorithms. Does it fit a line/curve to each edge? Or try to create vectors from each point?

[Chainmax]

tritical: great, I hope you get some good results :).

foxyshadis: I never was much of a fan of image enhancement filters until HQ2X, but it's currently my favorite image enhancement filter for emulators. What do you mean by "followed by a strong sharpen filter with few halos", by the way?

[foxyshadis]

Tritical, I'm curious. I know EEDI has a very specific purpose, to double a field height, but as avisynth is currently without any really good vectorization functions this is about as close as it gets. Spline36resize is pretty decent too, sometimes even better, but overall it's all eedi. I've been toying around and found it can do amazing things, although artifacts build up after a few generations.

So I'm wondering if it would be possible to use the already calculated vectors to resize to any arbitrary width, interpolating as necessary, rather than repeated application and a final resize? I guess for too-large sizes it'd have to smooth out the vectors/curves more than normal. If that'd be too difficult or interfere with its primary use, then disregard.

Chainmax, I just meant to cut down the effect from the card's bilinear upsize, although I didn't find anything that's really preferable without being cartoony. HQ2x's (and 3/4) biggest issue is that it's limited to 3x3 blocks, which is why eedi and spline resizers look much smoother on long edges and curves, but are also significantly slower.

Quick comparison to the above 2 pics:
Photozoom Pro (http://foxyshadis.slightlydark.com/random/mmx_win_pzoom.png)

If I can compile it too, I'll add greycstoration's wavelet-ish resize, just for kicks.

Now that I look closer, eedi2 really does the best job by far on the background, even with a few generational artifacts.

[tritical]

I'm a little curious on how it works, since I get fuzzy on C algorithms. Does it fit a line/curve to each edge? Or try to create vectors from each point?
Basically, it finds a vector at each point and does linear interpolation in that direction, but it does it in two steps. It does an initial pass to get tentative directions at each point in the field and then it does smoothing/eroding/dialating of the direction map. When it interpolates the missing field off those directions it only connects pixels that are detected as having the same direction and matching in intensity. The directions are found using a 4 scan line vector matching method (which is a custom method that combines ideas from 3 or 4 papers I've read). The vector matching vs direction based off gradients (the tdeint type=1/pp=7 in tfm modes use a gradient direction method) is one of the differences when dealing with interpolation for deinterlacing versus normal resizing. The direction based of gradients method simply doesn't work very well for deinterlacing interpolation since half the signal is gone (poor detection as slope gets smaller), so vector matching is pretty much the only good method for connecting long edges/lines. Whereas if I was gonna make a normal resizer I would probably use a combination of gradient/vector detection.

So I'm wondering if it would be possible to use the already calculated vectors to resize to any arbitrary width, interpolating as necessary, rather than repeated application and a final resize? I guess for too-large sizes it'd have to smooth out the vectors/curves more than normal. If that'd be too difficult or interfere with its primary use, then disregard. It would be somewhat complicated, but if you simply used the direction of the point nearest to the interpolation point and then fit the desired curve or function in that direction it wouldn't be too difficult. Although, if you wanted to get sub-pel accurate on finding the desired points for interpolation it would definitely take a while to finish.

For the moment I've started working on the motion-mapping filter again (I am getting tired of seeing TDeint's motion-mapping failing all the time), but I've already added a directional based resizer for normal images to my todo list.

[Zarxrax]

@tritical or others:

With a mixed 24/30fps source, is it possible to create an output which preserves all frames but is NOT variable frame rate? This would essentially be like doing ivtc on the 24fps parts, and like an assumefps() on the 30fps sections.
This will produce an incorrect running time, but with the advantage of preserving all frames perfectly while still using a constant framerate.

This could be useful in some instances where the purpose is to do some kind of editing on the video stream, and you are only concerned with having all the frames look correct.

[foxyshadis]

Process as you normal would a hybrid in mode 4+5, and then use mkvout="nul". No timecodes, no problem! :D Set your fps as desired. (Will cause pain if you ever need to turn hybrid edits into watchable video though.)

120 fps is generally a better solution since durations and sync are preserved, and reverting it back to vfr or hybrid is relatively painless. Tritical made a neat little tool to turn a mode-5 decimated video into a super-fps one called avi_tc_package (http://bengal.missouri.edu/~kes25c/avi_tc_package.zip) you might like.

[Zarxrax]

Awesome, I should have thought of that :p

[Zarxrax]

@tritical: I have encountered what might possibly be a bug in tdeint... or maybe its a bug in blendbob, I really cant tell.

source = MPEG2Source("ep3.d2v", cpu=0)
deint = source.Tdeint(mode=1).BlendBob()
return deint


This script works fine for the most part, however there are certain frames where, while they will normally look correct, if I choose "refresh" in virtualdubmod, the frame will suddenly appear jagged like it was not being passed through blendbob. If you back up quite a ways and then come back, it will show fine again.

If I use a different bobber besides tdeint, this problem does not occur.


Edit: Also, there is a bug in cthresh of TFM. The documentation says that the minimum value is -1, and this should cause all frames to be postprocessed. However, -1 actually seems to do the opposite, and cause all frames to be reported as "clean".

[tritical]

Edit: Also, there is a bug in cthresh of TFM. The documentation says that the minimum value is -1, and this should cause all frames to be postprocessed. However, -1 actually seems to do the opposite, and cause all frames to be reported as "clean".
I'm not in a position to test atm, but it is probably a problem with the mmx/sse2 routines and negative values for cthresh. Try setting opt=0 in tfm and see if it works correctly.

I'll try to look into the problem with your blendbob script tonight sometime and see what's wrong.

EDIT: I took a quick look at the source and the assembly comb detection routines are indeed broken for cthresh values < 0. I'll fix it in the next release, thanks for reporting it.

[Zarxrax]

Regarding the tdeint problem, I *think* older versions of tdeint were working properly. I don't have an old version on hand to try it anymore though.

Edit: and here is a sample clip you should be able to see the problem with.
Open this script in virtualdubmod and look at frame 141. Then do edit>refresh.
http://zarxrax.kicks-ass.net/test.demuxed.zip

[tritical]

The reason the frame changes on reload is because BlendBob bases part of its decision about which frames to blend together on the average abs diff of the past 7 frames. When you run up to frame 141 linearly that history is completed filled, but when seeking to frame 141 it only has one valid value (from frame 140 which is processed before). This changes the result of the following if statement due to the fact that the average value calculation does not adjust for having less than 7 values (the unfilled entries are treated as 0 and bring down the average value substantially):

if ((pnThresh != 0.0) &&
(absDiffPN > (avgAbsDiff/5)) && // Don't freak out on plain Bob()
((double)usedDiff-(double)avgAbsDiff > (double)absDiffPN*pnThresh))

When run up to linearly that statement will be false for frame 141, but on seeking it evaluates to true and the average of P/N is used for frame 141 instead of P/C. I tried a quick hack of tracking the number of valid entries in the history when computing avgAbsDiff and then adjusting the calculation as necessary. After making the change it picked the correct blend for frame 141 when seeking. I'm afraid that Leak might have made it the way it is for reason though... so you might want to ask him about it. And even with the change, there is still the possibility for seeking to give different results than linear play.

[Zarxrax]

Ah, I see. Thanks for checking into it.

[rig_veda]

Does TTempSmooth have some effect on decisions of TDecimate further up in the filter chain?
I'm having a script that goes like (plugin loading part omitted):

mpeg2source("D:\Big\Project_Ghibli\Totoro.d2v")
TFM(PP=0,display=false,input="Totoro_TFM.txt")
TDecimate(input="Totoro_TDecimate.txt",display=true)

and another one

mpeg2source("D:\Big\Project_Ghibli\Totoro.d2v")
TFM(PP=0,display=false,input="Totoro_TFM.txt")
TDecimate(input="Totoro_TDecimate.txt",display=true)
TTempSmooth(maxr=7)

The TTempSmooth(maxr=7) is the only difference in the script.
Now TDecimate's output shows that even though it's running on the precalculated input files, it makes different decisions on one and the same frame in the first and in the second script. In one case, for example, it decided to use frame 336, in the other it to blend 334-336 (50.00,50.00). Now i thought that filters further down in the chain wouldn't effect the ones higher up, but just eat what they are fed with. And in this case, I was all the more surprised since i was using an input file with TDecimate. (Not using one doesn't make a difference in this case, though.)
What is going on here? :confused:

[tritical]

Would you be able to post the clip that causes the problem? You're right that having ttempsmooth shouldn't change things... I tested out that scenerio using the settings you posted on a clip on my computer but didn't encounter any problems.

[rig_veda]

I have no webspace to upload available atm, but I can send you a small version of the clip that exhibits the problem, if your mailbox handles large attachments. The size of the rar is 17.3 mbyte. You can contact me at fede2a01 at uni-trier.de
to give me your contact information, if you want to go that way. If not, I'll look into getting a space to upload.

[tritical]

You can upload the clip to my ftp if you haven't already found someplace else:

ip: 68.119.245.113
port: 17252
user/pass: upload/upload

as long as it's not larger than 2 or 3 GB's, any size is fine :). I don't think my email will take attachments larger than about 5 MB's.

[rig_veda]

Done uploading

[berrinam]

I've done a fair bit of searching and been unable to find anything said on this topic. I hope I'm not hijacking this thread by asking this here.


Is there a big difference between DeComb's combed frame detection and TIVTC's one?
Is TIVTC's more accurate?
Are the thresholds normalized to be consistent with each other?

I am asking because I am working interlace detection for MeGUI, and at the moment I'm using DeComb's IsCombed() function. I wonder if it would perhaps give better results to change to IsCombedTIVTC() for this.

[tritical]

@rig_veda
I looked at the clip and I was able to reproduce your results if I seeked to frame 268. If I ran up to frame 268 linearly from the beginning both scripts produce the same result.

Involved explanation of what happens
The difference is w/o ttempsmooth seeking to frame 268 causes only the cycle starting at frame 335, plus the previous (330) and next (335) cycles to be evaluated. However, with ttempsmooth in there to request extra previous frames, the cycle at 325 is also evaluated. The blend decision in mode 0 with hybrid=0 is based on seeing match dups + lowest metric'd frames at the right interval that it looks like an ivtc pattern change that produced two dups in one cycle. In this case the match dup in the previous cycle appears at pos 0 (frame 330), but without evaluating the cycle prior to that (starting at frame 225) it won't know the match used for frame 329 and therefore can't tell that frame 330 is a match dup. So with ttempsmooth it can tell frame 330 is match dup which triggers the detection and without it it can't so it doesn't trigger.

So the solutions are:
1.) don't seek
2.) if you don't want the blending for such cases when hybrid=0 then use "noblend=true" in tdecimate()
3.) Use "d2v=source.d2v" in tfm. In this case your source is almost entirely flagged (94% reported by tfm). With the d2v option enabled it doesn't try to blend in that cycle and picks up the correct ivtc pattern... plus it is faster :).

A problem though is that the following script should support seeking (assuming you use the output parameters on a previous pass to generate input files):

mpeg2source()
tfm(input="tfm.txt")
tdecimate(input="tdec.txt",tfmIn="tfm.txt")

but I seem to have made it so you can't use a tfmIn file with hybrid=0 for some reason... I'll have to fix that. I'm also considering making noblend default to false since most people would rather just keep the extra dup than see a blend in such cases.

[tritical]

@berrinam
They are mostly the same except that tivtc's function uses overlapping windows (the normal set plus 3 others offset at 0.5*blockx in the x direction, 0.5*blocky in the y direction, and 0.5*blockx in the x + 0.5*blocky in the y, the window size is variable (blockx, blocky) (fixed at 4x8 in iscombed), the window threshold amount is variable (MI) (fixed at 15 in iscombed), chroma parameter is adjustable (iscombed uses true), and it uses a different per-pixel combing metric.

Assume we have 5 pixels vertically and we want to know if 'c' is combed or not:

a
b
c
d
e

IsCombed's per-pixel metric:
val = (b - c) * (d - c);
if (val > threshold*threshold) it's combed;

IsCombedTIVTC's per-pixel metric:
d1 = c - b;
d2 = c - d;
if ((d1 > cthresh && d2 > cthresh) || (d1 < -cthresh && d2 < -cthresh))
{
if (abs(a+4*c+e-3*(b+d)) > cthresh*6) it's combed;
}

That pseudo code is not exactly how the metrics are implemented code wise, but is equivalent to what they do. So tivtc's cthresh and the threshold of iscombed() are both on a 0-255 scale, but will give differing results. IsCombed's threshold defaults to 20 and IsCombedTIVTC's cthresh defaults to 9... so it would seem to be more sensitive. However, tivtc's metric is tougher to trigger and it uses a larger default window size (16x16 vs 4x8), has to see more combed pixels (80 vs 15), and chroma defaults to false.

[scharfis_brain]

tritical, I have a question (or suggestion) regarding fieldmatching principles.

I want to discuss the low-motion (also no-motion) case.

The most fieldmatchers I know of will prefer a blind match on them even if the underlying video is true interlaced stuff thus risking the well known mouth combing.

How about introducing a threshold to identify these low/no motion scenes and the apply a motion adaptive deinterlacer on them?

The problem is pretty obvious with tdeint(mode=1,tryweave=true, full=false). -low motion interlaced scenes always are fieldmatched instead of being deinterlaced.

EDIT: IMO smartdecimate does something like this, in past I always wondered, why it started bobbing in low/no-motion scenes...

I mean fieldmatching should only be applied, if the algorithm is pretty sure that two fields are derived from one frame.
This is only the case with a reasonable amount of motion.

Of course, on can continue a running pattern (let the fieldmatcher adapt to a pattern), if the motion gets low or stops, but after a scene change it needs to reset this immediatelly.

[tritical]

@scharfis
Just a warning, I did a lot of rambling and thinking while writing this reply so it kind of jumps arounds. Also I was not sure whether you were specifically talking about TDeint or also TIVTC.

I mean fieldmatching should only be applied, if the algorithm is pretty sure that two fields are derived from one frame.
This is only the case with a reasonable amount of motion.

I think it depends on the material type. If you know the source is primarily telecined then I don't think having it use a motion adaptive deinterlacer in low motion areas is preferable because even the best ones aren't perfect and will give inferior results compared to pure field matching. However, if the source is primarily interlaced then it would probably give better results. I see the problem as one of combed frame detection and not field matching (which are two completely separate processes in TIVTC/TDeint). In fact, TIVTC's field matching has no notion of a frame being combed or not, it just decides which of two fields fits best to another field.

TIVTC's field matcher can handle low motion scenes (mouthes moving) pretty well (assuming a good match exists) because it uses temporal differencing in combination with field differencing (if it wants to compare matching two top fields A/B with a bottom field C, it first compares A/B to find where they are different and then only applies the spatial metric to the created A/C and B/C frames in those areas). That said, the field matching built into TDeint is much simpler than that of TIVTC's (though at this point it probably runs slower than tfm w/ slow=2). It doesn't use temporal differencing at all and just runs the spatial metric at every point and sums the result... so I would expect it to be much more inaccurate in low motion areas. Too check if the chosen match is combed the created frame is then passed off to the combed frame detection routine.

The reason the combed frame detection routine has such a hard time detecting frames which have very little combing or 'weak' combing (falls under the cthresh level) is simply that it is a tough problem. The comb metric isn't perfect and causes false detections due to noise and high frequency detail. Because of this, it cannot be made that sensitive (low cthresh and low MI values) without causing a lot of false detections.

In TIVTC, which works under the assumption that a large majority of the clip should be matchable, but that you will occasionally get a frame that isn't, it is usually better to let a few combed frames slip through every now and than to have a lot of false detections. However, in TDeint, which w/ tryWeave=true should work under the assumption that a majority of the clip is interlaced, it should be better to have a some false detections than to let a lot of combed frames slip through. So the low-motion idea might help. Or it might help to simply adjust the combed frame detection based on the motion level (the defaults for cthresh/MI/blockx/blocky are 6/64/16/16, so maybe use 6/15/4/8 in low motion areas). The filter could then vary the settings between those two ranges based on the measured motion level.

[tritical]

More thinking... Source type detection in areas without adequate motion is almost impossible. Therefore, the type of such areas must be determined based off assumption. TIVTC works under the assumption that the clip is mostly matchable. Therefore, it should assume that low motion areas are matchable. TDeint works under the assumption that the clip is mostly interlaced (not matchable). Therefore, it should assume that low motion areas are interlaced. Roughly speaking, the number of combed pixels present in frames from an interlaced clip is directly proportional to the amount of motion. Based on that, the amount of combing that the combed frame detection routine looks for (controlled by MI, blockx, blocky) when tryWeave=true should be based on the amount of motion, and it should be requiring less combing as the amount of motion decreases.

[MfA]

The directions are found using a 4 scan line vector matching method (which is a custom method that combines ideas from 3 or 4 papers I've read).You know it's funny, Ive only ever been able to find one other written account (admittedly a while ago) of somoene using vector matching instead of pel matching with an ELA type approach ... and that was in medical imaging to reconstruct dropped scanlines.

[Zarxrax]

I have a question about TDecimate modes 0 and 1. Do they both support random access, or do either of them expect the frames to come in a linear order?