TDeint and TIVTC

[Leak]

Could someone tell me, what are the advantages as to using/disabling mchroma?
Well, this controls whether TFM takes the color information into account when matching fields, in addition to the luma information.

There's material out there where the chroma has been botched (rainbows, color delayed a field or three vs. luma, plus other stuff I don't even want to imagine) so disabling it for matching will probably give better results in those cases. That, and it of course takes some extra time for the extra checks.

Of course, if you have a scene where the luma stays the same and only the chroma changes the outcome will be pretty random - but that's rather unusual for normal footage.

[foxyshadis]

tritical, here's a sample of a scene that's been giving me trouble:

http://www.bestsharing.com/files/HL7M0Yw241160/Sample.demuxed.m2v.html
I think it's pretty safe to say it's not TIVTC/TDeint's fault... output of separatefields (http://foxyshadis.slightlydark.com/random/badsimpsons.jpg).

BTW, tritical, did you have a chance to test out my newer compile? I think it's worth replacing the old one, just because apparently it crashes for older cpus.

[Chainmax]

I think it's pretty safe to say it's not TIVTC/TDeint's fault... output of separatefields (http://foxyshadis.slightlydark.com/random/badsimpsons.jpg).
...

I had already seen those scenes, but take a look at this:


Result:
http://img473.imageshack.us/img473/7389/result47923hu3.th.png (http://img473.imageshack.us/my.php?image=result47923hu3.png)

Corresponding frame with AssumeTFF().SeparateFields():
http://img473.imageshack.us/img473/2695/corresp47931fh5.th.png (http://img473.imageshack.us/my.php?image=corresp47931fh5.png)

Previous frame:
http://img232.imageshack.us/img232/3313/47915di1.th.png (http://img232.imageshack.us/my.php?image=47915di1.png)

So maybe some parts could be slightly improved.

[rottweily]

I have serious trouble in deinterlacing this tennis sequence:
LineNastyness-Interlaced.mkv : http://www.mediafire.com/?5nkmnk2nzyd
LineNastyness-TDeint.mkv : http://www.mediafire.com/?ci2jiitbbmi
LineNastyness-TDeint+EEDI2.mkv : http://www.mediafire.com/?amdxqmymie2
LineNastyness-TDeint-Type4.mkv : http://www.mediafire.com/?0dyzdq3nv2n
LineNastyness-TDeint-Type5.mkv : http://www.mediafire.com/?djcyjd0enm3

You can't miss the artifacts.
I've always used TDeint with all the matches I've encoded and till now never noticed any obvious artifacts.
As you can see I've also tried using EEDI2, but it didn't help.
Any simple solution?

Edit: I already see that I have an outdated version of TDeint, time to upgrade. Already tested it, has the same problem.
The new blend interpolation already works a lot better type=4 or 5, but still not ideal.

[tritical]

Sorry for the lack of replies, I just haven't had much free time lately. Next week is spring break though :) (Yes, we have spring break 2 weeks after everyone else).

@rottweily
The links wouldn't work for me, but if you set mthreshL=0,mthreshC=0 do the artifacts go away? If so (I am guessing they will), then it's just standard motion-adaptive artifacts, and there isn't much you can really tweak as far as TDeint to make it go away other than mthreshL/mthreshC/mtnmode. You might try running vinverse() afterwards, but that might not help much if the artifacts are a lot stronger than normal residual combing. It might be worth giving securebob() a try.

@Chainmax
I must be blind, what exactly am I suppose to be looking at in the images you posted?

@foxyshadis
I haven't, but I'll try it soon. The link to your version that I have on my website just links to Eedi2mt.zip on your webspace so anyone who gets it from there should get the latest version. I need to put a link to it in the first page of this thread also.

[foxyshadis]

Oops, I never noticed that. Hah!

I think chainmax refers to this:
http://foxyshadis.slightlydark.com/random/sresult.jpg
But that comes from:
http://foxyshadis.slightlydark.com/random/sorig.jpg

There's a lot of large and small artifacts just in that short piece posted, whomever transferred it really did a great job of mangling it.

[Didée]

@ rottweily (& tritical not having seen the clips)

The problem here is just the usual threshold disease ("up to x is static, bigger than x is motion").
Remember the look of the scene when a tennis match is filmed from the usual bird's view camera position. When the camera is perfectly static, then the "deinterlaced" lines are 100% static. But heaven forbid that the camera starts moving even so slightly ... then hell breaks loose. (The "deinterlacing hell".)
In that regard, it's a worst-case scenario ... the lines are of very high contrast (white lines with a small black halo). Because of the high contrast, even with mini-minimal motion the usual motion thresholds are exceeded immediately. And because of the high white-to-black contrast, the wobbling seems so extreme and annoying.

SecureDeint won't help here ... it's more "anxious" about motion, hence it will produce even more wobbling. (Probably, didn't try it.)

Bigger thresholds won't help either, you'll loose the ball to blending land.

What would (and will) help is MCBob's smart thresholding, which is adaptive to local complexity.
(Have been wondering that there were no requests to combine this smart-thresholding with some not-mo'comped deinterlacing, à la MVBob-->SecureDeint ...) :)

***

BTW, the idea behind and justification of the smart thresholding in MCBob is very easy: at each pixel's location, there is a certain min-max value of the neighborhood - it's small for flat areas, big for detail areas. Now: If we consider up to 1/4 pel motion as "static", and 1/2 pel motion as "motion", then changes up to (max-min)/4 (+N) for that pixel are considered static, pixel changes bigger than (max-min)/2 (+N) are considered "motion", and inbetween is considered inbetween. (N is a small tolerance value to compensate for general flicker.)

One example with numbers: on a black-white transition of some high-contrast detail, the min-max-span is, say, (max-min)=200. Now, if there is some minor motion of only 1/4 pel (which is neglectable for the matter of deinterlacing), the pixel might change by already +/- 50. Fifty! With (fixed) motion thresholds being usable mostly/only in the range 2~10, it's no wonder that deinterlacing usually produces shimmering, flickering and wobbling ...

Hard thresholding obviously fails miserably here.

Smart thresholding is a simple idea, works out, and free to use. ;)

[Chainmax]

...
What would (and will) help is MCBob's smart thresholding, which is adaptive to local complexity.
(Have been wondering that there were no requests to combine this smart-thresholding with some not-mo'comped deinterlacing, à la MVBob-->SecureDeint ...) :)
...

Well, McBob seems to be the best available deinterlacer/bobber we have right now, and people who use it are willing to sacrifice speed for quality. Therefore, I don't see much use in trying to enhance MVBob as it's already very slow and apparently not as good. SecureDeint might be a better candidate for this enhancement, but my favorite candidate would be Tdeint as it's extremely fast in comparison and according to WorBry's tests it's pretty close to the big three.

[rottweily]

What would (and will) help is MCBob's smart thresholding, which is adaptive to local complexity.
(Have been wondering that there were no requests to combine this smart-thresholding with some not-mo'comped deinterlacing, à la MVBob-->SecureDeint ...) :)


Didn't help enough, I will keep the interlaced version.

[tritical]

I might add the smart thresholding idea to TDeint. When you talk about the min/max of the neighborhood do you mean within the same field or both fields? say the x's are the top field pixels, the o's are the bottom field pixels, and we're trying to create the missing pixel 'c'. The neighborhood for determining the min/max for the motion threshold for 'c' would consist of which pixels?

frame 0 frame 1 frame 2
xxxxx xxxxx xxxxx
ooooo ooooo ooooo
xxxxx xxxxx xxxxx
ooooo c ooooo ooooo
xxxxx xxxxx xxxxx
ooooo ooooo ooooo
xxxxx xxxxx xxxxx

Maybe you could just copy the above diagram and indicate the needed pixels by making them bold. I was thinking it would be the 3 x's above and below 'c', but want to make sure.

EDIT: I've been looking at this a little more and I don't quite understand why the threshold is based on the range (max-min) of the neighborhood. Let's say we have the following 9 pixel neighborhood from a single field:

abc
def
ghi

if we allow 1/4 pel motion for pixel 'e' and assume linear interpolation will give an accurate result for this motion. Then the possible values for the new 'e', assuming only vertical or horizontal motion for the moment, would be:

0.75*e+0.25*b
0.75*e+0.25*f
0.75*e+0.25*h
0.75*e+0.25*d

Which would mean the threshold for 'e' should be the max of:

abs(e-(0.75*e+0.25*b))
abs(e-(0.75*e+0.25*f))
abs(e-(0.75*e+0.25*h))
abs(e-(0.75*e+0.25*d))

i.e.

max(abs(e-b),abs(e-h),abs(e-d),abs(e-f))*0.25

and for half pixel motion it would be:

max(abs(e-b),abs(e-h),abs(e-d),abs(e-f))*0.5

That reasoning can then be extended to simultaneous motion in both vertical/horizontal directions.

[Didée]

First of all, generally: I don't claim that things should be done in exactly the way as they're done in MCBob; heaven forbid. In MCBob it's a rough compromise for speed's sake, because some things are just too inefficient when done by limited means of scripting.

What I want is to raise the general idea, because it seems quite feasable. If we have an idea of the local detail level - and that "idea" is rather easy to get by spatial evaluation -, then the allowed range for pixel variance can be derived from it. After thinking a little over it, making the thresholds dependend on the present local variance seems much more logical to me, compared to simply allow "this_fixed_value" for just everything ...


Specifically:

The neighborhood for determining the min/max for the motion threshold for 'c' would consist of which pixels?
...
I was thinking it would be the 3 x's above and below 'c', but want to make sure.
Hereby ensured. ;)


I've been looking at this a little more and I don't quite understand why the threshold is based on the range (max-min) of the neighborhood.

Using the maximum difference to the center pixel (like you suggested) theoretically is more precise. I used local min/max for two-andahalf reasons:

a) it was much faster. min/max I get with mt_lutxy(mt_inpand,mt_expand). for max-diff I'd need mt_lutxy(mt_lut_xy(clip,clip.mt_inpand),mt_lutxy(clip,clip.mt_expand)).

b) the center "doesn't exist" yet. The center is the pixel we want to construct.
Therefore: calculating the maximum difference of the neighborhood to the interpolated-from-the-neighborhood center value, in the end is the same as taking min/max of the neighborhood, and divide by 2.

c) the "double sized" value of min/max is levelled out trivially: if we want to allow 1/4 pel motion as being static, this means 1/4 pel between neighbored top/bottom fields. But the field motion check is done between same-parity fields, i.e. the doubled distance. So, to allow 1/4 pel between two directly neighbored fields, the motion check has to allow 1/2 pel between same-parity fields ... that's the point where the "bigger" result of min/max gets levelled out. (That's not fully exact science, sure ... but after trying, it seemed to suffice.)


Somehow, talking about it seems to make it more complicated than it really is ...

[tritical]

I made an initial implementation of an adaptive-threshold motion-mask that can be used with tdeint (mmask.dll (http://bengal.missouri.edu/~kes25c/mmask.dll)). Currently, it is just a bunch of separate filters: mthresh, mmask, mcombine, and buildMM, that can work with tdeint through the emask option. Be aware that this is straight c and completely unoptimized so it is rather slow. I've implemented it as a set of separate filters to make experimenting easier.

mthresh(clip child, int type) -

This filter takes in a field based clip and will output a threshold map, where the threshold for each pixel is based on it's 3x3 neighborhood. The output frames will be twice the size vertically of the input. The top half of the output is the 1/4 pel threshold and the bottom is the 1/2 pel threshold. 'type' can be 0, 1, 2, or 3 (default is 1). For type = 2, the thresholds are simply:

max(abs(cp-nmin),abs(cp-nmax))*0.25
max(abs(cp-nmin),abs(cp-nmax))*0.50

where cp is the current pixel value and nmin/nmax are the min/max of the current pixel's top/left/right/bottom neighbors. type=3 is the same as 2, but considers all 8 neighbors. type = 0 is the same as 2 but compensates for the 2x difference in distance between lines (due to it being only a field), as well as for the distance differences due to downsampling in chroma. type=1 is the same as 3, but with the compensation as in 0.


mmask(clip child, clip mask, int type, int nt, int minthresh, int maxthresh) -

This filter takes in the mask output by mthresh (the 'mask' clip should be set to this) and the field based video that was given as input to mthresh (each frame needs to be stacked vertically with itself) (the 'child' clip should be set to this). It will then output a mask indicating which pixels are moving (0) or static (255). A pixel is static if the difference between it and the pixel in the other frame is less than min(cthresh,othresh) where cthresh is the threshold determined by mthresh for the current pixel and othresh is the threshold determined by mthresh for the other pixel. type can be set to 0, 1, or 2 (default is 2). type=0 will do difference to previous frame, type=1 will do difference to previous/next, and type=2 will do difference to next. nt is the noise threshold (default 2)... nt will be added to min(cthresh,othresh). minthresh is the minimum threshold value (if min(cthresh,othresh) is less than minthresh then it will be increased to minthresh) (default is 0). maxthresh works the same way as minthresh but sets the maximum (default is 75).


mcombine(clip child, int cstr) -

This filter takes the output from mmask, which is still double size vertically and combines the 1/4 pel and 1/2 masks into one mask. Basically, if a pixel is marked as moving in the 1/4 pel mask but is marked as static in the 1/2 pel mask, then mcombine checks to see how many of that pixels neighbors in its 3x3 neighborhood are marked as static in the 1/4 pel mask. If at least 'cstr' of its neighbors are, than that pixel is considered static in the final mask (default for cstr is 4). All pixels marked as static in the 1/4 pel mask are also considered static in the final mask.


buildMM(clip tfields, clip bfields, int mode, int order, int field, int length, int type) -

This filter takes in two clips corresponding to top fields and bottom fields. These clips should be the output of running mthresh, mmask, and mcombine separately on the top and bottom fields of the video. buildMM assumes that type=2 was used when running mmask(). mode/order/field are exactly the same as the same parameters in tdeint and should match the settings you intend to use in tdeint (defaults are 0/-1/-1, -1 means the same as it does in tdeint). length specifies the static field period for weaving a pixel (to get a 5-field check as in tdeint use length=5, to get an 8 field check as in securedeint use length=8, etc...) (length must be >= 4 and <= 60, default is 8). 'type' specifies whether both neighboring vertical lines of the line attempting to be weaved have to agree on the direction (back, across, or forward) of the static period. If type=0 then they don't, if type = 1 then they do (default is 0). The output of buildMM is a mask usable by tdeint through its 'emask' parameter.


Here's an example of doing bobbing on a bff clip:

mpeg2source("C:\t4.d2v")
bot = separatefields().selecteven()
top = separatefields().selectodd()
topf = mmask(stackvertical(top,top),maskc=mthresh(top)).mcombine()
botf = mmask(stackvertical(bot,bot),maskc=mthresh(bot)).mcombine()
tdeint(order=0,mode=1,emask=buildMM(topf,botf,mode=1,order=0))

here's what it would be for tff:

mpeg2source("C:\t4.d2v")
top = separatefields().selecteven()
bot = separatefields().selectodd()
topf = mmask(stackvertical(top,top),maskc=mthresh(top)).mcombine()
botf = mmask(stackvertical(bot,bot),maskc=mthresh(bot)).mcombine()
tdeint(order=1,mode=1,emask=buildMM(topf,botf,mode=1,order=1))

and here's same rate deinterlacing on a tff clip (keeping top fields):

mpeg2source("C:\t4.d2v")
top = separatefields().selecteven()
bot = separatefields().selectodd()
topf = mmask(stackvertical(top,top),maskc=mthresh(top)).mcombine()
botf = mmask(stackvertical(bot,bot),maskc=mthresh(bot)).mcombine()
tdeint(order=1,field=1,mode=0,emask=buildMM(topf,botf,mode=0,order=1,field=1))


Any suggestions are welcome. I've been testing this on some difficult clips and it seems to work considerably better than tdeint by itself. Probably the part most up for change is mcombine(), as the method it uses to combine the 1/4 pel and 1/2 pel masks isn't exactly ideal.

mmask.dll also includes the following two filters:

dilate(clip child, int dstr)
erode(clip child, int estr)

dilate will mark any pixel that is 0 as 255 if >= 'dstr' of its 8 neighbors are not 0.

erode will mark any pixel that is not 0 as 0 if <= 'estr' of its 8 neighbors are not 0.

Those are there for possible processing of the intermediate masks.

EDIT: mthresh, mmask, mcombine, buildMM, dilate, and erode will all work with YV12 or YUY2.

[tritical]

Not sure if anyone else tested, but I found that there were still some artifacts on a couple sequences. Having mmask check differences both 1 frame forward and 2 frames forward and then anding those two masks fixed it though. Gonna post an updated version of mmask.dll later. Hopefully after a little more testing I'll start merging it all into one filter.


New version of mmask.dll (http://bengal.missouri.edu/~kes25c/mmask.dll). Changes from before:

mmask() -

The 'type' parameter can now be set to 0, 1, 2, 3, 4, or 5 (default is 3).

0 = diff to n-3
1 = diff to n-2
2 = diff to n-1
3 = diff to n+1
4 = diff to n+2
5 = diff to n+3

static vs moving is determined as explained previously. 'minthresh' now defaults to 5. This may result in a tiny amount of residual combing, but was needed to stop the background from wavering in the "scrolling text" test clip from http://media.xiph.org/vqeg/TestSeqences/Reference/.


andMasks(clip clip1, clip clip2, int dis) -

This filter will output an and'd mask of clip1 and clip2. clip1 should be generated with mmask(type=3) (i.e. single frame forward diff), clip2 should be generated with either mmask(type=4) or mmask(type=5) (two frame or three frame forward diff). dis should be set to 2 if you use type=4 or 3 if you use type=5 (default is 2). For the dis=2 case, the filter will grab frame 'n' from clip2 and and it with frames n and n+1 from clip1. If dis=3 then it will and frame 'n' of clip2 with frames n/n+1/n+2 of clip1.


buildMM() -

There is a new parameter called 'dis' (an integer) (default is 1), which if you use andMasks() to create the motion masks should be set to the same value as the 'dis' parameter of andMasks(). Basically, 'dis' tells buildMM how many frames forward a pixel is stationary if it is marked as 255 in the mask. Also, 'type' can now be set to 0, 1, or 2 (default is 1). 1 and 2 are the same as 0 and 1 were before. 0 is now a less restrictive version of 1... theoretically, 0 should weave the most but be the most artifact prone, 2 will weave the least but be the least artifact prone.

bobbing, tff, 1+2 frame diff:

top = separatefields().selecteven()
bot = separatefields().selectodd()
topm = mthresh(top)
botm = mthresh(bot)
tops = stackvertical(top,top)
bots = stackvertical(bot,bot)
topf1 = mmask(tops,topm,type=3)
botf1 = mmask(bots,botm,type=3)
topf2 = mmask(tops,topm,type=4)
botf2 = mmask(bots,botm,type=4)
topf = andmasks(topf1,topf2,dis=2).mcombine()
botf = andmasks(botf1,botf2,dis=2).mcombine()
tdeint(order=1,mode=1,emask=buildMM(topf,botf,mode=1,order=1,dis=2))

bobbing, bff, 1+2 frame diff:

bot = separatefields().selecteven()
top = separatefields().selectodd()
topm = mthresh(top)
botm = mthresh(bot)
tops = stackvertical(top,top)
bots = stackvertical(bot,bot)
topf1 = mmask(tops,topm,type=3)
botf1 = mmask(bots,botm,type=3)
topf2 = mmask(tops,topm,type=4)
botf2 = mmask(bots,botm,type=4)
topf = andmasks(topf1,topf2,dis=2).mcombine()
botf = andmasks(botf1,botf2,dis=2).mcombine()
tdeint(order=0,mode=1,emask=buildMM(topf,botf,mode=1,order=0,dis=2))

same rate, tff, keep top field, 1+2 frame diff:

top = separatefields().selecteven()
bot = separatefields().selectodd()
topm = mthresh(top)
botm = mthresh(bot)
tops = stackvertical(top,top)
bots = stackvertical(bot,bot)
topf1 = mmask(tops,topm,type=3)
botf1 = mmask(bots,botm,type=3)
topf2 = mmask(tops,topm,type=4)
botf2 = mmask(bots,botm,type=4)
topf = andmasks(topf1,topf2,dis=2).mcombine()
botf = andmasks(botf1,botf2,dis=2).mcombine()
tdeint(order=1,field=1,emask=buildMM(topf,botf,order=1,field=1,dis=2))

So far this has managed to get through every clip that tdeint, using its internal motion masking, produces motion related artifacts on.

Seems I spoke too soon. Defaulting to minthresh=5 in mmask() to fix the wavering on the scrolling text clip results in artifacts in another video. Those artifacts can be fixed by setting minthresh=2, but that causes problems for the scrolling text clip, or by setting length=10 in buildMM().

[Terranigma]

You and Didée makes this all seem like jibber-jabber. :p
I'll give the new mmask a try. Thanks for the .dll and thorough explaination. :D

[tritical]

I noticed two bugs in buildMM() that would cause it not to weave pixels when it should have. I updated mmask.dll (same link as above). I also changed the default for 'length' in buildMM() to 10. mmask() was also changed so that instead of requiring diff < min(cthresh,othresh) it's now diff <= min(cthresh,othresh) and the default for 'minthresh' was changed to 4 (was 5 previously).

[ChiDragon]

Cool, if I set length=12 with this mmask thing it fixes a short sequence that causes huge artifacting normally. I guess that's not really a test of the new thresholding though? I'm guessing if TDeint by itself could be set to 12-field check it would also be fine on these frames (but then I don't really understand most of this discussion).

[tritical]

I added a new filter to mmask.dll (http://bengal.missouri.edu/~kes25c/mmask.dll) called createMM(), which combines mthresh/mmask/andMasks/mcombine. Syntax:

createMM(clip, int type, int nt, int minthresh, int maxthresh, int cstr)

type is from mthresh
nt/minthresh/maxthresh are from mmask
cstr is from mcombine

dis is fixed at 2. The defaults are the same as for the separate filters.

Now things are a little simpler:

bobbing, tff:

topf = separatefields().selecteven().createMM()
botf = separatefields().selectodd().createMM()
tdeint(mode=1,order=1,emask=buildMM(topf,botf,mode=1,order=1,dis=2))

bobbing, bff:

botf = separatefields().selecteven().createMM()
topf = separatefields().selectodd().createMM()
tdeint(mode=1,order=0,emask=buildMM(topf,botf,mode=1,order=0,dis=2))

same rate, keep top field, tff:

topf = separatefields().selecteven().createMM()
botf = separatefields().selectodd().createMM()
tdeint(mode=0,order=1,field=1,emask=buildMM(topf,botf,mode=0,order=1,field=1,dis=2))


@ChiDragon
You're are probably correct. The longer the static period required the more resistant it will be to artifacts from repeated motion. I think a good example of the adaptive threshold in action is src20_ref__525.yuv from http://media.xiph.org/vqeg/TestSeqences/Reference/. TDeint requires mthreshL=13 to get the same level of stability as createMM()+buildMM(), and actually using mthreshL=13 on any material with motion will lead to lots of residual combing and artifacts which you wont get with createMM()+buildMM().

Would you be willing to upload the short sequence you mentioned earlier to my ftp?
12.216.251.99:17262
upload/upload

[ChiDragon]

I uploaded the clip. :)

[tritical]

@ChiDragon
Thanks :).

@All
I updated mmask.dll again, only changes were a few optimizations. Atm, using tdeint w/ createMM()+buildMM() is about 3x slower for me than using tdeint w/ its internal motion-masking. It can probably be sped up some more, but considering it's doing a 10 field check (by default) vs tdeint's 5 field check and is using per-pixel adaptive thresholding, 3x slower seems pretty reasonable.

[Adub]

Well, it looks like I might have to start checking this out.
Thanks alot for the new filter Tritical!

[tritical]

Updated mmask.dll again. Changes were all to mthresh(). 'type' now has options 4 and 5, which use 4 and 8 neighbors respectively, but use the range (max-min) of the neighborhood instead of diff to the center pixel. I also added four new parameters: mtqL, mthL, mtqC, mthC which can be used to set hard thresholds instead of using adaptive thresholds. mtqL sets the quarter pel threshold for luma, mthL sets the half pel threshold for luma, mtqC/mthC are the same but for chroma. If these parameters are set to -1 (the default) then an adaptive threshold is used, if they are between 0 and 255 inclusive then the value of the parameter is used as the threshold for every pixel.

The above changes to mthresh() were also included in createMM(), whose type parameter can be set to 4 or 5, and which now has mtqL, mthL, mtqC, and mthC parameters.

I'm planning to create one filter called tMM() that does everything internally so that deinterlacing would simply be:

tdeint(order=xx,mode=xx,field=xx,emask=tMM(order=xx,mode=xx,field=xx))

[tritical]

Didn't take long... TMM.zip (http://bengal.missouri.edu/~kes25c/TMMv1.zip). It includes a readme and source code. Basic usage examples:

bobbing, tff:

tdeint(mode=1,order=1,emask=TMM(mode=1,order=1))

bobbing, bff:

tdeint(mode=1,order=0,emask=TMM(mode=1,order=0))

same rate, keep top field, tff:

tdeint(order=1,field=1,emask=TMM(order=1,field=1))

same rate, keep bot field, bff:

tdeint(order=0,field=0,emask=TMM(order=0,field=0))

[Terranigma]

I'll try this out, thanks. Hey, any news on what we've discussed via pm? :D


Update:
Very nice work tritical. Would it be possible to use tmm with, lets say, bob() to make it adaptive? If so, how would I go about doing that?

By the way, you forgot to add another bracket at the end of the second example =P
tdeint(order=0,field=1,emask=TMM(order=0,field=1)

[tritical]

I've told about 5 people that I'd look into something for them and haven't gotten around to any of it due to working on this motion-mask thing. Anyways, I did download tprivtc earlier in the week to try and fix the problems you mentioned, but it turns out that it uses fstream.h functions for doing the file/text stuff so I couldn't get it to compile straight away with vs.net 2003 (which doesn't have fstream.h and switching to iostream+fstream didn't fix everything) so I put it away for later. I'll probably mess with it tommorrow.

Very nice work tritical. Would it be possible to use tmm with, lets say, bob() to make it adaptive? If so, how would I go about doing that? Not sure in what fashion you mean, but the main problem with using it in conjunction with bob() via some type of masking would be that bob shifts both fields. If you just want cubic interpolation with motion-adaptation then TMM w/ tdeint(mode=1,type=0) would work.

thanks for the heads up on the typo in the read me.

[Terranigma]

Not sure in what fashion you mean, but the main problem with using it in conjunction with bob() via some type of masking would be that bob shifts both fields. If you just want cubic interpolation with motion-adaptation then TMM w/ tdeint(mode=1,type=0) would work.

That's what I meant. I'll try your suggestion there instead. :thanks:

[Chainmax]

@ChiDragon
Thanks :).

@All
I updated mmask.dll again, only changes were a few optimizations. Atm, using tdeint w/ createMM()+buildMM() is about 3x slower for me than using tdeint w/ its internal motion-masking. It can probably be sped up some more, but considering it's doing a 10 field check (by default) vs tdeint's 5 field check and is using per-pixel adaptive thresholding, 3x slower seems pretty reasonable.

Only three times slower? That's still ~20% faster than SecureBob/Old TDeint+EEDI2, ~220% faster than MVBob and ~310% faster than MCBob! :eek: tritical, you're a genius :) http://smilies.vidahost.com/otn/wink/thumb.gif.

[foxyshadis]

You might still want to use eedi2 with tdeint's edeint parameter though, which will slow it down a bit more. Using emask+edeint, there's hardly any reason to keep tdeint around though. :p

[Chainmax]

Well, like I said, this is still ~20% faster than Old TDeint+EEDI2. Besides, someone here said that using EEDI2 with the edeint parameter yielded a softer picture than plain TDeint. Not to mention that EEDI2 has artifacts of its own.

[canuckerfan]

so are we better off leaving out EEDI2 or is it best left to our discretion?

[Chainmax]

I never noticed the differences myself, but then again I never conducted a test either. Theoretically speaking, using EEDI2 might have some benefits in specific situations so I'd advise you to conduct a few tests with a variety of sources (at least 1 real life clip and one animation clip) and see if you can spot a difference.

[tritical]

You just have to try it and see which you prefer, as every interpolation method (edge-directed or not) has it own good points and bad points. If you want to look at individual frames than eedi2 will usually look better (especially if there are high contrast diagonal lines/edges), but it also has worse compression (usually) and worse temporal stability (not really the correct term, but not sure what to call it). Also, since interpolation will mainly be used in moving areas, which people are less sensitive to during normal speed playback, the benefits of edi may not be that great. kernel interpolation will also tend to look better in areas falsely detected as moving or in very slow moving areas.

The other thing you have to consider, if you are encoding the result and have size requirements, is the compressibility of the output. A deinterlacer that looks great but whose output is much harder to compress will have to suffer more distortion to reach the same file size as the output of a deinterlacer that is more compressible.

I did some tests using two sources, the first combined srcs 3/6/8/9/10 from http://media.xiph.org/vqeg/TestSeqences/Reference/, and the second was just src 20 (from the same place) by itself. I bobbed both clips with a number of deinterlacers, and then encoded the result with xvid at constant quant 2 using my usual settings (motion 6, vhq 4, chroma motion, bframes: 1/1.5/0.75, no qpel or gmc, h.263). The results were:

deinterlacer - Size (KB) Time (Min:Sec)

source 1:
bob - 124,449 8:02
securebob(type=2) - 133,936 8:46
tdeint+tmm - 136,485 11:39
tdeint - 139,728 9:24
tdeint+tmm+eedi2 - 142,171 32:00
securebob(type=3) - 144,748 30:08
tdeint+eedi2 - 146,589 30:21
leakkernelbob - 148,618 8:21
tomsmocomp(x,5,0) - 162,642 7:59
yadif(mode=1) - 162,672 8:35
yadif(mode=3) - 164,219 8:34
dgbob - 185,154 9:13

source2:
tdeint+tmm - 8,975 1:53
tdeint+tmm+eedi2 - 9,386 6:40
yadif(mode=3) - 11,812 1:19
yadif(mode=1) - 13,271 1:20
securebob(type=2) - 13,746 1:22
securebob(type=3) - 14,579 6:10
leakkernelbob - 16,663 1:15
tdeint - 17,253 1:24
dgbob - 18,616 1:17
tomsmocomp(x,5,0) - 20,437 1:19
bob - 21,637 1:23
tdeint+eedi2 - 21,875 6:11

The first source was pretty much all motion (except for the scrolling text sequence which is stationary apart from the text, which takes up probably 1/4 of the frame). The second source is almost completely stationary, but does have some noise (TDeint would require mthreshL of about 14 to handle it well). I did the tests on my laptop which has a 1.6 Ghz Pentium M and 1 GB of PC2700.

[canuckerfan]

Thanks for those figures, tritical. Makes things easier around here. Looking forward to improvements to tMM:)

[ChiDragon]

tritical, Fizick's Yadif plugin doesn't create artifacts on that clip I sent you when the "spatial interlacing check" is enabled. Do you know if this is just the equivalent of enabling "AP" in TDeint, or something else?

[tritical]

Roughly speaking, it is. The thing about yadif is it works sort of in the opposite direction than tdeint. It starts with a spatial prediction (interpolated value) and then limits that value to be within p2 + diff and p2 - diff, where p2 is the temporal prediction (weaved value) and diff is (again roughly speaking) a measure of local temporal change or complexity (for mode = 0/1). If there are no temporal changes then you end up with p2, the more temporal change there is the more likely it is you will just get the spatial prediction value back out. Using only the temporal differences to generate the 'diff' value makes yadif as susceptible to motion-adaptive artifacts on repeated patterns as any other motion-adaptive deinterlacer with a 5-field check. On the other hand, it is less susceptible to residual combing because in such cases, the diff value will exceed the difference between the spatial prediction and the weaved value (which in the case of soft residual combing, such as on a fadeout, is usually < 4) and you'll just get the spatial prediction value (or close to it).

When you switch to mode 2/3 diff is also based on local, vertical spatial complexity between the current field and the temporal predictions of the current line and the above/below lines in the opposite parity field. Basically, that means diff will also be based on how large of a vertical gradient weaving will create (that is maxed with the temporal diff calculation). Therefore, areas that would experience the types of artifacts you get from incorrect weaving will have a large diff value and you'll get the spatial prediction value. The downside is that true static areas with lots of detail (specifically those with large vertical gradients) will no longer be weaved. This is pretty much exactly what 'AP' in tdeint accomplishes.

EDIT:

I actually find the idea behind yadif quite interesting, but the major drawback IMO is the edi method it is uses is overly prone to artifacts. It uses the typical two sliding window method which is highly likely to choose the wrong direction around anything more than a single, thick, lone edge. Every other filter/program that I know of that uses this method caps the output value to be within +-2 or 3 of the min/max of the vertical neighbors to prevent major artifacts. Just to see what would happen, I created a c only version with such capping and to me it looked much better. I also made it possible to take spatial predictions from an external clip. I have some results here: deinterlace_comparison.txt (http://bengal.missouri.edu/~kes25c/deinterlace_comparison.txt) from a comparison I am working on, which I think show the benefits.

I have also been thinking of testing out a filter that would take tdeint's output, along with it's motion-map, and then limit only the interpolated values using the same method as yadif, and I am still working on a new interpolation method that should give edi benefits without taking hits in psnr/ssim as most edi methods do now. Although, that project has been on going for almost 9 months now so it could still be a while.

[AVIL]

Hi:

I've tested Tdeint and Yadif in a very noisy video. Yadif have less jagged edged but more orphan pixel type artefacts. I prefer Tdeint results.

To eliminate all the residual combing (my main problem with tdeint) i've setted mtqL, mthL, mtqC and mthC to zero.

IMHO the sole thing it rests is fight against the aliased edged. But, anyway , I am very pleased whith the last addition.

Thanks.

[swaaye]

Does TIVTC use SSE2 on Core 2 CPUs? I ask because it says P4 only in the changelogs.

[Boulder]

It uses SSE2 instructions if your CPU supports them. So the answer to your question is : yes.

[tritical]

I changed it a little while ago, must have not added it to the changelog. It requires either:

1.) family code = 0xF
2.) family code = 0x6 and model number = 0xF or 0xE

So pretty much the only models excluded are pentium M/celeron M. I have those use iSSE/MMX as testing with my laptop (pentium M) showed it was slightly faster. Of course, you can force tivtc to use sse2 via the 'opt' parameters of tfm/tdecimate.

[ChiDragon]

tritical, could you please add TFM hints to TDeint with tryweave=true? I think it would make it a lot more clear what TDeint(mode=1,tryweave=true) is doing when followed by TDecimate(display=true) for videos with weird orphan field pulldown patterns.

EDIT: Actually, I guess what I'd really like is an output file that could be read with TDecimate's tfmIn option, since these patterns are part of large cycles.

[Asrial]

Tritical, did you get my e-mail about the problem I encountered with video sections being decimated in the later versions?

[swaaye]

I have a source that is identified by DGIndex as 100% video, but it still reacts favorably to an IVTC. This is a USA TV show and I did a frame by frame analysis manually to check the frame order and it is definitely telecined.

[ChiDragon]

swaaye... no offense, but what is your point? DGIndex's report about 100% video is just based on the lack of pulldown flags (i.e. it's ENCODED as video rather than film, not that the source itself wasn't telecined from film).

[swaaye]

Didn't know if you knew of that possibility.

[canuckerfan]

tritical, how's tMM development coming along? I'm looking forward to the speed improvements :)

[tritical]

@ChiDragon
I can do that.

@Asrial
I did, but I haven't gotten to it yet. I will though.

@swaaye
That scenerio is pretty much the norm in R1 anime. Though pretty rare as far as most movies goes.

@canuckerfan
It hasn't gone anywhere so far. I'm busy with work and school again. However, there really isn't a whole lot of optimization that can be done aside from making it multi-threaded and better internal caching.

[ChiDragon]

For now at least I've found that this seems to do the same job as TDeint(mode=1,tryweave=true) except that it gives hints:

Interleave(TFM(mode=0,slow=2,micmatching=0,PP=7),SeparateFields().Trim(1,0).Weave().TFM(mode=0,slow=2,micmatching=0,PP=7))

AVSTimer even shows it running at the same speed. Of course there's no way to get a useable output file for tfmIn from that so a real option would be preferable. :p

EDIT: Hmm... I'm attempting to use the above code with TDecimate(cycleR=2,cycle=4) before another TDecimate call so that I can use smaller cycles and sdlim with the second one. The first one detects and uses hints while the second doesn't (forcing hint=true doesn't help either). Shouldn't the first one be preserving the hints? I'm using v1.0.1.

EDIT 2: Darn, TDecimate(cycleR=2,cycle=4) doesn't quite work right for this anyway. It drops an extra frame for a cycle like this:

Z | A B | C
Z | A C | C

(frames on x, fields on y)

[Myrsloik]

Version 1.0.1 of TIVTC (no idea about earlier) seems to have a bug when TFM's d2v argument is used with mpeg2dec3 and dvd2avi 1.77.3.

Whenever I try to use it it says:

TFM: d2v frame count does not match filter frame count (1968 vs 1966)!

or similar where the difference in the number of frames always seems to be 2 in all tested files. If I use dgmpgdec 1.4.9b17 it works properly with all files.

[foxyshadis]

mpeg2dec3 and dvd2avi 1.77.3.
Why would you do such a thing to yourself.

[tritical]

@ChiDragon
TDecimate currently destroys tfm hints on output (it preserves dgdecode hints, i.e. colorimetry). The hints from tfm don't really make sense after tdecimate has removed frames. What hint information would still be relevant?

@Myrsloik
The 2 frame difference is due to the following reasons (taken from neuron2's dgmpgdec page):

2.) MPEG2DEC cuts two frames from the frame count as a workaround for 3 below. This is a kludgy hack that should not be necessary. Thus, thanks to this and 1 above, 3 frames will always be lost. They are lost at the end.

3.) If the opening GOP has B frames before the first P frame (IBBPBBP...), then DVD2AVI generates an incorrect D2V file, in which the first digits for the orphaned B frames and some remaining digits are written out of place. Also, MPEG2DEC cannot decode the B frames prior to the first P frame, and so discards them. A number of frames will be lost equal to the number of B frames prior to the first P frame. They are lost at the beginning.

So, for example, if you process a VOB that has an IBBPBBP... opening GOP, you will lose a total of 5 frames, with 2 lost at the start and 3 lost at the end.

In addition to the lost frames, MPEG2DEC does not implement random frame access correctly. In fact, it always throws away the first B frames in the GOP prior to the first P frame. If they are (say) frames 12 and 13 (in display order) and you try to seek to 12, MPEG2DEC will toss them and return frame 14 to you, without any warning or indication about it.

Finally, when 3 above applies the TFF/RFF flags in the D2V file are misaligned to the frames.

Specifically, the part about mpeg2dec3 subtracting 2 from the d2v frame count as a workaround for 3. You might be able to work around this using tfm's trimIn() option, but I wouldn't trust anything other than flags=3 (which doesn't require matching frame counts) with d2v files generated by 1.77.3. Especially due to the tff/rff flag to frame misalignment problem. I'm with foxyshadis, why use dvd2avi 1.77.3+mpeg2dec3?

[ChiDragon]

Well I use TDecimate's display mode with TFM's hints to see what both filters are doing, but I guess I understand if keeping them in would interfere with the second decimation.