TDeint and TIVTC

[tritical]

Well I made a new motion adaptive deinterlacer/smart bobber called TDeint. The main three reasons for its existence are it is bi-directionally motion adaptive... meaning if you have the following field and pixel sequence, where '-' is the pixel position to be filled:

a d g j m
c f - i l
b e h k n

Then TDeint will try either weaving the average of f and i, or simply f or i based on the motion in the surrounding pixels. Afaik, the only deinterlacer that currently does this is scharfis_brain's intellibob. All other motion adaptive deinterlacers currently only weave whatever pixel gets weaved into the current frame by default. That means that all pixels will be detected as moving in the first frame after a scene change. (If I am wrong about this and another deinterlacer already does this please tell me!).

Second, this filter is using a new form of modified ELA interpolation that I have been working on recently. It takes into account the direction of the normal line to the gradient vector when deciding in which direction to average, and attempts to tune the interpolation so that it can adapt to any edge direction within its range. In terms of quality, it is better then plain 5+5 tap ela as in tomsmocomp, but not quite as good as some other techniques such as NEDI or edge post processing as in antialiasing that sangnom uses. The nice thing about this method, though, is that it is still pretty fast (running about 30fps in dumbbob mode on 720x480 images on my comp). For comparison, here are a few images of a simple dumbbob (no motion adaptation), the first was made using this modified form of ELA, the second was made using tomsmocomp(-1,-1,0) (5+5 tap ela w/ clipping), the third was made using AviSynth's built in bob which uses cubic interpolation, the fourth was made using sangnom(aa=0), and the last was made using sangnom().

modified ELA (http://bengal.missouri.edu/~kes25c/modifiedELA.PNG)
tomsmocomp(-1,-1,0) (http://bengal.missouri.edu/~kes25c/tomsmocomp.PNG)
AviSynth's Bob (cubic) (http://bengal.missouri.edu/~kes25c/cubic.PNG)
sangnom(aa=0) (http://bengal.missouri.edu/~kes25c/sangnom0.PNG)
sangnom() (http://bengal.missouri.edu/~kes25c/sangnom.PNG)

Third and finally, it has support for user overrides. Not only allowing the user to set which frames should be deinterlaced as fielddeinterlace() does, but also allowing the meaningful parameters to be adjusted for single frames or ranges of frames. Also, when working as a same frame rate deinterlacer it allows the user to control which field is kept and which field is interpolated.

Anyways, the reason I posted this in the development forum was that I don't really have much if any true interlaced material to test with, aside from what I can capture here in my dorm room. Thing is, we have about the worst cable reception known to man so its hard to tell if a deinterlacer is any good. I was hoping someone with some true interlaced material would be inclined to test it a little or someone could send me a few interlaced clips to test with to make sure everything works alright. It does seem to work great as a per pixel field matcher, as I have only been testing it with telecined material from dvds. With default settings the filter runs around 45-55fps on my 2.2Ghz p4.

TDeint v1.1 (http://bengal.missouri.edu/~kes25c/TDeintv11.zip)

TIVTC v1.0.5 (http://bengal.missouri.edu/~kes25c/TIVTCv105.zip)

EEDI2 v0.9.2 (http://bengal.missouri.edu/~kes25c/EEDI2v092.zip)

TMM v1.0 (http://bengal.missouri.edu/~kes25c/TMMv1.zip)

avi_tc_package v1.5 (http://bengal.missouri.edu/~kes25c/avi_tc_package.zip)

If you're looking for older versions try: old_stuff (http://bengal.missouri.edu/~kes25c/old_stuff/)

[scharfis_brain]

in mode=1 it only returns the half number of frames of the video. the framecount is the one of the bobbed video, but it stops moving after the half of the length.

could you make it optional, whether, both fields are weaved or only one of both, even if low motion allows it?

btw. great work!

I will test it more intensive tomorrow.
I was hoping for such plugin for along time :)

[tritical]

Sorry about that, stupid bounds checking, really not sure how I missed that one when I was testing it :o. There was also a problem with one of the motion checks being incorrect (not getting set to false) on the very first and very last frame of a video. Hopefully, there isn't anything else wrong... I actually tested it and looked over the code quite a bit before I released it, so much for that :).

Yeah, I can add the option to have it only weave from previous or only weave from next... I had tried to keep the # of options as low as possible for this filter as compared to some of my other filters.

[Mug Funky]

wow, this is a really f***ing good bobber.

i've gotta go rip one of my field blended animes... i might finally have something to give it that (a) doesn't make odd block artefacts (intellibob is great, but i get strange bits unpredictably so i can't use it for serious stuff), and (b) is fast enough.

hmm... i've got some PAL interlaced DVD crap if you want to see it. interlaced fade-ins and outs cause hell with anything, but it might be worth seeing how this performs anyway.

will post a clip in a few mins (what's the max length of a sample i can post to avoid being flayed alive by the MPAA? i doubt the publishers of this DVD really care, but i'd like to keep things legit)

[edit]

here's 20 sec or so from the start of John Safran's Music Jamboree. i left the sound in cause it's pretty funny...

rightclicksavetargetas, or you'll end up with plain text garbage (at least i did when i tested it).

if it's dead, tell me because i'm new to apache...

http://210.49.108.136:8080/safran.vob

14.4MB

[edit edit]

the default thresholds are a little too high for field-blended anime - it gets chunky very fast. here's a sample of the Chobits intro, r4 pal (field blends up the whaazoo)

http://210.49.108.136:8080/chobits_intro_chunk.m2v

hope these are enough to start tweakage...

by the way, i'm hosting these on my machine, so the links will go dead when i go to bed...

[edit 3]

changed URLs in the hope that they will work properly...

[tritical]

@Mug Funky
Just tried the links and they didn't work... but maybe your comp is still off? I would very much like to run a few tests on those samples.

@scharfis_brain
When you said make it optional if both fields are weaved, do you mean make it so that it can be set to only be motion adaptive in one direction? or that it should not use the avg() of the two center pixels but just one or the other?

As for the filter, I'm implementing another mode for motion detection that requires a 5th field worth of no motion in order to declare a pixel position stable. It will be slightly slower, but should help eliminate any of the stray artifacts from the usual 4 field motion check. Have to see how it performs...

[scharfis_brain]

forget about that anti-averaging request.

but it would be good IMO, if the filter decides for the whole plane, whether the next or prev field is used, instead of doing this pixelwise.
I ask for this, cause fieldblended video always swaps its phase, so the pixel of the wrong-phased (non-fitting) field would not be relevant (the sum up to less than 1% of the whole weaved pixels).

the next thing I would think, that it would be good, is kernel-interpolation.

it is not as jaggy-free as edge-directed interpolation, but it produces a much more stable (lesser flickering) image in bobbed areas, cause my intellibob uses a one-way kernel, that is turned to fwd or bwd direction to fit the in-phase-field. (but I think, this would be hard to implement)

EDIT:

my very raw intellibob() replacement with hardmatching, when fields are near to equal:

function matchbob(clip c, int "combth", bool "show")
{
show = default(show, false)
global cth = default(combth,6)

ord = c.getparity() ? 1 : 0
odd = c.doubleweave().selectodd().telecide(order=1-ord,post=0)
even = c.telecide(order=ord,post=0)
tel = interleave(even,odd).duplicateframe(0)

analyse = tel.greyscale()

tel = show ? tel.subtitle("fieldmatched") : tel

bobbed = c.TDeint(mode=1, order=ord, mthreshL=8, mthreshC=6,type=2)
bobbed = show ? bobbed.subtitle("bobbed") : bobbed

conditionalfilter(analyse,tel,bobbed,"iscombed(cth)","==","false")
}

[tritical]

k, I wont do the non-avg option. On the kernel interpolation I will definitely try it... probably the one-way and one-way sharp kernels from kerneldeint. An easy way to switch between using the prev or next field in the kernel would be to simply subtract each field from the curr field and use whichever one is most similar. Its not super complex, but should be a little better then always using the previous field. The per field weaving instead of pixelwise weaving I think I'll leave alone for now because it would effectively make it a field matcher and probably a rather poor one :D. Atm, this filter is more intended for true interlaced material or material where you can't do pure field matching as opposed to a combination of each like your matchbob function.

[Chainmax]

So, this filter followed by a decimation results in IVTC? In any case, I could provide you with some samples of the "Spawn 1º Season" DVD if you're interested.

[Mug Funky]

@ tritical:

sorry... looking at the time you posted, it was 4:38 AM my time.

i'll leave my machine on tonight. i just hope apache is behaving itself... it works when i click the links myself, but if there's firewalls and stuff happening i might have to set up one of those port redirect things.

post again if you can't get them and i'll see what i can do.

[tritical]

@Mug Funky
Just tried them an they wouldn't work :(. Internet explorer just says it can't establish a connection to the server.

I posted v0.9.2, I added kernel interpolation... the one way sharp and normal kernels from kerneldeint. It switches between using the forward or backward field based on which one is most similar to the current field. In terms of flickering it was definitely a large improvement. I also added the 5 field motion check, well... half way added it. I got the extra 2 field checks in there to make it 5 field, but didn't implement any of the 4 field checks I was planning and which would be the most effective with fades. The other thing about the 5 field check, is that it can deal with having duplicated interlaced frames in the video where the 4 field can't. Anyways, still have things to test.

[Mug Funky]

try again :)

just changed the links.

i wish i had 2 computers working on different lines so i could test these things myself... the only current test i have is to give people links over instant messenger progs and ask if they worked...

[edit]
just got a friend to test it on his machine and the link seems to work fine. i suppose any other problems will be from firewalls blocking port 80 and 8080.

[edit edit]

oh, and Chainmax:

in theory it's a simplified IVTC on the pixel level, but in reality it'll deinterlace parts that can be IVTC'd. it works spectacularly in conjunction with BlendBob though - it handles the black orphan field in the Eva renewal NTSC intro sequence perfectly! no blurring or anything.

so i'd say that Tdeint bobbing + blendbob + decimate will get you a pretty good IVTC. i'd prefer to use it as a fallback for that though.

my main interest is in pure interlaced and field-blended stuff.

Tritical: assuming the links work now, would you like some other samples? i've got a lot of crap DVDs lying around, and a small amount of home-shot DV footage.

[Chainmax]

I repeat my offer to post some samples from my Spawn DVD then.

[tritical]

@Mug Funky
Yeah it worked this morning, I had to leave before they finished dling but they should be done. I have enough samples to mess with for now... was searching through some of the assorted clips I have and found a small vob piece of ranma ep 1 which is quite horrible and has blended fields :).

@Chainmax
Thanks for the offer, but I have plenty of plain dvd type material... I was mainly interested in pure interlaced samples which I have none of other then a few very bad tv caps. About the other stuff... like Mug Funky said you could do an ivtc by following TDeint with decimate()... which would work with any deinterlacer. It would not be nearly as good as using a pure field matcher + decimate though if your video can be cleanly ivtced. This filter is just a pure and simple per-pixel, motion adaptive, deinterlacer as in kerneldeint, smoothdeinterlace, alignfields, etc... the main difference being that TDeint detects motion both backwards and forwards instead of just backwards.

[scharfis_brain]

hey trictical!

great work!

now I have the ultimative bobber for my fieldblended crap.

many thanks.

edit:

and again: a request.

could you knock out the order-parameter?
tdeint should be able to detect the correct fieldorder itself by reading the avisynth fieldorder flags, which are set using AssumeTFF() and AssumeBFF().

[Chainmax]

tritical, the Spawn DVDs can be called everything but plain. Quoting Gizmotech:

I have seen a vob sample of this cartoon from a friend.

The series contains real interlacing, as well as full frame blending (in spots).

You can read more about it in this thread (http://forum.doom9.org/showthread.php?s=&threadid=70353&highlight=Spawn) (the clips posted in there are no longer online, BTW).


Anyway, I'm going to try TDeint's bobber along with BlendBob as soon as I can and report back.

[joshbm]

I downloaded your TDeinterlacer. Gives me an error message and I can't open VirtualDubMod anymore! How do I fix that?


AVISource("G:\0829 TV.avi")
Trim(94104,-389)

converttoyv12()
TDeint(order=FOrder())
converttoyuy2()

[Mug Funky]

you need to define Forder...

try it with just "order=1"

[edit]

oh, if you're using DV as a source, you should go "order=0" i think.

or use this to determine it:

#assumetff()
assumebff()

order = (last.getparity==true)? 1 : 0

tdeint(order=order)

unless you're making a customized function, there's probably not much need for all that, and a simple 1 or 0 would suffice.

[joshbm]

It works now lol. It was because I was working on a function and I didn't debug it yet and it contained errors. It was in the plugin folder and was trying to be loaded into VDubMod and AviSynth. Didn't work too well :p.

Thanks Though!
Josh

[tritical]

I will make the filter use the internal parity if the user does not set the order parameter, and also add the option to read telecide's hints and set the field to 0 if the user does not specify it since telecide matches off the bottom field.

Been busy the last few days, but I should get a new version up this weekend with the above options, and a completed 5 field motion check which includes the 4 field distance checks to help with fades and motion adaptive related artifacts in general. That will of course make it even slower then it already is :D.

@Chainmax
Didn't realize it was so screwed up ;). If you do test out TDeint on it I would be interested in the results, but I'm not sure it would be much better then anything else if its a really bad source.

[Chainmax]

I am uploading a 17MB Xvid clip. Here's the script I used on it:

TDeint(mode=1,order=1,mtnmode=1)
BlendBob()
Decimate(cycle=5,mode=2,quality=3)
Crop(8,0,706,480,align=true)
LanczosResize(640,480)

Once it's finished uploading, you can get it from here. If you want, I can also make some vob samples available as well.

[tritical]

Put up version 0.9.3, changes:

+ Added order = -1 option, will detect parity from avisynth (this is now the default)
+ Added hints option for reading telecide hints for interlaced/progressive (use hints=true,post=1 in telecide)
+ 5 field motion check now includes checks over 4 field distances
- Fixed a bug in YUY2 type = 1 deinterlacing

Also, after testing on the sources Mug Funky sent, I would not recommend using type=1 (modified ELA) on non anime/cartoon material. Either go with kernel or cubic interpolation for natural sources. I will probably change the default in the next version.

@Chainmax
I'll take a look at the clip and see just how bad it is :p.

[Didée]

Originally posted by tritical
after testing [...] I would not recommend using type=1 (modified ELA) on non anime/cartoon material.
Some description about why you wouldn't recommend that?

Actually, I'm working on a Live recording (DVB source) where the upper fields are just plain unusable - they are fieldblends of the shifted bottom fields (!). The experts from television :devil:
So, I've just one field for upscaling available. After trying out lots of things, I got the best results on jaggy-ness reduction, shimmer reduction and keeping things as sharp as possible with
TDeint(mode=1).TomsMoComp(1,-1,1).LanczosResize().
That's the best base filtering I could come up with so far.

(And BTW, it's astonishing how good a 464*208 source can look at 704*512, with some dedicated slow filtering ;) )

[scharfis_brain]

didée: for you purpose (completely missing information of one field) the ELA is the best way to go.

but for bobbed deinterlacing of natural video, the ELA produces more flickering (but lesser jaggy diagonals) than the kernel interpolation method.

this is caused due to the nature of the kernel: it just has some information from the other fields. the ELA in opposite works blind.

[
I've done something, too some months ago. the television-experts sometimes knock out one of the fields and interpolate this field by line averaging (like shifting the remaining field by 1/2 pixel).
They seem intend to get the film-look, but I say, they produce a "pop-sta(i)r-look".
for exactly this scenario I wrote a function, that detects this kind of pseudo progressive and interpolates those blurred fields using tmc(-1,-1,0). video-overlays, that are interlaced/true progressive are not touched using my function.
]

[erratic]

I compared the speed of TDeint(mode=1,order=1,type=2) with Leak's KernelBob(order=1,sharp=true) and KernelBob was nearly 4 times faster. The source was normally interlaced PAL (not telecined stuff). Is there any way to make TDeint as fast as KernelBob? Or should I just use KernelBob for normally interlaced stuff and TDeint for fieldblended crap (as scharfis_brain calls it)?

[Leak]

Originally posted by erratic
Is there any way to make TDeint as fast as KernelBob?

Well, a quick peek at Tritical's sources shows that he's using pure C++ while my implementation of KernelBob uses MMX to speed things up; I'm not sure if you'll get a 4x speedup out of using MMX here (didn't look at the algorithm closely enough), but it sure would make things faster.

Then again, I don't have the intention to MMXise yet another deinterlacer in the near future, so somebody else would have to do it...

np: Arovane - Cry Osaka Cry (Lilies)

[tritical]

@erratic
There is one thing I can say with absolute certainty... it will never be as fast as Leak's MMX optimized kernel deint :( even if it was MMX optimized. Its simply a matter of complexity... in terms of the motion checking (which is substantially more involved then kernel deint's), weaving from either the prev or next field (goes back to the motion checking), the motion map denoising (though it is pretty weak atm), the switching of the one way kernel between forwards and backwards, etc... There is also the fact that I pretty much refuse to code or even try to make MMX/iSSE versions of things anymore... mainly out of laziness :) and not having the time, but also because it tends to limit how much you can test new ideas since a major algo change can cause you to have to rewrite all that assembly code. I am still testing/changing quite a bit of the main code atm... I may look into doing optimizations for some functions when things get more stable... but don't count on it. I'm actually surprised it was only 4x slower... try mtnmode=1, maybe we can get it 8-10x slower :devil:.

@Didee
I probably generalized that statement too much. Like every edge adaptive interpolation scheme this one suffers from problems/artifacts in very detailed areas with multiple edges/lines. Usually anime does not have as much fine detail like this compared to movies, etc... except for kanji and some other things off course. So I'll take back what I said and simply say that if ELA interpolation is causing a lot of artifacts try using kernel or cubic interpolation instead. Kernel interpolation is definitely much better when it comes to flickering in almost static areas, this can even be noticeable when doing same framerate deinterlacing on noisy sources.

@Chainmax
Aside from being noisy I didn't think the sample looked that bad, it was better then what I was expecting. In terms of getting progressive frames I thought it was pretty good for how it was decribed in the other thread.

[Chainmax]

Well, the description in the thread corresponds to the (unfiltered) vob samples. The clip you downloaded was compressed using the script I posted earlier. That's why I offered to post some vob samples so that you could make a comparison.
And yeah, the results are very impressive. Did you see the scenes with close-ups of Spawn's face? In all the other methods I tried, the eyes showed a lot of jaggyness. TDeint is the only filter that almost eliminated that :). I think I have finally found an adequate script for ripping this DVD (just need to throw some deen at it):) :).

[joshbm]

Originally posted by Didée
Some description about why you wouldn't recommend that?

Actually, I'm working on a Live recording (DVB source) where the upper fields are just plain unusable - they are fieldblends of the shifted bottom fields (!). The experts from television :devil:
So, I've just one field for upscaling available. After trying out lots of things, I got the best results on jaggy-ness reduction, shimmer reduction and keeping things as sharp as possible with
TDeint(mode=1).TomsMoComp(1,-1,1).LanczosResize().
That's the best base filtering I could come up with so far.

(And BTW, it's astonishing how good a 464*208 source can look at 704*512, with some dedicated slow filtering ;) )

I attempted your TDeint(mode=1).TomsMoComp(1,-1,1).LanczosResize(). It said that the video framerate's did not match. Except when I set the mode=0. BUT I want to bob my video to get double the framerate, so this does not help. What am I doing wrong here?

Thanks!
Josh

[Didée]

Aaargh. I swapped parameters when recalling from my (poor) memory.
Of course I used TDeint(mode=0, field=1), since there is nothing in my stream that could be bobbed ... and the blends are in the bottom's, not in the top's. *Cough*.

Joshbm, I assume the framerate mismatching you're experiencing comes from the one-line-script with bobbing. Try to separate it:

TDeint(...)
TomsMoComp(1,-1,1)
LanczosResize(...)

Although TMC should not bother, since with (1,-1,1) it does spatial-only interpolation without field matching ... but you never know. There are worse things in life.

***

However, with spatial-only interpolation, I never got anything out of any deinterlacer that *really* satisfied me. At least not in "directly" applying the filter. There's always way too much jaggyness left for my taste - though it gets (much) better if one first blows up the fields to double the height.

So, here's another thought that's wandering through my mind for quite some time now. It's about spatial-only interpolation of missing fields, to avoid jaggyness. Perhaps this is already sort-of-implemented in any filter, perhaps it's not. I just don't know...

I think that all methods so far start out to work on each single pixel, and then examine a more-or-less small area around the current pixel to gather information - be it 5x5 tap, modified or unmodified ELA, or whatever.
Well, as far as jaggyness is concerned, it is only an issue on edges close to "horizontal". Starting with edges having a slope of "2", like

____XX
__XX__
XX____

the problem already starts to go away, and on 45° edges a simple resize() works near-to-perfect. So, what we do have to conquer are only the edges "below" 45°.

Now, what about ... a "scanline search"? A filter that searches 2 complete scanlines at a time, finds the correlating areas between them, and interpolates accordingly?
Searching (sort of "parsing", in this respect) basicly would enable something like the following, relatively easy:


..........XXXXXXXXXXXXXXXXXX...........
....::::::.::::::::::::::::.:::::::....
....XXXXXXX................XXXXXXXX....

..........XXXXXXXXXXXXXXXXXX...........
.......xxxoxxxx........xxxxoxxx........
....XXXXXXX................XXXXXXXX....

o = found area match between scanlines
: = not-matching area of a "matched feature"
x = interpolation around matches

where the "horizontal length" of the interpolation is based on the length of the not-matching area. Could be 1/3 of the not-matching area in "dumb mode", or could be adaptive based on the scanlines +2 and -2, if it can be afforded to include even these into evaluation.

For areas, or "features", that have no "matching" on one side or the other, a simple (lanczos) resize would be appropriate, then.

Now, that's draft, crude & basic, and needs of course some more comparing & computing. But basically, it seems not too unreasonable to me. Or is that already something like an "ultra-primitive ELA"?

I even tried to script something in that direction ... but you now, *directional* searching is almost impossible in an avisynth script. :/

[joshbm]

@Didée:

Weird, it still doesn't work... lol.. but this works:

TDeint(...).AssumeFps(last.framerate).TomsMoComp(1,-1,1).LanczosResize(...)

Whatever works :p.

Thanks!
Josh

[tritical]

@Didee
Your idea is very similar to plain n+n tap ela, though your probably thinking of a slightly larger scale. Plain 5+5 tap ela works like this (where o is the pixel to interpolate):

a b c d e
. . o . .
f g h i j

Take the absolute value of the differences of pairs (a,j), (b,i), (c,h), (d,g), and (e,f). Then simply average the two pixels that have the lowest difference and use that for o. This is exactly what tomsmocomp uses, except that it clips the calculated value into the range of +-2 or 3 of min and max of c and h. The reason being that this method is insanely prone to artifacts since the minimum difference can easily have nothing to do with edge direction. i.e. the edge runs through (b,i), but (f,e) has a minimum value. etc... However, this clipping eliminates a lot of the benefits of the algorithm since on most edges interpolated values need to be outside this range. It is easy to see why this method can't be used to search more then a few pixels away.

DCDi (directional correlational deinterlacing), which is very similar, uses the same technique but uses horizontal vectors of 2 pixels when doing the minimum difference comparisions and also does a couple of extra pixel difference comparisons to make sure that the direction is along that of the edge. This helps to reduce artifacts somewhat and gets closer to what you are probably thinking of. You can expand on that idea and use 3 or 5 pixel horizontal vectors to search for corresponding areas in the above and below scan lines. This works well except when you start expanding the search area to say 8-10 or more pixels to either side. If there is only one line in the surrounding area then your probably ok, but in detailed areas you'll get all kinds of mismatches. I've actually tried out this exact idea... using 5 pixel groups to search the above and below scan lines while limiting the search to edge areas only) it works well in areas with only one major edge. Another problem is that both of the above algorithms are limited to a small set of angles to which they can accurately adapt, since they use linear interpolation with only one pixel from each of the scan lines.

Now, what TDeint uses and what I've been calling modified ELA, I should probably just call edge-directed interpolation since it does exactly that. It doesn't do any direct pixel comparisons at all. It literally finds the direction of the gradient vector via the local partial derivatives of the surrounding pixels in the above and below scan lines and then rotates it 90 degrees to get the isophote direction or direction of least change. It also calculates gradient magnitude and local variance and drops back to linear interpolation in non-edge areas. Once it has the direction, it then uses linear interpolation on the above and below scan lines to obtain the points at which a line passing through point o with the calculated direction would intersect each line and then averages those values together. The main problem with this method is it will almost always mess up in detailed areas where two lines intersect, since the gradient vector will most likely end up pointing in the isophote direction. However, I think I have finally come up with a simple way to fix this and eliminate some spurious directions due to noise. In general I've found that this method usually surpasses that of pixel matching/searching. However, since it calculates the derivatives in the x direction in a small area (only over 3 pixels) it can't accurately detect the direction of nearly horizontal edges (it simply says they are purely horizontal). Atm, I allow it to use directions that would go out to 4 pixels to either side, which corresponds to rougly 15 degrees. However, it is really only good for edges with around 25-30 degrees slope or more. I'm currently working on improving this method around areas where lines intersect, and an obvious improvement would be to calculate the directions for the 4 other surrounding pixels which would give a better indication of the local geometry. Handling edges with smaller slopes could also be done by calculating approximations of the local partial derivatives over a wider area.

So your method would be pretty similar to DCDi or plain ELA, though with a slightly enlarged search area and possibly search group, maybe 3-5 pixels instead of 1. TDeint's method isn't really that similar to ELA except for the fact that it still does the interpolation using pixels that correspond to the intersection points instead of computing the final value by tuning the coefficients of the neighbor pixels as in NEDI or some other methods. I have been looking into edge directed methods of interpolation for deinterlacing (read... faster then NEDI) quite a bit recently and am quite interested in any ideas anyone has...

[MfA]

Does DCDi do a lerp between the 2 horizontal pixels before doing the average between the 2 lines or something? ELA with a wider aperture doesnt necessarily have to pick 2 opposing pixels to average straightaway ... you could first find the best match between two opposing X pixel vectors, and then do subpixel search. Just a couple more lerps.

BTW you might want to try local centered moments to determine isophote direction (can be implemented with running averages, so the amount of operations per pixel is pretty low ... it is probably cheaper than using DoG convolution masks greater than 3x3 for getting the gradient).

[Didée]

Thanks for the thorough explanation, tritical :) - It's much clearer now.

Well, the words of my description probably were a bad expression of my thoughts. What I was thinking of is, in some way, more like analog signal processing (better, a digital simulation of that). A "tension" between the scanlines, that "tears" everything to the right places ... dunno how to explain it better.

Perhaps a comparison to "the travelling salesman problem" is appropriate. You can throw a supercomputer at that and let it calculate hours and days. Or you can get the result within just some moments from a dedicated analog system that more or less lets the problem optimize itself (any given "start" routing modifies itself by moving towards a routing with lower overall tension).
However, the trick of analog systems is that they are working - analog ... :|

[tritical]

@MfA
Your right, DCDi first averages the two pixels from each line together and then averages those values together... I had forgotten. And sub-pixel search for ela with larger apertures is possible as well. Image moments I hadn't really thought about, mainly cause I don't know much about the subject. However, I'm not actually planning to use larger then 3x3 masks for the derivatives, it's just a possibility. Also, in the normal deinterlacing case, the derivatives are only calculated for about 15-20% of the pixels to begin with since most are handled by the motion adaptation or are within +-3 of both the above and below pixel in which case I just leave it. So the advantage to being able to use running averages wouldn't be that much.

@Didee
I see what your saying, there is probably a good way to do that but I can't think of one. At least not one that could work for lines with both very small and very large slopes without dynamically adjusting the aperture.


@All
I posted version 0.9.4. I haven't been able to work on this much the last week, but here are the changes:

+ Added auto detection of hints (if you don't set a value for hints manually, then TDeint checks the first frame to determine if hints are present and sets the value accordingly)

+ Added mtnmodes 2 and 3... these accomplish what I think scharfis_brain was suggesting earlier in the thread. They act like 0 and 1, but wherever 0 and 1 would have used the avg() of two pixels, 2 and 3 instead use the pixel value from the field that is most similar to the current field.

+ Added clip2 parameter. When using TDeint as a postprocessor for telecide you can get weird results since telecide changes the order of the fields (that's not good for a motion adaptive deinterlacer). So you can specify clip2 for the actual deinterlacing to be done from... here's an example of how it works:

mpeg2source("c:\mysource.d2v")
orig = last
telecide(guide=1,order=1,post=1)
tdeint(order=1,clip2=orig)

Then TDeint reads the hints and output from telecide as usual, but whenever a frame needs deinterlacing it does it from clip2. It also preserves the hints in case any filters later on need to read them. If you don't specify a clip for clip2 then the deinterlacing is done from the input clip as usual.

- Fixed field differencing in kernel interpolation using the wrong fields, and not correctly adjusting the direction of the kernel to that of the field that is most similar to the current field.


I'm still planning on improving the edge directed interpolation in TDeint. Also plan on improving the motion map denoising and adding built in combed frame detection.

edit:
Posted v0.9.5, right after posting 0.9.4 realized that I was doing mtnmodes 2 and 3 the hard way instead of the simple, faster, and blatantly obvious way. So now 2 and 3 aren't any slower then their 0 and 1 forms.

[tritical]

Well, either no one has had any problems or no one uses it :).

I just posted v0.9.6, haven't had time to work on improving the edge-directed interpolation, but I did manage to add built-in combed frame detection and per-field weaving.


Changes from v0.9.5:

+ Added full parameter, allows for ivtc post-processing. full defaults to true.
+ Added cthresh, chroma, and MI parameters... these are used when full=false and with tryWeave option
+ Added tryWeave option, allows TDeint to adaptively switch between per-field and per-pixel motion adaptation. tryWeave defaults to true.
+ Improved field differencing
+ changed mtnmode default to 1


Some explanations...

Full works the same way as the full parameter in fielddeinterlace(). If set to false, then all input frames are first checked to see if they are combed. If a frame isn't combed, then it is returned as is. If it is combed, then the frame is processed as normal. Full can only be used when mode = 0. Full defaults to true.

tryWeave, if set to true, works like this... The most similar field to the current field (either prev or next) is calculated. A new frame is then made by weaving this field with the current field. This new frame is then checked for combing. If it isn't combed then it is returned. If it is combed, then normal processing (as if tryWeave=false) is done. This allows TDeint to adaptively switch between per-pixel and per-field weaving. This idea is taken from scharfis_brain's suggestion earlier in the thread and his matchbob() function. tryWeave defaults to true.

[Mug Funky]

you rock.

...that's all i've got to say until i test this a little further.

[edit]

shit, man, you REALLY rock. i barely need IVTC anymore now :):):):)

[edit 2]

seems to be some odd field-order problems with the new matchbobbing... perhaps when it does the weaving it doesn't re-set the original field-order? i'm getting havoc with the intro sequence to Lain, which has a mix of interlaced, 30p and telecine.

i suspect some of the interlaced stuff off this disc to be b0rk field order to begin with (there's bits shot on DV cam in the show where the field-order is obviously wrong, but it was probably an effect, as it's that kind of show). however, assumeTFF().bob() is returning different stuff from tdeint(1,1).

i can post a sample if you like.

[tritical]

Yeah, if its not too much trouble a sample would be helpful. On normal material I can't produce the problem you described. I do have some weird clips that have what I would call "out-of-order" fields every once in a while, that might be what your clip suffers from. Even when the field order is set correctly, if you do a separatefields() and step through a field at a time it will jump backwards on some fields. This will cause major problems for any motion adaptive deinterlacing approach. If there are entire sections of the clip that have a different field order from the rest of the clip that will obviously produce garbage output as well. Even on this type of input though, TDeint should preserve the same field as bob().

Providing a work around for clips that have single out-of-order fields or weird stuff in general would not be that difficult, just need to test the output frame to see if it is combed and if it is, deinterlace it w/o motion detection. That isn't ideal though since it would end up detecting some good output frames as well. If a clip has whole sections with a different field order that is a tougher problem, probably have to just process it in sections...

[pdottz]

I read the readme file but can't understand it to clearly
what would be the settings to use for ntsc captured tv? i cap at 720x480. this is the script i use.


LoadPlugin("C:\PROGRA~1\GORDIA~1\TomsMoComp.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\undot.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\FluxSmooth.dll")
AVISource("D:\captures\maincap\eene.avi")
TomsMoComp(1,5,1)
Crop(0,0,720,480)
LanczosResize(512,384)
Undot()
FluxSmooth(7,7)
ConvertToRGB()



tomsmocomp does an awesome job but leave me with some aliasied line that are clearly noticeable on the capped toons.
i want to try this new filter and would look to try it on this file.
can anyone help?

[Mug Funky]

hmm... looks like my bad. i tracked the problem down to MPEGsource rather than MPEG2source. looks like there's some oddness in the repeat-field flag handling, causing chaotically reversed fields.

everything works fine using DGdecode. (pity, because every little bit of speed is useful to me, so the very small decoding speed difference of nic's mod was useful to me).

here's the sample anyway, if you want it - it contains some good testing material besides the field-order oddness that turned out not to be a bug in your plugin (sorry... but i'll let nic know about this).

http://210.49.108.136:8080/lain_intro.m2v

[xappy]

Hi!

Thank you for the great plug-in. I have used it for bobbing my interlaced MiniDV PAL material before creating amazing slow motions ;)

The plug-in (0.9.6) still produces some very disturbing bobbing artefacts. I know the reason, why artefacts are produced and I can send you some images about it. There is one havily compressed image as an attachment.

I was thinking to make some modifications to your code, but when I tried to compile your source code without any modifications I got the following errors:

--------------------Configuration: TDeint - Win32 Release--------------------
Compiling...
TDeinterlaceYUY2.cpp
E:\TDeint\TDeinterlace.h(199) : error C2065: '_aligned_malloc' : undeclared identifier
E:\TDeint\TDeinterlace.h(500) : error C2065: '_aligned_free' : undeclared identifier
TDeinterlaceYV12.cpp
E:\TDeint\TDeinterlace.h(199) : error C2065: '_aligned_malloc' : undeclared identifier
E:\TDeint\TDeinterlace.h(500) : error C2065: '_aligned_free' : undeclared identifier
Error executing cl.exe.

TDeint.dll - 4 error(s), 0 warning(s)

What am I missing?

[Wilbert]

There is one havily compressed image as an attachment.
You should be able to post attachments again. But I don't see any?

[tritical]

@xappy
Hm, afaik it should compile fine as long as you have malloc.h included, and since it didn't say it couldn't find it I'm not sure, but I'm no expert. You could go ahead and change the _aligned_malloc and _aligned_free to just plain malloc and free, won't make much difference, or use new and delete instead like I probably should :). If you know some obvious improvements to make I'd be grateful if you would share them... the few things I've attempted to use as a post-processing to clean up left over bob artifacts do work, but also end up messing with good frames more then i would like. Either setting an ap threshold as in dgbob or testing the output frame for combing and then dumbbob deinterlacing it or storing the previous output frame from TDeint and using that to do motion adaptive deinterlacing from.

@Mug Funky
I'll still try to grab the sample if you don't mind... problem is I'm never at my own comp when you seem to have yours up and running.

[Mug Funky]

it's up now. i'm installing some software and have had to reboot a few times, but i'll refrain from that for a while.

think i'll leave this box on for a good day or so, hopefully that will give you time to d/l it.

[xappy]

@Wilbert
I tried to send the image again, but got the following error:

You have already attached this image to a post. Please refrain from uploading duplicate images. If you need to refer to this image it can be found in this thread: Another motion adaptive deinterlacer and smart bobber

@tritical
_aligned_malloc and _aligned_free do not exist in my malloc.h. Should I install some SDK or some other package? I'm using MS VC++ 6.0, but I'm totally newbie with it. However, I managed to compile some other filters...

Can I send you some (small ~10kB) pictures about those artefacts I mentioned before to your email seen in your home page?

[tritical]

@Mug Funky
Finally getting that sample, thanks :).

@xappy
An email is fine... though I can see the attachment now. It would actually be easier if I could get a small sample of the video. Just looking at the pictures leaves a lot of guessing, but I'm fairly certain what caused the artifacts in the attached image. With VC++ 6.0 I think you need the processor pack to use _aligned_malloc if you don't have it. Should have mentioned that before, but I always forget that not everyone uses the same compiler I do :D.

[lordreign]

Hey Mug Funky, I'm just curious as to settings u use with TDeint... so I could perhaps see if they work for some of my anime. Bad field blended PAL anime that is.

Cheers.

[Mug Funky]

sure thing... i rarely do anything other than Tdeint(1,1) for top-first bobbing.

i'll only really tweak it when i get too much coming left over - i'll set Cthresh to 5 or 4 (default is 6), and/or MI to about 20 (default 32). this is only when i get combs left over, as it also risks getting shimmer on good frames.

[edit]

the last few anime encodes i did were disc 1 and 2 of Azumanga Daioh. i just left it bobbed and at 50fps, and let Xvid sort it out.

512x384, lanczos4resize, limited removedirt (just to stop removedirt from removing actual motion - i'll post the script in a jiffy.. it's probably sub-optimal, but doesn't hurt things) and encoded with CQ 3, and 4 b-frames instead of 2 (i figure 50fps has twice the redundancy, so i could get away with it). all other settings are at default.

episodes are coming out at around the 110MB mark (sans sound) which is quite acceptable to me.


by the way, speaking of Azumanga Daioh, you should see the rating disc 1 gets... the stupid australian office of film and literature classification gave it an MA 15+ rating... disc 2 is a G. that puts a _very_ tame anime on the same rating as Kill Bill 2, which is utterly stupid. however, it costs Madman several thousand dollars to have these discs re-assessed.

[Chainmax]

tritical: I finally was able to encode the clip I promised in my "need help IVTCing..." thread. You can download it here. I'd appreciate it if you could comment on the image quality of it besides assessing TDeint's work as a fieldmatcher.

Have you considered posting this thread in Avisynth usage? I bet more people would see it in there than in here.

[tritical]

I'd appreciate it if you could comment on the image quality of it besides assessing TDeint's work as a fieldmatcher.
I'm not sure exactly what you mean, but after looking at the clip it seemed like all the fields had been matched correctly where possible and the ones that couldn't be were deinterlaced (the fades). On the subjective side, I thought it looked pretty good considering the source :D.

As for posting it in usage, I plan too, but I still haven't been able to find a good method of post-processing to remove left over artifacts as in xappy's picture. In the end, I think I may simply add an optional artifact threshold as in dgbob and be done with it. The major problem is that those type of artifacts are inherent to motion adaptive deinterlacing/bobbing, and detecting them will also detect lines/edges pretty much mitigating the benefits of motion adaptation. I've also been working on a new method of edge-directed interpolation that is considerably better then TDeint's current method, but haven't gotten all the kinks worked out just yet. I would like to get the above two items finished and added before posting it in usage.

On a side note about field matching, the blind field matching of TDeint is actually better then that of Telecide, especially when it comes to the small movement problem (flapping mouths, etc...). TDeint's field matching is currently lacking one improvement that would optimize it for field matching of telecined sources vs finding the overall most similar field for deinterlacing as it does now. For that reason, and that I was a little tired of working on TDeint :p, I started working on a stand alone field matcher called TFM a day or two ago. The only other field matcher I've seen that can match small flapping mouths in anime as well as it is bruteivtc. Its main advantage over bruteivtc is that is it about 9-10 times faster and supports user overrides like telecide. Anyways, that's probably more then you wanted to know... but I wanted to show that I'm still working on this filter.

[Mug Funky]

your efforts are very much appreciated :)

if you need some problem samples (i've got some flapping mouth stuff that even bruteIVTC baulks at), you know where to get them. i'll just have to remember to leave my computer on longer (maybe i'll start some encodes :))