TDeint and TIVTC

[tritical]

[link removed], changes from v1.0 beta 1:

+ Added modes -2 and -1... will upsize vertically by a factor of 2 using ELA or ELA2
+ Call SetCacheHints()
+ Some small optimizations, should give a very small speed up Ok, no new features after this release... only changes will be bug fixes if any are necessary before v1.0 final here in a few days.

This doesn't mean the end of TDeint development for anyone who might be concerned (I am probably the only who qualifies)... v1.1 plans currently include adding motion compensation, though not in the usual way. TDeint is very modular, meaning each step is separated (motion map, linking, denoising, interpolating), which is part of the reason it is so slow, but which also makes it easy to add new interpolation types and intermediate steps. My plan is to add optional motion compensation as an intermediate step between motion map creation and interpolating. That is, use the current motion mapping to identify static areas, and then use motion compensation to try and get values for the pixels that are marked to be interpolated (moving). Any pixels that can't be recovered via motion compensation will then be interpolated as usual. Just imagine how slow that will be :D. However, that is a ways away... gonna spend some time on TIVTC now.

[Marius-the-Mad]

Originally posted by tritical

This doesn't mean the end of TDeint development for anyone who might be concerned (I am probably the only who qualifies)...No way. :) Tritical, your deinterlacer is just great. :) Thank you very much for it! I have some NIN videos here (from the new 'The Downward Spiral' remaster) and I don't know what I would do with them without TDeint. :)

In other words: thanks a lot! :D

[warath]

Aye, I love your TIVTC, works wonders on my Star Trek backups!

[LigH]

Due to the recommendation of scharfis_brain, I just tested TDeint 1.0b2 on the VQEG (PAL) test samples, where many deinterlacers clearly show flaws (e.g. KernelDeint leaves "stripes" of not deinterlaced, or even apparently totally unrelated, content). We discovered that even TDeint hat to fight hard here, I had to lower the "AP" parameter even below 10 to suppress such artifacts; even at AP=5, there are still some hints of stripes visible, especially in the part 08 with the scrolling text, when the background liquidly waves forth and back.

[tritical]

Is there a link where I could get those test sequences? Or what is the name of the part 08 sequence? Anyways, from your description it sounds like standard motion adaptive artifacting. Setting AP at 5 you almost might as well do a dumb bob.... though there isn't really anything you can do to prevent motion adaptive artifacts besides turning to a spatial combing metric. Out of curiosity, were you testing by viewing deinterlaced clips during playback or single frames in vdub?

[MacAddict]

Impressive results here with Tdeint doing a typical deinterlace film. I'm thinking of using it full-time in place of KernelDeInt now. Many thanks for your effort tritical!

[tritical]

@Marius-the-Mad, warath, and MacAddict
thanks for the comments :).

@LigH
I found a place to get those sequences and downloaded sequence 8. It is definitely a killer for a motion adaptive deinterlacer.

[LigH]

I once downloaded all the PAL sequences (1 .. 10). A few are progressive (#1 is even a "long still" - but e.g. QuEnc gets crazy about that content), but most are interlaced.

scharfis_brain told me, especially "short shutter" material is hard to compensate, because such algorithms seem to rely on the last bit of motion blur during one field. And computer generated frames often have a duration of 0, not (near) 1/fps seconds - no motion blur at all.

Another "killer" is the "rugby game" (#9), which might also have short shutter, and in #6 - Formula 1 - scharfi told me about possible flaws in the { hoarding | perimeter advertising } ("FOSTERS").

Here a short overview over the VQEG sequences:

ftp://ftp.crc.ca/pub/crc/vqeg/TestSequences/Reference/

1: Tree (25p, long still)
2: Stadium (50i)
3: Harping (50i)
4: EBU Ant (50i, CG)
5: Kayak (50i)
6: Formula 1 (50i) -- btw: M. Schumacher does fail sometimes... ;)
7: Fastfood (25p)
8: Scroller (50i, CG)
9: Rugby (50i; short shutter?)
10: Toy train (50i)

If someone prefers, you may add NTSC clip descriptions here, too. But unfortunately, the download there is sloooowwww (and the files are huge, because they are plain UYVY).
__

Okay - just downloaded the NTSC files, too:

13: Fun fair (TC)
14: Skyscrapers (60i)
15: Toy train (60i - similar to 10)
16: Jungle (30p, CG)
17: Letters (2x15p)
18: Waterfall (60i?)
19: Football (60i)
20: Sailing ship (? - too less motion)
21: Blonde@phone (60i)
22: Flowers+leaves (60i)

[tritical]

Yep, that was the site I eventually found. Sequence 6 I got and tested and it suffers from the same thing, the motion of the FOSTERS sign occurs at just the right rate/frequency that each field appears stationary on a single pixel level. Increasing the length of the static area detection from 5 to 6 fields could help, but wouldn't eliminate all artifacts, would be slower, and would sacriface some true static areas. I am still playing with the idea that in a majority of these cases, especially with the FOSTERS sign, most of the artifacts occur in areas where a majority of the pixels are correctly detected as moving and thus interpolated. Perhaps a dilation type operation on the motion map, or an AP mode where it identifies pixels with an ap value over the threshold and a majority of the pixels inside a window around it are set to be interpolated could help. Or perhaps overlapping block based static area detection. Will have to think about it some more, those samples are definitely nice for testing... thanks for mentioning them.

[rkk44]

Dear Tritical,

I cannot get TIVTC 0.9.8.5 to work in mode 5. Here is the pertinent portion of my script (based on your examples) that deals with TIVTC:

V=mpeg2source("c:\sg1614.d2v",cpu=4,idct=7,ipp=true).assumetff()
A=MPASource("c:\sg1614 MPA T01 DELAY 0ms.mpa",normalize=false).BufferAudio()
AudioDub(V,A)
ConverttoYV12(true)
clp=last.rgbob().selecteven() #Bob Deinterlace then 60 => 29.97 fps
tfm(pp=4,clip2=clp,output="matches.txt")
tdecimate(mode=4,output="metrics.txt")
tfm(pp=4,clip2=clp,input="matches.txt")
tdecimate(mode=5,hybrid=2,vfrDec=0,input="metrics.txt",tfmIn="matches.txt",mkvOut="mkv-timecodesfile.txt")


When I try to run this in VDubMod it gives me the following error:

"Tdecimate: Input Error: Mode 5 -- All Frames Must Have Entry"


However, if I replace the above "TFM/Tdecimate" script with clp.tdecimate(mode=2,vfrDec=0) it works. I also tried directly replacing the clip2 with the d2v but I get the same error.

Thanks for any help which you can provide.

[tritical]

I guess it is not that clear in the documentation, but for mode 5 it needs all the stats so it can calculate everything at the beginning, which means two passes need to be made. In the first pass use/run this script:

mpeg2source("c:\sg1614.d2v",cpu=4,idct=7,ipp=true)
assumetff()
converttoyv12(true) <= is your source 4:2:2?
clp = last.rgbob().selecteven()
tfm(pp=4,clip2=clp,output="matches.txt")
tdecimate(mode=4,output="metrics.txt")
crop() <= change this to crop the image down to 32x32 or 16x16 to speed things up

That pass is only to allow tfm/tdecimate to gather the needed info (i.e. create the output files), so the actual video output of that pass doesn't matter.

Then on the second pass use:

V=mpeg2source("c:\sg1614.d2v",cpu=4,idct=7,ipp=true).assumetff()
A=MPASource("c:\sg1614 MPA T01 DELAY 0ms.mpa",normalize=false).BufferAudio()
AudioDub(V,A)
ConverttoYV12(true)
clp=last.rgbob().selecteven() #Bob Deinterlace then 60 => 29.97 fps
tfm(pp=4,clip2=clp,input="matches.txt")
tdecimate(mode=5,hybrid=2,vfrDec=0,input="metrics.txt",tfmIn="matches.txt",mkvOut="mkv-timecodesfile.txt")

If you are encoding to a loseless file first there is a one pass vfr mode (mode 3), but it will not have the correct # of frames.

Even though the above process requires two passes it can all be set up in vdub's job control to execute at once. That is, load the first script and save it to job control then load the second script and save it. You will need to add the parameter batch=true into the tdecimate() line of the second script for tdecimate to allow it.

However, I think vdub's avi handling has been changed somewhere along the line so that if you load a script once and save it to job control, and then run the process later, while having altered the script to return a different number of frames then when you originally loaded it, vdub still sets the frame count to that of when the script was first loaded. That means loading both scripts at once into job control wont work, because the first time you load the second script tdecimate uses junk info and the framecount is not the same (it will be less) as when it gets loaded later with the real info. I'm not sure exactly which versions of vdub do that (v1.6 does, and probably v1.5.10, not sure about v1.5.4/v1.5.1), but I still use a rather old version of vdubmod (v1.4.13.2) and it works fine.

Not sure if all of that made sense, it is rather difficult to explain easily.

[rkk44]

Tritical,

Thanks for the help. That did the job.

[Just for your information, I am using VDubMod (VDub) Build 1.5.10.]

[tritical]

Alright, I lied about no new features in TDeint. After testing on those VQEG sequences I decided to add two new AP post-processing modes that take motion of surrounding pixels into account and allow for low AP values w/o completely sacrificing static areas or reverting to a plain spatial deinterlacer.

Specifically, sequence 6 (racing w/ fosters sign) is handled much, much better with TDeint(type=3,AP=7). Sequence 8 is tougher but most of the major (visible) artifacts are removed while leaving the logo intact using something like TDeint(type=3,link=3,AP=7,APType=2).

Complete changes:
05/14/2004 v1.0 beta 3
+ Added APType parameter, adds 2 new AP post-processing modes that take surrounding
motion into account
+ Small changes (hopefully improvements) to type 3 (ELA-2) interpolation
TDeint v1.0 beta 3 (http://bengal.missouri.edu/~kes25c/TDeintv1b3.zip)

EDIT: there was a little off by one loop bug in the AP pp routine.. updated the zip archive with a fixed version.

[Mug Funky]

there seems to be some horizontal (!!) artefacts showing up with "TDeint(1,1,type=3,link=3,AP=7,APType=2)".

i'll try narrow it down a little... right now it's happening on an NTSC anime sample (the ADV trailer for "cromartie high school").

[edit]

my apologies... it's in the source. now THAT's weird.

i can anticipate this series giving deinterlacers hell though - all the characters have thin horizontal lines as shading - it flickers like hell on an NTSC TV.

[scharfis_brain]

Mug Funky, would you like to show me a short sample?
(cannot imagine how that should look like)

[Mug Funky]

sure. i'll send you the trailer if you PM me your Gmail (if you got it...)

[edit] oh. it's over 60 megs... maybe i'll just cut the start bit out :)

[scharfis_brain]

this kind of flickering seems to be caused by bad interlaced scaling and rotating of this letter.
so the characters became more flickery.

[tritical]

And another new version... [link removed]. Finally got the new mode 2 added to tdecimate. It is definitely a different approach to arbitrary decimation. It works by applying a series of M-in-N decimations that recursively minimize the error to the required ratio, ensuring decimation is evenly spread throughout the video. Its behavior is controlled with 3 settings. "rate", which sets the decimated output rate as before, "maxndl", which controls how much non-uniformity is allowed in the decimation (for dealing with clips with uneven duplicate distribution), and "m2PA", which removes the default 100 frame look ahead restriction and allows tdecimate to look as far ahead as needed (that enables it to produce the same results as if the metrics were available from an input file).

Anyways, I have not gotten to extensively test it so if anyone finds clips/settings that make it crash, blow up your comp, or do other weird things please report :). It should deal well with clips that regular M-in-N decimation usually doesn't, especially clips that do not have uniform duplicate distribution (as long as maxndl is tweaked to fit the requirements).

Complete Changes:
TDecimate:
+ First cut at new mode 2 operation.
+ Added maxndl/m2PA parameters, go with mode 2.

[MOmonster]

first of all, sorry for my bad english. I´m new to these things and don´t speak it so often.
I´live in germany and all my interlaced material seems to be bad telecined. I tried some filter combinations, like matchbob with unblend and fdecimate or for matchbob the doubletelecide function with tfm(mode=2, pp=7) and also one time restore24, but all these functions and combinations have big disadvantages. First of all, the speed, but also the accurate deblending and the right decimating isn´t a small problem. Of course tfm and telecide are fantastic filters, but for such weird material we have to live with such functions till now.

Tritical, if you could implement such features in your TIVTC, it should work also great for bad telecined material.
Please read this text and also tell me my mistakes.

[MOmonster]

Oh, sorry, here is the text:


Before I will explain how these features should work, I give you four
points, that say us how the material has to be, that the feuture could
work:

1. If 1, 2, 3... are the frames, a1, a2, a3... are the
top-fields and b1, b2, b3... are the botton-fields, there
shouln`t be a match from b2 and a1 (for top-field first),
because b2 would step back in movement. (no float pictures)

2. a blending-field should be between to clear fields
(between the both source-fields for the blending), becouse
otherwise it wouldn`t be possible to detect the blending.

3. If a 0-step means the step between two matching fields,
a 1-step is the step of a whole frame and a 0.5-step is
the step between a clear and a blended field, there
shouldn´t be a 1.5-step in the source (because of point two)

4. Between two matchings (0-steps) there should be only one
(max.) full step (1-step) between two fields, otherwise
audio and video would be asynchron.

Ok, somebody could say, that for his source this four points
aren´t all right, but for this source also restore24 shouldn´t
work correctly.

Now I try to explain how this noblend-feature could work
(for top-field first).

All posible matchings in a source should be found, just by
trying to match the top-field (or botton-field, depends on
the parity) with the previous and the next field. All other
searchings for matching should be useless (so, TFM(mode=0)
should be enough, I don´t know for what the 4. and 5. way
are, maybe as debugger for a wrong parity of some frames,
but should be normally useless). But mode=0 is not enough
for bad telecined material. Let me explain:
If there is a c-match in frame 1, a2 could be a
blending-field. In this case it would be good to look for
a match for b2 with the next field.

a1 a2(b) a3 a4...
l /
l /
b1 b2 b3 b4...

match: c u


But this is only useful after a c-match (not after a p-match).

And now to the bigger problem. A frame has no matching-fields.
It´s time for the postprocessor! But what field should be
choosen for postprocessing. We have to find out what fields
are clear and what are blends. There are many methods for
detecting blends. We could compare the number of edges, the
complexity, the number of colours or for example the
sharpness. Also we can do like unblend, but all these methods
aren´t accurate enough or just need to much time.
I thought about a method, that use the possibility of a
matcher and that should solve this problem. So easy it sounds,
so good should it work.

To detect a field as blended (or not), we just blend the both
fields around together and try to match this new field with
the tested field. If both fields really match, the tested
field is blended.
This is the theory.
To make it more practicable, we just have to test both fields
after the last possible match. The clear field is the field
where the match results in more combing than the other.

I try to make it more clear with some examples:

A. a1 a2 a3(c) a4 a5...
/ l
/ l
b1 b2(b) b3(b) b4 b5...

match: p c


1. frame 3 gives no c- or p-match
(u is unneccessary, because of the p-match)

2. less combing for matching b2 with blend of a2
and a3

3. more combing for matching a3 with blend of b2
and b3, so a3 should be reproduced.

B. a1 a2 a3(b) a4 a5...
/ /
/ /
b1 b2(c) b3 b4 b5...

match: p p

1. frame 3 gives no c- or p-match
(u is unneccessary, because of the p-match)

2. more combing for matching b2 with blend of a2
and a3

3. less combing for matching a3 with blend of b2
and b3, so b2 should be reproduced, through
it is a field of frame 2.

C. a1 a2 a3(c) a4(b) a5...
/ /
/ /
b1 b2(b) b3(c) b4 b5...

match: p p

1. frame 3 gives no c- or p-match
(u is unneccessary, because of the p-match)

2. less combing for matching b2 with blend of a2
and a3

3. more combing for matching a3 with blend of b2
and b3, so a3 should be reproduced.

4. also frame 4 gives no c- or p-match

5. more combing for matching b3 with blend of a3
and a4

6. less combing for matching a4 with blend of b3
and b4, so b3 should be reproduced.

D. a1 a2(b) a3(b) a4(c) a5...
l l
l l
b1 b2(c) b3(c) b4 b5...

match: c c

1. frame 2 gives no match (also no u-match)

2. less combing for matching a2 with blend of b1
and b2

3. more combing for matching b2 with blend of a2
and a3, so b2 should be reproduced.

4. also frame 3 gives no match (also no u-match)

5. less combing for matching a3 with blend of b2
and b3

6. more combing for matching b3 with blend of a3
and a4, so b3 should be reproduced.

There could be much strange pattern in bad telecined material, but
with such a feature TFM should be able to reproduces also such weird
material. I´m no coder and knowledge about the functions and filters
of avisynth is to bad, to be able to create a function, that would
enable this feature. It would be great, if you could implement such
a thing in your filter, tritical.
The advantages over restore24, matchbob, doubletelecide and so on
are clear. It would be really faster and should be a bit more
accurate against blends.


And there is also an other feature for your tdecimate.
If the tfm would have the possibility to tell tdecimate, what matched
frames use a same field, for example if after a c-match follows a
p-match. This could be possible for example with the hints
(an use_hints option for tdecimate).
With such an option tdecimate could work a lot faster, if there
are given enough frames for decimating. And tdecimate should also work
better, because it would prefer these frames more then other, what
maybe are doubled just because of no motion.

It would be great, if this text could help you to tweak your
fantastic filters.

[tritical]

@MOmonster
Sorry for not getting back to you sooner, but I was on a trip the last four and a half days with no internet access. I read through the text and didn't quite follow some of the specifics, mainly the assumptions about which comparisons would produce more or less combing, but I think I got the general idea. Would you mind sending me a sample to test on? I don't have any clips with blending patterns as you describe. If so just pm me.

@All
[link removed]. This version adds iSSE optimizations for the default metric calculation case of TDecimate. That is when blockx = 32, blocky = 32, nt <= 0, and width and height are mod 16. The speed up is pretty significant. On a 10,000 frame test clip tdecimate went from running about 120fps to 1200fps on my comp, so roughly a factor of 10. Your mileage may vary. The optimizations are for both YV12 and YUY2, both with and without chroma.

[Chainmax]

I have an interlaced clip that gives TDeint some trouble. I will try to upload it somewhere soon.

[MOmonster]

Sorry for my late answer. I only have internet on weekend till now.
I have a sample of 8 mb for you, but I don´t know how to upload. This sample is so bad telecined, that tfm and telecide use the postprocessor on really many frames. But many of this postprocessed frames are blendings, because your filter just choose the first field to recreate the progressive frame.

Can anybody help me to upload the short sample?

[LigH]

Free uploads:

http://rapidshare.de

[MOmonster]

Thank you, LigH.

Here is the sample (http://rapidshare.de/files/2014414/sample.m2v.html).

[tritical]

@MOmonster
Thanks. After looking at it I think your idea could work well. I'm gonna implement it into tdeint first though cause it will be easier. Bascially, add a new mode that outputs at same framerate but instead of always using the top or bottom field it will choose adaptively based on your blend detection by comparing combing idea. Will see how it goes.

@Chainmax
Any luck uploading that clip? If you don't have a spot I can just give you my ftp info.

[Chainmax]

I have a place to upload it to, but I had a virus infection a couple of weeks ago and want to make sure I'm clean as a whistle before uploading anything anywhere. It will take a few more days (basically, I need to find someone willing to scan any file from my machine which should be infection free now).

[MOmonster]

@tritical
Oh, I havn´t expected, that it will be easier to implement it into Tdeint.
I´m really happy, that you try to implement this blend-detection. :D
Will it also work together with your tryweave option? If yes, this should work similar to my idea for your tfm filter (maybe not in all cases same good, but of course really nice).

Thanks for your attention.

[tritical]

I'm not sure exactly how it's all gonna work in the end, but as of right now I think it will be achieved via a tfm+tdeint combination. A script like:

deinted = tdeint()
tfm(clip2=deinted)

So far, I've added a new 3-way match mode into tfm that is like mode 4, but with all matches that go against the field order removed. So for a tff stream, matching off top field, it first tries c/p, if the best of those two is detected as combed then it tries u and if u isn't combed it uses that else it goes back to the best of c/p and does PP.

I've tested the blend detection method you mentioned above and while it seems good in principle it doesn't work that well in practice. The major problem is that the blend ratio is not a constant and that screws up the comparisons. Instead, I decided to try a method that simply checks to see if the current field is a blend of the previous/next field. Since the two surrounding fields should be clean, simply determining that the current field is a blend should be enough. To do that it creates 5 frames, assume we have the following fields:
B <= top field
A C <= bottom fields
and we want to know if B is a blend of A/C. The five frames it creates are:
0 1 2 3 4 <= frame #

B B B B B <= top field
A (25%A/75%C) (50%A/50%C) (75%A/25%C) C <= bottom field
It then checks the overall combing level of those five frames. If min(1,2,3) is less than min(0,4) by a significant margin then it decides that B is a blend of A/C. The extra 25/75, 50/50, and 75/25 frames are to account for uneven blend ratios. It seems to be working quite well so far, but will need more testing. Of course the above method will not work for true interlaced material, as it assumes that B either matches with A or C or is a blend of those two.

[Revgen]

@Chainmax

Try to use http://www.yousendit.com to send the file to Tritical.

You can upload any file up to 1 GB to their server for somebody to download, and the file will stay on their server for seven days.

It costs nothing to use. It also scans files for viruses, so you don't have to worry.

I hope this helps.

[MOmonster]

@tritical
Oh, I also thought about this problem, but I wasn´t able to test it before. I thought, we still get the blended field, if we compare the combing levels for both fields, but I seem to be wrong.
If MIN(0,4) is really significant higher than MIN(1,2,3), would you also test the field C and than compare the differences or would you use a combing difference parameter to decide, if the test of field C is really necessary.
Your method seems to be accurate enough to find the blend fields.
By trying to match A & B or B & C we get the the combing levels for 0 and 4 and now compare it with MIN(1,2,3) to find the blending.
Maybe you could make this method a bit more effective.
If B is really a blend of A and C, maybe we could compare the combing levels from matching A with B and B with C to find out how strong the blending factor is.
If B is for example an A:C => 30:70 blend, the combing level from matching B with C should be much higher than for matching A with B.
So, it shouldn´t be necessary to create three new frames. We should be able to use an adaptive blend factor by comparing the combings. The rest would work how you said it.
Maybe it is possible to choose the right blendfactor a bit more differenced, for example choose between 20, 35, 50, 65 and 80 percent, but it is only an idea.
If C is the blend and not B, the blending factor should be unimportant, because the combing level should be still much higher than for the blended field, so we could work the same way.

Thank you for your work, tritical.

[Didée]

Hardly I dare to mention Restore24, but its blend detection works pretty good. Of course it is slower, since it works with edgemasks. But the percentage of blend reckognitions in not-static parts is usually >95%.
The idea is free to use, that's why Restore24 once was posted at all :)

(And also, that's why some are waiting for a suited decimator, less unstable than SmartDecimate, to fall from heaven.) ;)

[MOmonster]

Hi, Didee

You´re right, the blend detection of restore24 is really good.
But that is not the reason why I search for another way to make a good restoring.
restore24 has many disadvantages, not quality wise, but there are many other things, like maximum video length, the stability of this complex function and of course the speed.
The most important point is the dividing of the operations like deblending, decimating and so on. There is no need to search for blends on matched frames because there shouldn´t be any blending. Another thing to look at, is the fact that only, if there are blends, there are no possible matches, so this way, we only have to decide what of two fields is the blended field and don´t have to find out the less blends on the whole bobbed source. Also it is harder to decimate a bobbed source right than source with same framerate. We could tweak the decimating also if the decimater would be able to read the informations from the matcher and so on.
One more time, I don´t think restore24 works bad, no, it is an amazing function, that gets clear with nearly all bad telecined material. But there are other things, I said before, why I look for an alternative and this here seems to be also a good way.

[Chainmax]

Revgen, thanks for the tip, I'll use that.

tritical, yousendit needs a recipient email adress. Please create a temp account in a free mail and send me your adress in it via PM so that I can upload the file for you.

[MOmonster]

@tritical
I use the display info from Donald A. Graft´s Telecide to get something like a combing level for the different matches. You said, you don´t have such bad telecined material, so I´ll try to test a little bit on my sources. Till now my idea should really work, but I have to make many more tests, till I´ll come to a conclusion. It will need some time, but I hope it will help you. I don´t know, if the combing levels of Donald A. Graft´s Telecide are that much useful for you, maybe one of your filters could also display something like the combing level, that you could use the results better for you, but first I´ll try it that way.

@all
Does anybody know a good possibility to show the combing level of the current picture on the display?

[MOmonster]

@tritical
I wait for your next tivtc version, because I want to test you new three way mode.
So far so good, for today I stop with my test. I have learned a lot from testing the different sources. First of all my idea that after a clear field without a match can follows a second clear field without any possible match seems to be wrong. BUt my special mode for tfm I thought of wouldn´t be useless.
I made tests on ten different sources and more then eighty different motion scenes. On four of ten sources, there are sometimes (really seldom and only light) two blended fields, that stands together, so bad are the conversations.
It is possible, that if we have an u-match after an p-match, we lost a clear field.
This simply means, that we only should have a look to the next two fields after the last possible match. If we have a p-match and no possible c-match (there are maybe both matches possible, but p is better) we just should try to match the next two fields (second field from the previous frame and first field frame the frame we try to restore). That means only c- or p-match would be recommed. With a possible c-match of the previous frame also an u-match would be useful. The other thing is of course, that only this two fields should be used for the postprocessing with blend detection.
If we use tdeint as postprocessor after tfm we won´t solve this problem. Maybe you can think about the implemetation of such a mode in the feature.

But now to the more important thing, the blend detection tests.

Sorry tritical, but your idea also wouldn´t work that good. In nearly all scenes I looked at, I found some blends with blend factors between 10 and 15%, so the difference between MIN(0,4) and MIN(1,2,3) wouldn´t be that high or clear and sometimes we would detect the wrong field as blended.
A better method seems to be, that we compare the MIN(1,2,3) from testing the first field with the MIN(1,2,3) from testing the second field. In some cases this seems to work better, but also not for all absolut right.
I´m happy that my idea seems to work pretty good in all cases, I tested till now. To find the right blend factor by comparing the matches 0 and 4 seems to work really fine (till now not tested on really noisy and blocky source) also for the different sources. I still would prefer to compare the combing levels from the matched blend creations with the test of the other field. In some cases this seems to work better than compare the differences from MIN(0,4) with this combing level.

I made a list of the blending factors depends on the combing level ratio (0,4). I made it that accurate, clear because this should work better. Depening on the source the variance to the real blend factor isn´t higher than 3% (for my tests). I think this sounds not bad. I´ll make some more tests in the feature and maybe will modify the list a little bit, but till now this seems to work good. I hope these tests would help you a little bit.
So, here is the list :o


A = the ratio between the both matches with the blended field
(used the match values displayed from telecide)
B = the percentage of the blend factor from the field with the lower match values


A B A B
(value:1)

1 50% 2.5 29%
1.02 49% 2.6 28%
1.05 48% 2.7 27%
1.08 47% 2.9 26%
1.12 46% 3.0 25%
1.16 45% 3.2 24%
1.20 44% 3.4 23%
1.24 43% 3.6 22%
1.28 42% 3.8 21%
1.33 41% 4.1 20%
1.38 40% 4.4 19%
1.44 39% 4.7 18%
1.5 38% 5.0 17%
1.6 37% 5.4 16%
1.7 36% 5.8 15%
1.8 35% 6.3 14%
1.9 34% 6.7 13%
2.0 33% 7.2 12%
2.2 32% 7.7 11%
2.3 31% 8.2 10%
2.4 30%

[Chainmax]

tritical, did you get a chance to examine the sample?

[tritical]

@Chainmax
Yep, I took a look at it. I can't really give any suggestions or think of anything to improve the result significantly. It is just one of those tough clips to handle for a per-pixel motion adaptive deinterlacer. Hopefully handling of such cases will improve in the next version. Thanks for sending the clip, I will definitely keep it for future testing.

@MOmonster
Thanks for those tests. Indeed the 3 cases for the blend detection method I mentioned aren't always enough, smaller steps are needed.

I have been thinking more about the overall process and how it will work. After thinking about it, I am less and less inclined to try and include this in tfm/tdeint, and am thinking it would be easier to make a separate filter (that internally uses tfm/tdeint) specifically for handling such video. After looking at restore24 a little more in depth (I have never had any material that needed it) I think it's general approach is quite good. Turning restore24 into a filter (or making a filter that works like restore24) and adding some of the ideas from here (field matching, matching patterns, alternative choices for blend detection, etc..) seems like the best way to go. The filter would be able to grab the needed info about matches from tfm/tdeint since they already hint all relevant information. Anyways, I haven't had much time the last few days to work on it, hopefully that will change.

[scharfis_brain]

Hi tritical, I am currently playing wit TDecimate within restore24 :)

it works like it should!


Is there room for some minor impovement?

Restore24 delivers mainly this pattern:

Filmframes 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 ...
inputframes xd xd xd xd xd xd xd xd xd xd xd xdn xd xd xd xd xd xd xd xd xd xd xd xdn xd xd xd ...


x - new frame
d - absolute duplicate
n - near duplicate (is a duplicte, but affected by noise and deinterlacing artifacts)

currently tdecimate chooses one of x, d or n when it detect such a triple
(I didn't examine which one it choosed..)

is it possible to force tdecimate to mix all near-duplicates to get noise and artifacts reduced?

In the case of Restore24 for frames 11 and 23 either x&n or d&n should be mixed.

This duplicate-blending should also help on normal IVTCes,
because noise and/or comression artifacts will be reduced be half.

[tritical]

Yep, that is possible... but how do you want to define near duplicate? I mean, how will tdecimate know which frames are near duplicates and not simply different frames with a small amount of motion? I guess it could be by looking at each frame that isn't dropped and seeing if its neighbors were dropped and are less than a certain difference threshold?

[scharfis_brain]

mean, how will tdecimate know which frames are near duplicates and not simply different frames with a small amount of motion?

Tdecimate already knows, that x, d AND n are duplicates of each other, because I only see one of them in the final output.

So blending the most un-similar duplicates should be possible, because the string of duplicates (here xdn) is already known..

[tritical]

That is only true under perfect conditions, which is definitely not always true. There is the possibility that there are more non-duplicates in a given sequence then there should be given a specific decimation ratio. In the star trek voyager sequence you sent me a while ago there were a few places where this was the case. That means one of the dropped frames could actually be a non-duplicate and blending it will produce ghosting. Also, you want the blending only when there are triples? Or blending for all dups?

[scharfis_brain]

That means one of the dropped frames could actually be a non-duplicate and blending it will produce ghosting.

But it also will reduce jerkyness a bit, if tdecimate is unsure about which frame to choose.

Also, you want the blending only when there are triples? Or blending for all dups?

No.

I had this idea: blend all most different duplicates of one row.

examples of duplicate chains (non-real-world):
n1,n2,n3 -> slight different near/new duplicates
d -> bit identical, or better said: almost identical duplicate of the preceding frame

this x d n1 will blended to: x+n1 OR d+n1
this x d n1 d1 will blended to: x+n1 OR x+d1 OR d+n1 OR d+d1
this x d n1 n2 n3 will blended to: x+n1+n2+n3 OR d+n1+n2+n3

Most duplicates after R24-processing are bit-identical. So a near-duplicate(s) should be detectable...
Of course, this assumtion does not fit for other sources,
as well as for the ST-Yoyager sample I sent to you because I compressed that one, so the duplicates arent identical anymore.


I hope I didn't cause too much confusion.

[MOmonster]

@scharfis_brain
Nice idea. Reduce jerkiness by blending the near doubled frames. But wouldn´t it be better just to choose the frame with better quality, because the blending will also produce a little softness in many cases (in some sources I tested some time ago). Also the reducing of the jerkiness wouldn´t be so strong just by blending the frames together.
In most cases the second duplicated frame is the frame with the better quality. Another point is that the match with less combing also seems to have the better quality. Maybe the decimater could read the informations from the matcher, that he could decide what frame to choose. But as I know restore24 use telecide to match the fields, don´t know if something like this is possible. But in sources I tested last time it would be better just to choose the right frame than blending them together.
It´s just an idea.

@tritical
I can understand you. We will see, what is going on in the future. I have begun learning C and C++, maybe in the future I will make my own filters. We will see.

[tritical]

Well, here is [link removed] ... it adds the new matching mode (new mode is mode 2, old modes starting at mode 2 have been bumped up one so old mode 2 is now mode 3 and so on). It also adds support for the new dgindex d2v files (v9). Gonna see about adding a general purpose combing measure output filter based on the 5 point metric. Currently tfm can output the match statistics it calculates via outputdebugstring() if debug=true, but those statistics are match specific (i.e. can't be compared to statistics from other matches).

@scharfis_brain
So I'm still not exactly clear on how tdecimate will know which frames are near duplicates. You're saying to look for two bit identical (or very close) frames and then consider all frames after and up to the next pair of identical frames to be near duplicates?

[MOmonster]

Hi tritical,
it took some time, but some minutes ago I finished my function. I tried to create a function, that realize my idea of a blenddetection and here it is. :eek:

LoadPlugin("C:\program_me\dgmpgdec\DGDecode.dll")
LoadPlugin("C:\program_me\filters\TIVTC.dll")
LoadPlugin("C:\program_me\filters\FDecimate.dll")
#LoadPlugin("C:\program_me\filters\TDeint.dll")
LoadPlugin("C:\program_me\filters\leakkerneldeint.dll")

#AVISource("D:\video\inu\[Ingow] Pirates Of Japan (Trailer).avi")
#OpenDMLSource("C:\Dokumente und Einstellungen\Benutzername\Eigene Dateien\Eigene Videos\film.avi")
#DirectShowSource("D:\video\inu\[Ingow] Pirates Of Japan (Trailer).avi")

MPEG2Source("D:\dvd.d2v",cpu=4,iPP=false)


#trim(0,0)

crop(8,8,-8,-8)


a = Cdeint()

tfm(order=1, mode=0, display=false, slow=false, pp=2, clip2=a)

fdecimate(rate=23.976,threshold=1.0,metrics=false) # or another decimater


function Cdeint(clip clp)
{

###### PREPARATION ######
global sourceclip = clp.leakkernelbob(order=1, sharp=false, twoway=false, threshold=7, map = false)
global testclip = sourceclip.duplicateframe(0)

###### FRAMES FOR BLENDPARAMETERS ######
global combing_n0 = testclip.selecteven()
global combing_n1 = testclip.selectodd()
global combing_n2 = combing_n0.trim(1,0)
global combing_n3 = combing_n1.trim(1,0)

###### POSSIBLE OUTPUTS ######
out = sourceclip.selecteven()
out_1 = sourceclip.selectodd()

###### Conditional Function Chain, evaluated from bottom to top (!) ######
c99=ConditionalFilter(clp, out_1, out, "btest/Min(combl_0, combl_1)", "lessthan", "btest_2/Min(combl_1, combl_2)")

c9=FrameEvaluate(c99, "global btest_2 = LumaDifference(blendmade_test, blendpic_test)")
c8=FrameEvaluate(c9, "Evaluate_test()")
c7=FrameEvaluate(c8, "global blendl_test = combl_1 < combl_2 ? 0.5 * Exp(0.66 * Log(combl_1 / combl_2)) :
\ 1 - 0.5 * Exp(0.66 * Log(combl_2 / combl_1))")
c6=FrameEvaluate(c7, "global btest = LumaDifference(blendmade, blendpic)")
c5=FrameEvaluate(c6, "Evaluate()")
c4=FrameEvaluate(c5, "global blendl = combl_0 < combl_1 ? 0.5 * Exp(0.66 * Log(combl_0 / combl_1)) :
\ 1 - 0.5 * Exp(0.66 * Log(combl_1 / combl_0))")
c3=FrameEvaluate(c4, "global combl_2 = LumaDifference(combing_n2, combing_n3)")
c2=FrameEvaluate(c3, "global combl_1 = LumaDifference(combing_n1, combing_n2)")
c1=FrameEvaluate(c2, "global combl_0 = LumaDifference(combing_n0, combing_n1)")
return(c1)
}


# (running inside of the Conditional Environment)

function Evaluate()
{
vid = testclip.selecteven()
vid_1 = vid.trim(1,0)

global blendmade = MergeLuma(vid,vid_1,blendl)
global blendpic = testclip.selectodd()
}

function Evaluate_test()
{
vid_2 = testclip.selectodd()
vid_3 = vid_2.trim(1,0)

global blendmade_test = MergeLuma(vid_2,vid_3,blendl_test)
global blendpic_test = testclip.selecteven().trim(1,0)
}

function Min(float a, float b)
{
temp = a < b ? a :b
return(temp)
}



Yes, I know there are still many tweaks possible. Till now, I don´t use a special mode to save more clear fields (the mode I explained some posts ago), there is no noise parameter till now, we can´t change the bobber and the most important point is, that the used values are still tweakable.
But also this early version seems to work really good.
I just post it here, that you could have a look on it. We spoke about it some posts ago.

[Chainmax]

I am trying to rip one of my DVDs, but my current script crashed VDubMod (or Avisynth, don't know). After checking filter by filter, I discovered that TFM was causing the crash. Any idea why? Here's the script:

MPEG2Source("X:\wherever\Test.d2v",cpu2="ooooxx")
DeDot()
TFM(d2v="X:\wherever\Test.d2v",mode=3,PP=7,mChroma=true,chroma=true,mi=50)
TDecimate(mode=1)
FixChromaBleeding()
LumaYV12(lumoff=-2,lumgain=1.0)
ColorYUV(levels="pc->tv")
HQDering()
Crop(12,0,700,476,align=true)
BicubicResize(640,480,0,0.5)
BlindPP(quant=0,cpu=0,cpu2="ooooxx")
LimitedSharpen()

[tritical]

I'm gonna guess that you are using a very recent version of dgindex? If you open your .d2v file in notepad and look at the first line what is the version number (DGIndexProjectFilexx, the xx)? If it is 10, then I haven't gotten around to releasing a version with support for that yet, and tfm will throw an error when it finds that it isn't a supported d2v version. Actually, I added support for v10 a while ago (only needed to change 1 line), but also added a few new features to tivtc and haven't been able to force my self to update the docs so it can be released.

On a side note, when you say crashed... does that mean vdubmod disappears without an error msg? If you're on xp sp2 and using a 2.5.6 beta, try replacing the avisynth.dll in your system directory with the one from this zip (http://bengal.missouri.edu/~kes25c/avisynth 2.5.6 - 6.22.2005.zip) (save a copy of your current one if you want to revert back), it should correctly report the error. If it is indeed a problem with supporting the d2v version.

[Chainmax]

Yes, I'm using the latest DGIndex and it uses v10. VDubMod does disappear without an error message and I am using an updated WinXP and Avisynth v2.5.6b3. I will try the dll, even though it's kinda evident from your post that it's a problem with DGIndex v10 since I really miss the error messages.


[edit] do I only have to replace the avisynth.dll, or the directshowsource.dll and TCPDeliver.dll files too?

[tritical]

Just avisynth.dll should do the trick. Nothing of significance has changed in tcpdeliver since beta 3, and the only change in directshowsource was a fix for rmvb getting stuck at 100%.