TDeint and TIVTC

[Adub]

Oops!!! Sorry about that, I am browsing to threads at once.

[foxyshadis]

btw Tritical, whenever you find time to pick this filter up again, here's an interesting approach that may or may not speed some things up: http://www.compuphase.com/graphic/scale2.htm However, his result isn't as good as NEDI, so it may not be suitible.

[tritical]

His method is definitely quick, but the results aren't all that great. Using the gradient direction doesn't work at all for eedi2's main use of deinterlacing (detecting directions on a single fields) and the area he considers is too small to really be effective. Also, normal first order gradient calculations have a hard time accurately detecting directions of lines/edges with small slope (since they consider only a 3 pixel neighborhood). In his method that doesn't really matter since he also limits the interpolation to 50/50 averaging of a set of the two nearest neighbors. That fact alone makes it unable to accurately adapt to lines and edges that have directions of less than 45 degrees. For comparison I made 2x enlargements of the lena image using eedi2 (http://bengal.missouri.edu/~kes25c/lena-eedi2.PNG), nedi (http://bengal.missouri.edu/~kes25c/lena-nedi.PNG), lanczos3 (http://bengal.missouri.edu/~kes25c/lena-lanczos.PNG), and spline36 (http://bengal.missouri.edu/~kes25c/lena-spline36.PNG) in avisynth. Gotta go for the moment, but more to add later :).

[foxyshadis]

Hmm, ah well. Just something I came across, looking for something else. ^_^ Maybe it'd be useful for coming up with an initial gradient, unless your method is already faster/more precise. Looking at calcDirections, it seems to be a lot more efficient than I first thought anyway. Cool.

BTW Lena. (http://foxyshadis.slightlydark.com/random/lena-ekg.png)

[scharfis_brain]

tritical, how can I tweak EEDI2 v0.9.2 to behave the same like v0.9.1 ?

cause the new version detects a lesser amount of short diagonales as edges.

[tritical]

@foxyshadis
What settings did you use for ekg? I tried to reproduce your result, but with default settings the output is definitely not the same :).

@scharfis_brain
maxd=12,pp=0. Could you post an image(s) where 0.9.2 doesn't connect short diagonals and 0.9.1 does?

[foxyshadis]

ImageSource("lena-org.png",end=0)
converttoyv12(matrix="PC.601")
LimitedSharpenFaster(SMode=4,LMode=3,Strength=100,special=true,wide=true,soft=25)
EKG(20,iter=2,bp=.75,sharp=.6,multi=2,gthresh=1.4)
spline36resize(256,256)
return ConvertToRGB32(matrix="PC.601")

With a dash of overkill. ;)

My original intuition was that sharpening should be applied harder at every successive step to make up for the larger image, but that turned out to severely distort the image, and the new limits placed on it are quite arbitrary. Perhaps I have it the wrong way around; I think I need to indulge in more research in that area.

[johnmeyer]

Does anyone know what the "h" match code signifies:

Here's the line from the TFM output file:

9225 h + [196] (181 196 203 0 125)

It always appears next to the combed frame match (+), but so do other match codes. It doesn't appear anywhere in the documentation.

Edit: OK, I think it means a combed frame, when matching to the current (c) frame. For some reason, in this situation, the "h" gets substituted for the "c." If a combed frame results when matching to the next frame, the usual "n" is used.

[tritical]

As there is no explanation for tfm's output file anywhere, I'll give a brief description and answer the h match question. The format and fields are a mess from hacking in spots to hold new info, and I've tried to go for quick and easy readability instead of data compaction.

basic line format is:

frame_# match combed_frame d2v_duplicate [mic_value] (verbose output)

match can be one of:
p,c,n,b,u,h,l

p, c, n, b, and u are all explained in the tfm readme. h indicates a combed c match where the top field is to be kept (bottom interpolated) and l indicates a combed c match where the bottom field is to be kept (top interpolated). The distinction isn't important for tfm, but is needed by tdecimate for hybrid detection.

combed_frame marker will be either '-' for not combed or '+' for combed. If pp=0 then no mark will be present.

d2v_duplicate will be 1 if the frame is considered a duplicate because of rff flags and will not be present at all otherwise. If d2v is not used or flags=3, then d2v duplicates will not be considered and thus not marked.

[mic_value] lists the mic value for the match used. It is the value before any deinterlacing (i.e. a combed frame should have a mic_value above MI).

(verbose_output) will optionally list the mic values for other matches as indicated by the value for micout.

If pp=0, then none of the mic fields will be populated.

[johnmeyer]

Much appreciated! I am using the additional info that you now put into the output file to do my own MIC override in order to perfect my pattern matching. It works really well (this is for the film projector to video project). I can't tell you how impressed I am with your work.

I'm now doing full 24 fps "Cintel-style" film-to-video, thanks to TFM.

John

[tritical]

@johnmeyer
I'm glad that tfm is useful for your project. If you need any other info just ask :).

@All
Going back to the eedi2 discussion... one of things I've noticed is that when chaining multiple calls together for resizing sometimes changing the order of the turn() calls can make a very large difference in line reconstruction. For example, the lena image (http://bengal.missouri.edu/~kes25c/lena-eedi2-2.PNG) again, but this time using:

turnleft().eedi2().turnright().eedi2()

as opposed to:

eedi2().turnleft().eedi2().turnright()

which is what I used to generate the one in the previous post. In this image (http://bengal.missouri.edu/~kes25c/lena-eedi2-diff.PNG), I've stacked the two images side by side and tried to highlight a few of the major areas of improvement. Notice that most of the areas are edges/lines of >= 45 degrees.

After thinking about it, the reasons for the improvement are pretty simple. First, the vector matching method currently used to detect directions has only single pel accuracy up to the previous and next scan lines. So the angle directions drop off very fast at first (90 at 0 steps, 45 at 1 step, 27 at 2 steps, etc...). This leads to poor accuracy when dealing with edges that have angles of >= 45 degrees. Second, the interpolation of the new pixels is limited to half pel precision at the previous and next lines.. so basically all edges/lines with an angle of >= 45 degrees will be lumped into 3 possible output pixel values. What happens when you rotate the image 90 degrees is that the lines/edges that were previously in the 45-90 degree grouping are now in the 0-45 degree range where the accuracy of angle detection and interpolation are much improved.

I think the most obvious solution is when initially calculating directions to do one pass horizontally (to get <= 45 degree lines/edges) as is done now, and then to do another pass vertically (to get >= 45 lines/edges). Also, when doing the final interpolation, directions of > 45 degrees should use pixels from lines further away than just the prev/next lines in order to get improved accuracy. Of course, there are some fine details to work out, but it shouldn't be too hard to implement. Hopefully this improvement will make it into the next version along with the one step, general resize method. The downside is that this improvement will slow things down to ~3/4 to 1/2 the current speed.

[scharfis_brain]

The newest version of eedi2:
http://home.arcor.de/scharfis_brain/samples/eedi2-new.png

The predecessor of the current version:
http://home.arcor.de/scharfis_brain/samples/eedi2-old.png

I used maxd=12 and pp=0 as you suggested.

[tritical]

The difference is most likely because of the extra filtering step I added to the direction map post-processing. Any way I could get the original clip from you?

[scharfis_brain]

this is the original clip.
http://rapidshare.de/files/23657792/doom9.avi.html

it is pretty good evaluating interpolation algorithms and motion/comb masks

[tritical]

[link removed], changes:
tfm:
- fixed crash in modes 0/4 when pp=0 and micout > 0
- fixed "possible missed combed frames" listing in ovr help information sometimes
including frames that were marked combed via an ovr file

tdecimate:
+ set fps to average value of the clip in mode 5 (so duration stays the same)
+ use 6 decimal places in v2 tc output

I had planned to include a lot more in this release, but I just haven't had the time to do much lately. Things that will eventually get added (as I have promised in various places):

1.) exclusion band in tdecimate+framediff
2.) exclusion band for combed frame detection in tfm+is/showcombedtivtc
3.) option to require 'x' number of blocks above MI for combed frame detection in tfm+is/showcombedtivtc
4.) pp option in tfm that would try to replace combed frames with clean neighbor frames if available

[JuanC]

wow! that's great!

And the features you're announcing will make it even greater !

Thanks again :)

[anonymez]

thanks a bunch tritical! :)

ps has to be said: tdeint is well past RC stages, make it 1.0 final already! ;)

[tritical]

@All
I also updated the avi_tc_package to v1.4. The main changes were:

1.) added avs output option to cfr2tc. The avs file will exactly duplicate the avi file you would get if you ran cfr2tc with avi output. Bascially, it generates a bunch of trim().selectevery().assumefps() lines and then appends the clips together. This eliminates the need to create a new avi file.
2.) added tcConv... it does v1<->v2 conversions.
3.) updated tc-GUI to match.

@anonymez
Yeah, it is pretty much done... though I still have plans to make the field matching and combed frame detection in tdeint exactly the same (working and parameter wise) as tfm. Currently, the field matching in tdeint is equivalent to slow=0+mchroma=true in tfm. The combed frame detection is the same as tfm's atm, but the new options I am going to add to tfm should get added to tdeint as well. At some point it might just be worth it to combine tdeint/tivtc into one.

The main reason I would like to have the improved field matching modes of tfm in tdeint is because it's used in mode 2, which I pretty much use all the time. My usual script these days for anime ivtc is:

mpeg2source("")
inter = separatefields().eedi2(field=-2)
deint = tdeint(mode=2,edeint=inter)
tfm(d2v="",clip2=deint,MI=60)
tdecimate(mode=4+5)

[foxyshadis]

@All
I also updated the avi_tc_package to v1.4. The main changes were:

1.) added avs output option to cfr2tc. The avs file will exactly duplicate the avi file you would get if you ran cfr2tc with avi output. Bascially, it generates a bunch of trim().selectevery().assumefps() lines and then appends the clips together. This eliminates the need to create a new avi file.

Sweet.

2.) added tcConv... it does v1<->v2 conversions.

Very sweet. Does it attempt to use standard framerates, if they're close enough?

[tritical]

No, it doesn't. To get the framerates for v2->v1 it just averages over the duration of the current range... which will get pretty close usually. Afaict, this is exactly what mkv2vfr does, since in my tests converting the v2 output of mkvextract to v1 resulted in fps numbers that matched mkv2vfr's output.

[Chainmax]

...
tdecimate(mode=4+5)

How is that possible? It's probably a typo, right?

[foxyshadis]

Just shorthard for carrying out the full mode 4 and mode 5 translations.

Certainly, it would be interesting to have a mode 9 to do it all, like decombvfr, if only for those generated scripts that don't need to be tweaked... but if we did that we'd have to document that the mode 8 that you seek is not among these modes... ;)

[Isochroma]

I seem to be having some difficulty with tc2cfr. I have an avi file of fps 23.976 which I would like to convert to one with a framerate of 119.880 by inserting null frames at the appropriate intervals. However, tc2cfr wants a timecode file.

It seems according to the documentation that this program will take an avi and add null frames, producing a timecode file in addition. What should I do?

[foxyshadis]

Use this as the timecode file:

# timecode format v1
Assume 23.967

Alternately you can just use vdub, which can do 24 or 30->120 in the same way. (But no hybrid, obviously, and you can't automate it.)

[Isochroma]

Thanks! But I found AVI60, this nice Japanese program to convert various framerate AVIs to 119.880. It works quite nicely, though it's all in Japanese.

Really, someone should make a simple tool that doesn't require in-depth knowledge of obscure file formats. Just something that adds a certain number of null frames to an AVI and doesn't require anything else to do its job.

More than what is needed; less than what is required.

[tritical]

I could make it use the default framerate of the input avi if no timecode file is given, but what is the point of making a 120fps avi out of a source that is all one framerate to begin with?

[Isochroma]

Because I have 3 source files:

#1: 23.976 fps
#2: 29.970 fps
#3: 23.976 fps

They must be merged sequentially. Therefore, they are all first converted to 119.880 fps, then #1 is opened in vdub, then the other two are appended and the resulting file saved as the complete. In direct stream copy mode, the null frames inserted by AVI60 are preserved in the final output AVI.

Next, your cfr2tc app is used with the complete AVI as source. It removes all null frames and generates a v2 timecode file and an AVI with null frames removed.

Finally, the AVI and timecodes file are muxed into a MKV via cmdline (note: the manual and cmdline parameters explanation for MKVMerge does not provide any information on the --timecodes "x:name" function; also undocumented is that despite x used to identify which media track the timecode file applies to, the --timecodes "x:name" string must be preceding the track it applies to, with no other tracks in between.

I found out that if it is listed after, the resulting file will not use the timecodes. Slightly offtopic, but despite its many revisions, MKVtoolnix remains sorely lacking in frontend support for timecode files, both in the GUI (no support or documentation) and the cmdline (support but limited documentation).

Your applications and their GUI are useful as well on another point: AVI60 is both impossible to understand due to Japanese labels and documentation, and has a minor defect: the AVI files produced have a trivial structural corruption that prevents their use in your applications. The manifestation of it is that at the end of processing with cfr2tc an error will be reported. The problem is solved by direct stream copy with vdub; the resulting file is slightly different in size (indicating structural change) but has the same number of frames. And being produced by vdub, of course it works with all apps, including yours. Thankfully vdub preserves null frames in direct stream copy mode as well.

Finally, AVI60 has very limited convert-to options - 60 and 119.880 fps only. The cmdline seems to have other options, but I cannot read Japanese. Your apps have an excellent method of framerate input - the native ratio x/y used in the AVI file structure. Thus they are the natural successors to AVI60 provided they can fully emulate AVI60's functionality.

[foxyshadis]

You know, if you're going to mkv anyway you can just add the first and append the rest into mkvmerge and it'll apply their respective timestamps to the resultant mkv. I just tested to be sure and it worked great.

You also did a good job of highlighting exactly why I asked for avi_tc, if you go back in the avisynth VFR page's history you'll see the rather insane procedure that existed for making 120 fps avis with avi60.

[Isochroma]

Yes, you're correct. However using that method is pretty crude and doesn't present the kind of usefulness that vdub or other vfw applications do.

The idea is to get all avi files to the same framerate via insertion of null frames, so they may be edited/joined/subbed together. Long mkvmerge commandlines just don't do it for me, if you know what I mean :)

[Haali]

mkvmerge discards null frames automatically, if you mux a 120fps avi directly to mkv, you get a vfr file in one step.

[Isochroma]

Ah! Thanks for the info, Haali. There are good reasons to generate a file with null frames first, though. Editing in VFW apps is just one of them...

Was playing around with a lowest common denominator calculator (http://www.murderousmaths.co.uk/books/BKMM4xlc.htm) today and noted that it would be practically impossible to insert enough null frames in a 25fps PAL AVI to get it to LCD with a 29.97fps one (5272378157511475 fps required!).

But it seems that with a 0.1% speedup to 30fps, the 29.97fps clip can be made compatible with everything :)

[foxyshadis]

No, the LCM of 25 and 29.97 is still 150fps - realistically, does 1/1000 of a second judder bother anyone? Cameras aren't even that precise. Although a 0.1% desync doesn't sound like a lot, after 30 minutes you already have a 2 second difference.

It's fairly easy to just switch the framerates and resave in vdub, using direct stream copy, compared to AVI60, and you have more choices as well.

[tritical]

I added the ability to simply specifiy an input fps value instead of using a timecode file in tc2cfr. I updated tc-GUI as well.

[Isochroma]

Thanks! You now have my gratitude & thanks x 2...

[Mug Funky]

yo, to shift the thread in a slightly different direction, i noticed that tdeint(mode=2) doesn't seem to support yuy2. no biggie, as i can either use mode 1 or just switch to tivtc.

also, tryweave and emask don't appear to get along with each other. also no biggie - i was planning to use this but ditched it in favour of another method. but thought you'd like to know regardless.

[tritical]

mode 2 should support yuy2 just fine, but there was a little typo causing it to mess up and throw an error. I'll try to put up a fixed version tommorrow. Also, what do you mean by tryweave and emask don't get along?

[Velocity 7]

I'm having trouble using TFM on this particular video (http://naomi.thewafflehouse.net/testvid.avi). A few of the frames are not getting matched up properly; the low motion seems to be screwing up combing, especially near the end of this panning sequence.

Script I'm using at the moment:

LoadPlugin("TIVTC.dll")
LoadPlugin("mpeg2dec3dg.dll")
MPEG2Source("haruhiep2_3.d2v")
TFM(mode=5, pp=5, d2v="haruhiep2_3.d2v")

Here's another example video (http://naomi.thewafflehouse.net/testvid2.avi) which TFM is having trouble with.

[puffpio]

Hello..
I've used your plugins for awhile now and they always do the job.
However I have clip now that I cant solve

It's 60 fps progressive HD clip
there are two types of video inside
1) 24fps video that has duplicated frames to 60fps in a 'aabbb' pattern
2) 60fps pure video with no duplicated frame

Using TDecimate(mode=1, cycleR=3) I can remove the duplicate frames from #1 segments..but the pure 60 fps video then comes out jerky...there is a hybrid parameter, but that is only when cycleR=1, so I can't blend the 60fps to 24fps..

Im gonna expierment with mode 2 and 7...currently on a mode 4 stats generation pass...

any ideas?

[tritical]

@Velocity 7
Thanks for the clips. I'll do some testing and see if I can get tfm to match them correctly.

@puffpio
Currently, tdecimate doesn't have any way to do what you want automatically. It is designed around 24/30 mixtures. Its hybrid operation could be extended to cases where cycleR != 1, but it would require significant changes that I don't really have time to make atm.

[puffpio]

there's always more than one way to solve a problem..got some good help in the other forum tackling the problem in different ways :P

[tritical]

If the clip2 processing is really slow, you could probably get away with using it on only the final pass without any problems. The only thing that could happen is that the difference metrics calculated by tdecimate happen to be changed enough by the pping difference that tdecimate decides to decimate a different frame or switches its decision about video/film for a certain section. Even if that did happen I don't think it would noticable.

[ChiDragon]

tritical:
Would it be possible to add an option to disable the CRC check for input files? I use output and input files in combination with black overlays as a way to IVTC without having moving channel bugs and bumpers interfere with the proper pattern.

I know it's possible to work around the check by deleting the CRC from the metrics file, but that means I can't do batch processing. As a workaround for batching, I've used this sort of thing in both the analysis script and the encode script:
BlankClip(last,length=20)++last
TDecimate()
Trim(16,0)
This obviously won't work for TFM with a d2v input though. It would also be easier and look cleaner to just have a parameter to set instead of adding and trimming video, of course.

Thanks for your time and for making such a great combination of filters, I use TIVTC and TDeint all the time! :thanks:

[CruNcher]

@tritical
can blends like this get better in future versions ?

http://cruncher.mufflastig.com/hdtv/horror.png

[tritical]

@ChiDragon
Yep, I can do that.

@CruNcher
Probably not. Do a separatefields() on that image and you'll see that the individual fields that make up that frame are actually interlaced. Where did that image come from?

[Carpo]

nevermind problem has been sorted

[rig_veda]

I noticed, that when producing VFR with a TDecimate override of for example

0,2729 v
2730,45863 f

my script produced a timecode that went like:

# timecode format v1
Assume 29.970030
2730,19145,23.976024
19146,19148,17.982018
19149,31912,23.976024
31913,31915,17.982018
31916,37234,23.976024

Now i got curious what internally happens to make it so. Why those sections of 3 frames each that use a fps of exactly 17.982018? Why is it always 3 frames and exactly 17.982018 fps and why isn't the output just plain 23.976024 fps for the whole part markt in the override file as film. I'm assuming it's to keep things in sync, but how does it work behind the scene?
And are these 17.982018 fps sections distributed by chance or do they have a reason to appear exactly where they occur?

[Mug Funky]

you'll get that from a 3-in-5 decimation of 29.97fps...

i suppose you've got anime there, and there's some characters moving once every few frames, a few of which happen to fall on the interlaced part of the 3:2 cycle...

[tritical]

The 17 fps parts are from when tdecimate detects an ivtc pattern change that results in 2 duplicates in the current 5 frame cycle that it is looking at. For example:

nndnn nndnd nnnnd

n = new
d = dup

Is one possible scenario that could occur due to a pattern change. IIRC, the specific things tdecimate looks for when trying to detect this and mode=0 (or mode 5 with vfrdec=0) are:

1.) previous cycle has exactly 1 dup (indicated by matches by tfm and that frame has the lowest difference metric in the cycle)
2.) next cycle has exactly 1 dup (same requirements as previous cycle)
3.) current cycle has exactly two dups (indicated by matches by tfm and those two frames have the two lowest difference metrics in the cycle)
4.) the dups in the current cycle are not directly next to eachother
4.) the position of the first dup in the current cycle equals the position of the dup from the first cycle
5.) the position of the second dup in the current cycle equals the position of the dup from the next cycle

for mode = 1 (or mode 5 with vfrdec=1) those same requirements apply plus a check that none of the frames surrounding the two match duplicates in the current cycle, or any other frames in the current cycle besides the two match duplicates, had metrics < dupthresh (i.e. were detected as metric duplicates).

[Chainmax]

I have a bad NTSC-to-PAL conversion that needs reverting to NTSC in order to regain motion smoothness in some slowed-down scenes that look horrible. In order to do that, the source will be bobbed and then decimated. What would be the difference between using TDecimate(cycleR=2003,cycle=5000) and TDecimate(mode=7,rate=29.97) on it?

[tritical]

TDecimate(cycleR=2003,cycle=5000)

This one will actually break the clip up into sections of 5000, get a difference metric for all frames in a section, and then determine which frames in the section to remove.


TDecimate(mode=7,rate=29.97)

This one doesn't work on sections. It calculates an offset into the stream based on the decimation factor and then based on the difference metrics of a few surrounding frames and how the previous frames that were processed up to that point were decimated, decides whether to keep the frame or not. So this one wont have to sit there for long periods processing chunks of 5000 frames, it also will request all frames in linear order.

I would say that if your clip actually has a strict cycle (i.e. the number of dups in every 5000 is evenly spread and doesn't' vary) then mode 0 would be better than mode 7.