TDeint and TIVTC

[foxyshadis]

Tritical, would you mind updating your page with the smartdecimate compile? Just to have someplace to reference besides a lonely post when telling people where to get the latest. ;)

[Chainmax]

I am going to use TDeint+TMMv1 (this one (http://bengal.missouri.edu/~kes25c/mmask.dll), right?) as Clip2 for an IVTC, would this:

AssumeTFF()
Deinted=TDeint(order=1,field=1,type=1,emask=TMM(order=1,field=1))
TFM(mode=6,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)


be the correct way to do so?

[Wilbert]

Tritical, would you mind updating your page with the smartdecimate compile? Just to have someplace to reference besides a lonely post when telling people where to get the latest.
Tritical? I can't find that version in this thread.

[Leak]

Tritical? I can't find that version in this thread.
Try this post (http://forum.doom9.org/showthread.php?p=897598#post897598)... :)

np: Gui Boratto - Beautiful Life (Chromophobia)

[Zarxrax]

I'm having some trouble fully understanding the d2v parameter that tfm takes.

If I set the d2v parameter, then tfm will check the d2v for illegal field order, and create a new d2v if necessary. Also, tfm will use hints from the d2v to help its field matching process? Will this theoretically give more accurate results, or will be be faster?

Next, is it really necessary to specify the d2v parameter manually? Can't dgdecode pass the data to tfm as a hint or something? And also, is there anyway that tfm could simply fix the d2v file on the fly, rather than making a new one and requiring you to go back and change your script?

[tritical]

@Chainmax
The script looks right, but you'll need to use TMMv1 (http://bengal.missouri.edu/~kes25c/TMMv1.zip) instead of mmask.dll for that syntax to work.

@foxyshadis
I'll do that.

@Zarxrax
Next, is it really necessary to specify the d2v parameter manually? Can't dgdecode pass the data to tfm as a hint or something? And also, is there anyway that tfm could simply fix the d2v file on the fly, rather than making a new one and requiring you to go back and change your script?
It would be possible for dgdecode to pass the information to tfm, but dgdecode doesn't currently have this ability. Also, it is slightly easier reading from the d2v and it's faster. TFM could fix the d2v on the fly (meaning it could simply overwrite the existing file instead of creating a new one), but the dgdecode object would still need to be recreated (script closed/reopened) for the changes to effect decoding. My main reason for not overwritting the existing file is I sometimes like to examine the two side by side.

If I set the d2v parameter, then tfm will check the d2v for illegal field order, and create a new d2v if necessary. Also, tfm will use hints from the d2v to help its field matching process? Will this theoretically give more accurate results, or will be be faster?
What exactly happens depends on the 'flags' parameter. With the default setting of 4, tfm will check the d2v for illegal transitions and create a new d2v if necessary. It will also pass rff information to tdecimate. The rff information is only used to help aid hybrid detection (detection of 30p parts), so unless you are using hybrid > 0 it won't effect anything (and if the d2v is just all 2's or 0's it won't effect anything either). tfm will also use the trf flags for field matching in film sections (areas of the d2v with the 012301230123 pattern). That means it will determine what match to use based on the d2v flags. It will then check each of these d2v based match decisions to see if the resulting match is combed or not. If it is, then tfm will go back and run its usual match algorithm. Otherwise, it will use the d2v match. This can give a significant speed up if the video has a lot of sections that follow the 01230123 trf pattern, since it means tfm wont have to run its match algorithm. It can also improve things since tfm isn't always correct in its match decisions.

I will say that if you work mostly with r1 anime, the d2v parameter is usually not that helpful since most r1 anime never has 01230123 flagging (the d2v will just have 0's and 2's, 100% NTSC reported by dgindex). In that case, all that setting the d2v parameter accomplishes is checking for illegal transitions.

[Zarxrax]

Alright, thanks for the clarification.

[ChiDragon]

tritical, can you check this out?

Metrics file ("cap9_TDec.txt") (http://chidragon.thedessie.com/cap9_TDec.txt)
BlankClip(length=80061,pixel_type="YV12",fps=30000,fps_denominator=1001)
TDecimate(mode=2,rate=23.483,input="cap9_TDec.txt",batch=true,display=true)

This gives me a "System exception - Access Violation." Other rate values also cause it. 23.482 works fine, as do these rates with mode=7 instead of 2.

I've also got a request... would it be possible to add standalone "+" to the override options to force a decimated frame to be kept? From what I've gathered through use and TDecimate's doc, "+" is currently only used to separate kept and dropped frames for override cycles (which are hard to write out for hundreds of frames at a time) and doesn't actually override anything itself. Currently I usually just edit my TDecimate metrics files and give frames I want to keep huge values, but it's not really practical, especially when I mess up and want to undo something.

[chipzoller]

Would someone mind taking a look at this sample (http://www.megaupload.com/?d=ZJ2I3WHE). I'm trying to figure out how to setup tfm and tdecimate to restore progressive frames from this apparent 2p4i pattern clip but after reading the help files for both and trying samples/experimenting around with decimation cycles am not sure how to restore this. It actually looks like blended fields, but I know this series (Sherlock Holmes with Jeremy Brett) was shot in film, and all the DVDs in the series except the one from which this episode was taken were just soft pull-downs. Or is this not a candidate for tfm+tdecimate but rather some function like restore24 or the likes? Thanks much for the advice.

[Boulder]

The series were originally shot as progressive 25fps I think. Does RePAL do any good?

[Livesms]

@All
Can anybody suggest good Bob deinterlace
I tried Tdeint as a bob deinterlace for my TFF PAL video
topf = separatefields().selecteven().createMM()
botf = separatefields().selectodd().createMM()
tdeint(mode=1,order=1,emask=buildMM(topf,botf,mode=1,order=1,dis=2)) and got this
http://img5.imagevenue.com/loc385/th_50064_TDeint_122_385lo.jpg (http://img5.imagevenue.com/img.php?image=50064_TDeint_122_385lo.jpg)
Yadif(mode=1) give me better result with straight lines but has lots of artifacts
http://img175.imagevenue.com/loc379/th_50068_Yadif_122_379lo.jpg (http://img175.imagevenue.com/img.php?image=50068_Yadif_122_379lo.jpg)

[chipzoller]

@Boulder,
Yes I think you may be right. It's just that with the majority of the releases in the "Adventures of..." series they did a proper telecine job, but with this one it seems they went the lazy route and just did a blend-conversion. I'm really not "up" on the methods and processes to get PAL to NTSC so I can't speak from a knowledgeable position...In any case, RePal seemed to do a good job, although I had to use convertfps(25) at the end as it was outputting a 24.75 frame rate otherwise. I really wish there were some tutorials out there that gave examples of various field blendings and avisynth scripts/plugins to try and tackle them.

[Boulder]

Instead of ConvertFPS, use AssumeFPS.. then again, Restore24 has a RePAL equivalent which outputs 25fps IIRC. You might want to post in the R24 thread or create a new one in the Avisynth usage forum so it will get noticed by the pros :)

[ChiDragon]

You should probably use AssumeFPS(25,1,true) followed by an audio resample instead of ConvertFPS. Bump up the video framerate and alter the audio pitch slightly instead of adding blends to your unblended video...

EDIT: Sorry, I see Boulder already said that. I had a cached version open.

[chipzoller]

Thanks ChiDragon. I'll try AssumeFPS with those params as AssumeFPS(25) produces out of sync audio. However since I'm dealing with a demuxed DVD source, how would, once you demux the audio track, you get AssumeFPS and avisynth to consider and alter the AC3 file?

[manono]

@Boulder,
In any case, RePal seemed to do a good job, although I had to use convertfps(25) at the end as it was outputting a 24.75 frame rate otherwise.
What's the matter with 24.975fps? Is it any more peculiar than 23.976? If for AVI, keep it there. If for DVD, run the MPV through DGPulldown with the Custom box ticked and 24.975->29.97 filled in. That way you can keep the audio unchanged. ConvertFPS just reblends what RePAL unblended, and AssumeFPS, as you discovered, requires stretching the audio.

If you are going to use AssumeFPS and stretch the audio, you may as well make it for 23.976fps if, as you say, it was shot on film.

[chipzoller]

Manono, these are destined for my iPod and it needs a frame rate of PAL, film, or ntsc to comply with spec.

[manono]

OK, but if I'm not mistaken, I believe that's the first time you've mentioned requiring specific framerates.

Me, I'd make it 23.976fps (that's what you mean by film, isn't it?), and stretch the audio to do away with the PAL speedup and higher pitched audio. If you want to keep the same audio, then you can do a ChangeFPS(25) after RePAL, which will add one duplicate frame every 1000 frames, or 1 every 40 seconds, to produce a slight stutter which may or may not be noticeable.

[ChiDragon]

However since I'm dealing with a demuxed DVD source, how would, once you demux the audio track, you get AssumeFPS and avisynth to consider and alter the AC3 file?

I use NicAC3Source, part of NicAudio (check http://avisynth.org/warpenterprises/). You could also use AC3Source or DirectShowSource with an AC3 codec/filter installed. The AC3 wll be decompressed to uncompressed PCM of course.

[laserfan]

I know this is a Purist thread, in a (mostly) Purist forum, but I'm looking for some simple advice in handling conversions of my HDTV captures to Xvid. I routinely capture ABC TV shows that have been converted by the local affiliate station from the network's original 720p to 1080i, and then sometimes they are also time-compressed. Bottom line is they started life as Film and along the way get messed with--they're mostly 3-2 but then sometimes 3-2-2-2-2-1-3-2-3-2 etc w/no repeating pattern. 95% of the time my conversions look great, or at least "good enough for me" using this:

tfm().tdecimate(hybrid=1)

On rare occasions the above doesn't do quite as good a job as I might've hoped (the 23.976 result is more herky-jerky than I'd like) and I wonder if there is a new-or-better script I can or should use for these captures.

Bear in mind please that I don't want to analyze/count frames for each & every conversion I do--I'm not looking for perfection; I just want to choose a "good enough" method (a compromise) and stick with it if possible. TIA for any advice...

[Mug Funky]

@ chipzoller:

if something was shot on film for TV in PAL land, the cameras will have been run at 25fps as there is no reason to run them at 24. so i'd assumefps, and be assured the audio pitch will be correct as it was shot at that speed to begin with (and nobody would bother with a timestretch for a 1/1000 difference in speed).

it's possible this episode was either never aired in NTSC land, or a tape could not be found and thus a conversion was done off a PAL master.

[laserfan]

tfm().tdecimate(hybrid=1)
...is [there] a new-or-better script I can or should use for these captures.Hmmm, I wonder why no one has an opinion about this? I didn't think my question was too off-the-wall?

[ChiDragon]

laserfan,

EDIT: Nevermind, I tried my suggestion and it doesn't work... your script is the best I can think of, of the solutions that don't require manual intervention.

[Atak_Snajpera]

TDeint() generates strange artefacts.
Source is 1440x1080i (Canon HV20)
http://www.archive.org/download/jvc_hd7/hv20_05.m2t.mpg

After TDeint() look what happens

http://img524.imageshack.us/img524/3817/tdeintfx5.th.png (http://img524.imageshack.us/my.php?image=tdeintfx5.png)

I've also uploaded sample after conversion to 1280x720 (x264)

http://www.megaupload.com/?d=71Z12QI0


Another example:
Source: http://www.archive.org/download/jvc_hd7/hv20_07.m2t.mpg

Once again strange spots show up
http://img517.imageshack.us/img517/4987/tdeint2cd3.th.png (http://img517.imageshack.us/my.php?image=tdeint2cd3.png)

I used very simple script

LoadPlugin("C:\Users\Dawidos\Documents\Delphi_Projects\RipBot264\Tools\dgindex\DGDecode.dll")
MPEG2Source("c:\TEMP\job1\job1.d2v")

tdeint()
LanczosResize(1280,720)

[laserfan]

...your script is the best I can think of, of the solutions that don't require manual intervention.Thanks for that--I just had to ask, given the many, many pages of enhancements discussed here, most of which I just don't understand. :(

But thanks for responding! :)

[tritical]

Almost ready to start working on tivtc/tdeint again. I read back through the last few pages and this is the list of current issues:

tdecimate mode 2 problem non-standard framerates - Xesdeeni/ChiDragon
tdecimate problem with mode 0 decimation - ChiDragon
add tfm hints to tdeint - ChiDragon
tdecimate '+' override - ChiDragon
tdecimate problem with video section decimation in mode 5 (starting in version 0.9.12.7) - Asrial

If there are any other items I've forgotten please let me know :).

[JuanC]

Well, now that you mention other forgotten items, there's this old request (http://forum.doom9.org/showthread.php?p=821397#post821397), for an exclusion band for Tdecimate. Not really a "current issue" but it will certainly help people like me dealing with 30fps interlaced subtitles blended on top of 24fps hard-telecined material. Thanks ;)

[tritical]

An exclusion band in tdecimate could be accomplished by using the clip2 parameter:

mpeg2source()
tfm()
saved = last
stackvertical(crop(),crop()) # set crops to remove lines containing subtitles
tdecimate(clip2 = saved)

In this case, tdecimate will calculate everything based on the input clip, but will output frames from clip2. The clip2 clip only has to have the same number of frames as the input. All other properties: height, width, and colorspace can be different. You could replace cropping/stackvertical with another method of removing the subtitles (set the subtitle area to black using overlay, etc...).

It would be possible to add y0/y1 parameters to tdecimate and have it do the above internally, but I don't think it's worth it. Also, actually having y0/y1 parameters that are taken into account during metric calculation would be too much of a pain to add atm. However, it is on the list of things to add in the tdeint/tivtc rewrite.

[DarkNite]

However, it is on the list of things to add in the tdeint/tivtc rewrite.

Did I miss something, or have you been working on a whole new code base in your secret other life while cackling like a mad scientist who just built two tesla towers with just enough space for an operating table between them? ^_^

tdecimate problem with video section decimation in mode 5 (starting in version 0.9.12.7) - Asrial

Thank you for adding that to your current to do list.

[max24]

How is one this SOURCE Deinterlacen?
I despair of it :helpful:

Sampler:
http://rapidshare.com/files/31967235/Sampler.m2v.html
http://simpleupload.net/download/88293/Sampler.m2v.html

Thanks for your Help

[JarrettH]

http://forum.doom9.org/showthread.php?t=125990

Maybe I should ask this here instead.....

I'm trying to encode Pan's Labyrinth and MeGUI detects it as Partially Film (correct) with Top Field First. What's the difference between choosing TIVTC alone vs TIVTC+TDEINT?

:thanks:

[ChiDragon]

TFM by default will use PP=6 which is motion-adaptive cubic interpolation on frames detected as combed. TFM+TDeint by default will use TDeint's different motion masking and kernel interpolation instead of cubic.

Maybe you should post of a sample of the unprocessed VOB in a section where you say TIVTC creates jerkiness.

[JarrettH]

I think I figured it out. So apparently clicking analyze will make a different avs script than if I were to choose Partially Film + Top Field + TIVTC manually.

If I select myself:
tfm(order=1).tdecimate(hybrid=3)

If I click analyze
tfm(order=1).tdecimate(hybrid=1)

:devil:

[ChiDragon]

hybrid=1 is for mostly film (blends 30p->24p), hybrid=3 is for mostly video (blends 24p->30p).

[JarrettH]

It definitely did fix it though. The stream is 23.97 fps and looks proper. So TIVTC+TDEINT should produce better quality?

Thanks Chi:)

[ChiDragon]

@max24, except for 2 shots of the plane in the opening (before and after the kangaroo at the end), that is just plain old easy field shifting. Use "TFM()" or even "TFM(mode=0,PP=0)". The two plane shots mentioned have some duplicated fields which will cause the video to be a bit jerky there, but they're short sequences anyway.

[ChiDragon]

JarrettH, I believe tritical would agree that TDeint's deinterlacing is superior to what's built into TFM yes.

[JarrettH]

It probably is, but I was bored and made some captures :devil:

I think this movie only begins to have interlacing problems near the end. Three shots, one TIVTC alone the other TIVTC+TDEINT

If anyone wants to compare:

http://rapidshare.com/files/32249715/TIVTC_and_TDEINT.rar.html

Thanks!

[JarrettH]

http://img503.imageshack.us/img503/5196/tivtc1px1.png

http://img128.imageshack.us/img128/2616/tivtctdeint1kg5.png

http://img264.imageshack.us/img264/2695/tivtc2qk7.png

http://img503.imageshack.us/img503/2765/tivtctdeint2ue0.png

http://img264.imageshack.us/img264/46/tivtc3hp1.png

http://img264.imageshack.us/img264/4259/tivtctdeint3sh4.png

Top one in each set is TIVTC alone, bottom is TIVTC+TDEINT

[foxyshadis]

I don't think the TFM clip2 parameter is working as it should. I've been using:

clipt=TDeint(field=1,emask=TMM(field=1),edeint=separatefields.selecteven.eedi2(field=1))
clipb=TDeint(field=0,emask=TMM(field=0),edeint=separatefields.selectodd.eedi2(field=0))
TFM(clip2=clipt,pp=4)

But the frames I get back are exactly the same as with TFM(), and very different from clipt. I haven't had any luck figuring out why.

[tritical]

@DarkNite
You wouldn't happen to have a clip that results in different mode 5 output (latest version vs. older versions) would you? I've been trying to duplicate the issue but haven't been successful. I pm'd Asrial a couple days ago about getting a clip, but I don't think he visits very often.

@foxyshadis
Thanks for the report. I'll try to duplicate it.

[foxyshadis]

Never mind, I just didn't understand the docs well enough. I didn't realize that it only uses clip2 for combed frames, so I was checking the wrong frames, but using a much lower cthresh for testing forced it to work. Sorry!

[DarkNite]

I don't recall exactly what portion of which source it was atm (I have 6 projects going on now), but for you I will attempt to hunt it down in my free time.

[tritical]

@DarkNite
Thanks, though if you don't remember where it was don't worry about it too much... I'm sure you have better things to do with your free time :).

@All
I have managed to fix the mode 2 problems. The drifting on long sequences that Xesdeeni reported was caused by having a '>' instead of a '>='. The crash ChiDragon reported was caused by a bug in the sorting algorithm that was used for very large cycles (could only happen when using mode 2 with an input file and non-standard framerates). I also made some minor improvements.

I'm still trying to duplicate the mode 5 problem. I've been comparing TIVTC v1.0 RC1 vs v1.0.1 on a few hybrid sources and have yet to see any major differences. Although, I did find a bug in tfm's cubic pping. The handling of the very last line (for field=1) and very first line (for field=0) has been slightly messed up since ~ April, 06.

[Terranigma]

tritical, any new news on EEDI (http://forum.doom9.org/showthread.php?p=966991#post966991)'s successor? How is it progressing? :cool:

[Livesms]

I don't think the TFM clip2 parameter is working as it should. I've been using:

clipt=TDeint(field=1,emask=TMM(field=1),edeint=separatefields.selecteven.eedi2(field=1))
clipb=TDeint(field=0,emask=TMM(field=0),edeint=separatefields.selectodd.eedi2(field=0))
TFM(clip2=clipt,pp=4)

But the frames I get back are exactly the same as with TFM(), and very different from clipt. I haven't had any luck figuring out why.

Can you explain this script :) Please :)

[tritical]

@Livesms
clipt/clipb are just tdeint deinterlaced versions of the original source... clipt keeping top fields and clipb keeping bottom fields. In both cases TMM() is being used to build the motion mask instead of tdeint's built in motion masking, and eedi2 is being used to interpolate pixels instead of one of tdeint's built in interpolation methods. One of those two clips is then being used by tfm's clip2 parameter (in this example it is clipt). That means that when tfm detects a combed frame combing out of field matching it requests that frame from clipt instead of deinterlacing it itself. When pp is set < 5, the frame from clip 2 will be delivered as is. If pp is >= 5 then tfm will still apply its own motion masking.

@Terranigma
It's doing quite well. I haven't commented on it for a while because I'm planning on using the idea as part of my master's thesis, which means that a public binary is still a ways off.

[foxyshadis]

Maaaaaaaaan, that would be nice to see indeed. ;_;

The reason I used two different tdeints, if you're curious, was because I had a filter in between that would swap in frames from the other (a quick hack of fizick's badframes). It was an old VHS capture and had an annoying amount of partially smeared lines that could be mostly fixed by interpolating over the other field. I got sick of finding bad frames pretty quickly, though.

[Terranigma]

tritical, thanks for keeping me informed. I can only imagine the possibilities this new filter will be able to achieve (Specifically when it comes to aliasing/deinterlacing). :D

[audyovydeo]

@tritical

This is just a word of appreciation as I really like TDeint. I've read around the forums and seen some speed comparisons with other deinterlacers. I dont do comparative testing as it's precious time taken off real encoding.
I find TDeint is a real workhorse : I have two avs scripts each TDeinting two DV_PAL.avi files and layering them, both scripts fed into Adobe Premiere for yet another transparency, and it works like a beauty (on a T5200 cpu). That means very good workload scaling, which to me is more important than absolute speed.

So, for me, every minute you spend on multithread optimization is wecome.

cheers
audyovydeo