Confused about telecine, IVTC, deinterlace, telecide and decimate.

[mizumi]

I'm currently reading the tutorial "Force Film, IVTC and DeInterlacing" (http://www.doom9.org/ivtc-tut.htm).

I'm not exactly a newbie but I would like to get to the core and understand the scrypts I write to improve the quality of my avis. So I'm reading how to encode right from the beginning.

I use raw notepad for scripts, virtualdubmod and avisynth 2.51 for encoding.

So my first doubt arises when trying to understand how I can know if a NTSC DVD has been telecined or not.

Checking the tutorial I've noticed two scripts which I don't see the difference for:

One is this one:

A TRULY INTERLACED SOURCE CANNOT BE IVTC’D.
Unfortunately for us, DVD2AVI can not distinguish between bad telecining and genuine interlace. They will both appear in the statistics window as NTSC. One way of finding which is which is by using DVD2AVI to approximately find the NTSC parts of the film. Then use the following .avs to check the frames in Virtual Dub:

LoadPlugin("mpeg2dec.dll")
LoadPlugin("Decomb.dll")
MpegSource("clip.d2v")
Telecide(post=false)


And then, downwards, there's this one:


IVTC ADVICE
FOR NTSC PEOPLE: FORCE FILM WHEN YOU CAN, IVTC WHEN YOU CAN'T FORCE FILM, AND DEINTERLACE WHEN YOU CAN'T DO EITHER.

FORCE FILM. Most advice says to Force Film when the percentage in DVD2AVI shows as greater than 95%. Have you ever done an encode using Force Film at 99% and still found interlaced frames when done? I have -- many times. Force Film is not guaranteed to work perfectly unless it shows as Film (=100% Film). There may be a few interlaced frames, or the video may play jerky at a few spots. These interlaced frames may be in the pre-movie stuff such as the studio logos, they may be at chapter breaks, or they may be at other places where edits were done after the telecine process. Most of the time Force Film will work fine at those high percentages, but not always. When in doubt, create the .d2v with No Field Operation (29.97fps) [video>Honor Pulldown Flags in newer versions], and then use:

Telecide(Guide=1,Post=False)
Decimate(Cycle=5)

And that brings me to my next doubt: is there any difference between the 2 scripts in the tutorial?

As much as I try to see the difference, all I can understand is that both of them are useful to see if the DVD has been telecined or not. But I guess I'm not getting the clue.

This is what I've understood: interlacing is used in the process of telecining to get 29.97fps out of 23.976fps.

Thank you in advance!

edit: I erased the last part of the post which had another doubt, as I think I should open a different topic for it. It does not affect the answer from stereodude, which was for the first part :)

[Stereodude]

If you're dealing with a theatrical released movie on DVD it's very, very, very likely 23.976fps progressive content that had a pull down / telecine applied to it. As such you don't need to IVTC. Just used force film.

No those two are not identical. The idea with the first is to visually inspect the output and see if you get out progressive frames with a duplicated frame every 5th frame or still get interlaced output. The second one assumes there is an IVTC and removes the extra frame with decimate. If you want to know if material has an IVTC applied to it, use telecide with guide=1 and show=true and see if it finds a pattern (reported in the information put on the output video) as well as looking at the output as I mentioned.

Also, don't use old versions of the software. You should use AVIsynth 2.5.8. You should use DGdecode (http://neuron2.net/dgmpgdec/dgmpgdec.html) instead of DVD2AVI.

[mizumi]

First of all, thank you for your reply! :)

Well, I’m working with cartoon series dvds, so that case won’t likely be pure 100% progressive, will it?

LoadPlugin("mpeg2dec.dll")
LoadPlugin("Decomb.dll")
MpegSource("clip.d2v")
Telecide(post=false)

The idea with the first is to visually inspect the output and see if you get out progressive frames with a duplicated frame every 5th frame or still get interlaced output.

So this script is used to see if the DVD has interlaced material?
Let’s say I still get an interlaced output after applying the script. What should I do next: applying a deinterlace codec?


Telecide(Guide=1,Post=False)
Decimate(Cycle=5)

The second one assumes there is an IVTC and removes the extra frame with decimate.


So what can I use this script for?


If you want to know if material has an IVTC applied to it, use telecide with guide=1 and show=true and see if it finds a pattern (reported in the information put on the output video) as well as looking at the output as I mentioned.

Thank you for this script, I will try it!
So I should use
Telecide(Guide=1,Show=true)
(with no decimate script below, right?)

But... I don't understand which pattern should I look for, and where it will be reported. Could you help me with that please?
You said I do look for the pattern and also look at the output of the video: are these 2 different things? :confused:

Sorry but these scripts concepts are difficult for me to understand although I'm trying my best! So I really appreciate your help! :thanks:

[Stereodude]

Well, I’m working with cartoon series dvds, so that case won’t likely be pure 100% progressive, will it?No, and it may not have a clean pulldown pattern applied to it either.
So this script is used to see if the DVD has interlaced material?
Let’s say I still get an interlaced output after applying the script. What should I do next: applying a deinterlace codec?You would remove the Telecide line and replace it with a deinterlacing filter.
So what can I use this script for?NTSC content that has a 3:2 pulldown applied to it where you want to recover the original 23.976fps progressive frames.
Thank you for this script, I will try it!
So I should use
Telecide(Guide=1,Show=true)
(with no decimate script below, right?)Yes, assuming you're dealing with a NTSC DVD and believe the content has a 3:2 pulldown on it and you want to confirm.
But... I don't understand which pattern should I look for, and where it will be reported. Could you help me with that please?
You said I do look for the pattern and also look at the output of the video: are these 2 different things? :confused:Yes they are two different things. When you have show=true you get information from Telecide printed in the upper left corner of the frame. Like these:

http://img259.imageshack.us/img259/8654/telecideexample1.png

http://img718.imageshack.us/img718/4894/telecideexample2.png

http://img8.imageshack.us/img8/2451/telecideexample3.png

I used Telecide(guide=1, post=1, show=true)

Note the "pattern mismatch=100.00%" and "out-of-pattern" message. That means the material does not have a 3:2 pulldown applied to it (as specified by guide=1). In the first one the mismatch is 0.00% and in-pattern, which means it is good to process with guide=1.

Keep in mind...
guide=0: blind field matching
guide=1: NTSC 24fps->30fps telecine guidance
guide=2: simple PAL guidance (tries to maintain lock to the field phase)
guide=3: NTSC 25fps->30fps telecine guidance

If you confirm that guide=1 works, change the telecide line to the following, and add the decimate line.

Telecide(guide=1, post=2, show=false)
Decimate(mode=0, cycle=5)

[mizumi]

Wow, thank you so much! You've given me tons of information. That's lots of new things for me to learn.
So I will do my best and let you know the results! Thank you!

[Amateur]

oh wow this is GREAT! i'll be referencing this for some time i imagine.

question though. you mention what all of the guide values are for but what are the different post values?

The idea with the first is to visually inspect the output and see if you get out progressive frames with a duplicated frame every 5th frame or still get interlaced output. The second one assumes there is an IVTC and removes the extra frame with decimate. If you want to know if material has an IVTC applied to it, use telecide with guide=1 and show=true and see if it finds a pattern (reported in the information put on the output video) as well as looking at the output as I mentioned.

is telecide simply to determine if there is a pulldown or not and doesn't actually change anything?

[osgZach]

If you were to save your video as a new encoded video, you would get a modified stream based on the settings use in Telecide.

Used with Show=True, it is just a visual aid people use for previewing in Virtualdub, to help them decide how they want to proceed.

I haven't used Telecide in a while, but I think the "post" options you asked about are related to post-processing and how Telecide should handle (or not handle) the results it gets.. i.e if a frame still comes out combed, deinterlace it, and that sort of thing..

This should be covered in the documentation for Telecide, which if you have the plugin, you should also have...

[Stereodude]

question though. you mention what all of the guide values are for but what are the different post values?Per the documentation: post=0: Use this to totally disable postprocessing.
post=1: Use this to enable the metrics calculation and to display the vmetric values but not perform deinterlacing.
post=2: Use this to enable deinterlacing. Note that in this mode, the field matching occurs normally and the best matched frame is deinterlaced and delivered.
post=3: This is the same as post=2 except that the deinterlacing motion map is displayed in white on the deinterlaced frames.
post=4: This is the same as post=2 except that instead of using the best field match for the frame, the original frame is deinterlaced. You would use this to pass video sequences through when you have a hybrid clip.
post=5: This is the same as post=4 except that the deinterlacing motion map is displayed in white on the deinterlaced frames. is telecide simply to determine if there is a pulldown or not and doesn't actually change anything?No, telecide changes the video by rearranging the fields. If you've got an input video with a pulldown applied to the video you get out progressive frames with a redundant frame at a regular interval. Decimate is what takes removes the redundant frame. I'm not exactly sure what it does to input video that does not have a pulldown applied to it, but I'm pretty sure telecide doesn't pass the video intact.

[Amateur]

the extras for most of my shows are giving me the most trouble regarding this. they don't seem to follow the same rules as the show itself. here (http://www.mediafire.com/?i7dkx9v0enjm4cj) is a portion of the source.

i tried that Telecide(guide=1, post=1, show=true) to determine if it would work but i get mostly 100% mismatch so i can't use that.

i then went to the megui analysis tool which tells me it's hybrid film/interlaced mostly interlaced and gives me tfm(order=-1).tdecimate(hybrid=3) but that doesn't solve it for me

i then went straight to dgindex and tried force film but that didn't work.

is it possible to turn this into 24fps progressive?

[Stereodude]

Uh... That clip is a disaster. You definitely can't turn it into 24fps progressive. A lot of it is 29.97fps progressive already, but it has some interlaced video superimposed on it. It doesn't de-interlace particularly well either with the few AVIsynth filters I tried (FieldDeinterlace & TomsMoComp).

The best results I got turning it into progressive 29.97fps video was to use the hardware Nvidia PureVideo Deinterlacer via DGDecNV. It seemed to leave the progressive portions alone and only deinterlace the interlaced portions of the frames.

LoadPlugin("C:\HDTV Tools\DGDecNV\DGDecodeNV.dll")
DGSource("partial.demuxed.dgi", deinterlace=1)

This however, requires you have a supported Nvidia video card (which you may not have) and pay the $15 for a key for DGDecNV.

[Amateur]

i have radeon 4830 so i guess your solution won't work for me. looks like i'll just leave it as progressive and deal with the imperfections of the visible interlacing. at least it wasn't a simple fix that i simply overlooked. thanks for trying for me though.

[Stereodude]

There probably is a suitable deinterlacing filter for AVIsynth that can clean it up. I didn't exactly go researching them.

There are lots to try (http://avisynth.org/mediawiki/External_filters#Deinterlacing).

Try a few out and see what happens.

[Stereodude]

Looks like i'll just leave it as progressive and deal with the imperfections of the visible interlacing. at least it wasn't a simple fix that i simply overlooked. thanks for trying for me though.TDeint seems to work about as well as the nVidia hardware solution.

tdeint(order=1, mode=0)

Try that and see what you think.

[manono]

You'll probably want to use a deinterlacer that leaves the progressive frames alone (of which there are many in that sample) and deinterlaces only the interlaced ones. TDeint is good, and I'd recommend TDeint(Full=False) rather than letting it deinterlace everything.

[Amateur]

so would i use only tdeint and no ivtc? or a combination? and do i combine both suggestions to make a line that looks like TDeint(order=1, mode=0, full=false)?

[manono]

You can't IVTC it because large chunks of it are pure interlace. You have to keep it at 29.97fps. You can only InVerse TeleCine that which has been telecined.
...do i combine both suggestions to make a line that looks like TDeint(order=1, mode=0, full=false)?
If you like. The output will be the same whether you do or you don't. You could always read the TDeint doc and see what the various parameters are and what they do.

[mizumi]

I used Telecide(guide=1, post=1, show=true)

Note the "pattern mismatch=100.00%" and "out-of-pattern" message. That means the material does not have a 3:2 pulldown applied to it (as specified by guide=1). In the first one the mismatch is 0.00% and in-pattern, which means it is good to process with guide=1.


I used the script above you gave me and this is the result:

http://img691.imageshack.us/img691/1908/pruebatelecineimg1.png

http://img143.imageshack.us/img143/2336/pruebatelecineimg2.png

http://img832.imageshack.us/img832/7312/pruebatelecineimg3.png

Mostly, I got the message mismatch=0’00%, progressive, in-pattern.
A few “mismatch=0’00%, interlaced, in-pattern” and fewer “progressive, out-of-pattern” with mismatch ranging from 1% to 80%.

Also, I checked the output video, as you said:
see if you get out progressive frames with a duplicated frame every 5th frame or still get interlaced output

And couldn't find a duplicated frame or a fix pattern. The footage is an anime with little movement, so maybe there's 4 frames in a row that are the same image, so I don't get to see if one of those 4 is duplicated, as there is the same one x4. What I can observe is that every few progressive frames there's 1 or 2 interlaced frames: there's a little interlacing in the part of the eyes or the hands - the part of the character that moves from one frame to the next one. In fact, there's a sequence with "a lot of movement" meaning that every frame is different and almost every frame has a little interlacing in the part where the character is moving.

Since it's an old anime from a japanese dvd, and the message from the script mostly says "mismatch=0’00%, progressive, in-pattern" I guess this is telecined but I''m puzzled about the interlacing and why the pattern is not "2 interlaced frames 3 progressive frames"... I can upload that bit of the video if you want to see how it looks like.


I've been reading the decomb tutorial and it left me wondering about some things. If you could help me out again I would be very happy :)

1) The difference between using post=0, post=1 or post=2 in the script you gave me to see if there is telecine: Telecide(guide=1, post=1, show=true).

The decomb manual says to use post=0

AviSource("your_clip.avi")
AssumeTFF()
Telecide(guide=1,post=0,show=true)

Serve this into VirtualDub and step through several typical sections of your clip. If Telecide() reports "in-pattern", or "in-pattern*", then it is locking onto 3:2 pulldown. If it reports "out-of-pattern" or continuously jumps in and out of pattern lock, then it is not locking onto a consistent 3:2 pattern.


but you suggested to use post=2 (default)
If you want to know if material has an IVTC applied to it, use telecide with guide=1 and show=true and see if it finds a pattern

and post=1:
Telecide(guide=1, post=1, show=true)


I would like to know if it was just a mistype or if there's a difference, if it's all the same for the purpose of seeing if there's telecine...


2) What's the utility of post=3 and post=5 and the white map thing?
And if it's better to use post=2 or post=4 most of the time... I guess post=2 since it's the default value?

3) How can i set the proper values for vtresh and dtresh?
Regarding vtresh, going back to the pattern being shown with Telecide(guide=1, post=1, show=true), most of the time the "chosen value" in the vmetrics row would range from 10 to 25. Sometimes being 25-30 and every now and then raising above 30 (68, 75, 50, etc). I would say vtresh=20 or 25?? No idea about what dtresh value should I use. I guess here using post=3 or post=5 - the white map thing - would be helpful maybe?

4) Since you use telecide because you think there's a telecine in a footage, what's the point of using guide=0 in a video? Using guide=0 (blind field matching) implies I'm not sure if the footage is telecine, doesn't it?

Thank you very much!

[Stereodude]

I guess this is telecined but I''m puzzled about the interlacing and why the pattern is not "2 interlaced frames 3 progressive frames"... I can upload that bit of the video if you want to see how it looks like.Based on your description I'd safely say it is not telecined.
1) The difference between using post=0, post=1 or post=2 in the script you gave me to see if there is telecine: Telecide(guide=1, post=1, show=true).Post = 1 will cause it to tell you if it thinks the frame is progressive or interlaced, but it won't do anything to the frame.
The decomb manual says to use post=0Visually this is the same as 1, but you don't get the analysis of each frame.
but you suggested to use post=2 (default)If you're actually trying to use decomb post=2 will remove any residual combing that might appear during fades or transitions if the content was edited after the telecine was applied to the source material.
and post=1:This is for analysis.
2) What's the utility of post=3 and post=5 and the white map thing? not much unless your really trying to manually tweak the parameters.
3) How can i set the proper values for vtresh and dtresh?
Regarding vtresh, going back to the pattern being shown with Telecide(guide=1, post=1, show=true), most of the time the "chosen value" in the vmetrics row would range from 10 to 25. Sometimes being 25-30 and every now and then raising above 30 (68, 75, 50, etc). I would say vtresh=20 or 25?? No idea about what dtresh value should I use. I guess here using post=3 or post=5 - the white map thing - would be helpful maybe?Not sure. I've never really messed with manual adjustments.
4) Since you use telecide because you think there's a telecine in a footage, what's the point of using guide=0 in a video? Using guide=0 (blind field matching) implies I'm not sure if the footage is telecine, doesn't it?

Thank you very much!I'm not sure why you'd want to use guide=0 instead of looking for a pattern.

[manono]

I guess this is telecined but I''m puzzled about the interlacing and why the pattern is not "2 interlaced frames 3 progressive frames"... I can upload that bit of the video if you want to see how it looks like.

There are other choices besides telecine and pure interlace. And nothing beats a sample, especially from a section where every frame is different and unique. 10 seconds of an M2V should be plenty.

[Amateur]

so a show that doesn't largely display in-pattern should switch to guide=0 for blind matching?

is telecide a downloaded filter? i don't seem to have a guide or a filter named telecide yet i'm able to use it

[manono]

Telecide, Decimate, and FieldDeinterlace are part of the Decomb collection of filters.
so a show that doesn't largely display in-pattern should switch to guide=0 for blind matching?
Read the manual. Me, I've never had to use anything but Guide=1, 2, or 3.

[Amateur]

ah thanks...i found it. i'll be sure to look stuff up

[mizumi]

There are other choices besides telecine and pure interlace. And nothing beats a sample, especially from a section where every frame is different and unique. 10 seconds of an M2V should be plenty.

I have uploaded a sample for you to see: http://www.megaupload.com/?d=4XG9M1KA
Thank you!

[manono]

You said there were passages where every frame was different and I asked you to provide a sample of one such section. But you didn't. It has field-blending and no standard IVTC will work well with that kind. Try:

Yadif(Mode=1)#or the bobber of your choice
SRestore(FRate=23.976)

http://avisynth.org/mediawiki/Srestore

http://forum.doom9.org/showthread.php?t=95924

The better (and slower) the bobber used, the better the results. Yadif is an OK and very fast bobber, but there are better ones. Also, the video 'shakes' so you might also want to use Stab on it, or Depan:

http://avisynth.org/mediawiki/External_filters#Stabilization

[Amateur]

some of my sources don't exactly follow the previous mentioned in-pattern detection which always threw me for a loop. so if there are a few progressive frames and then an interlaced-looking one but it doesn't follow 3:2 pattern does that make it field blended? how do i really differentiate?

also, you mention yadif is quick but there are other, better options. care to name them so i could look into using them. i know i have yadif but if there are better ones i would prefer to use them. initial slowdown of the encode doesn't matter as much to me as the final result that i will be watching more than once.

[manono]

Well, if you're talking about anime, there are so many duplicates that it doesn't often follow the pattern. Try to find a section where every frame is different and then check for the 323232 pattern. How do you determine if it's field-blended? You separate the fields (SeparateFields()) or you bob it (Yadif(Mode=1)), and you have a look. If you see some of the fields have double images/blends/ghosts, then it's field-blended. Study that sample mizumi uploaded. Ordinarily field-blended stuff comes from a PAL source poorly converted to NTSC (or film badly converted to PAL). This one was field blended from film to 29.97fps, something you don't often see and something only done by the shoddiest fly-by-night DVD production companies.

[Amateur]

so is it more of a matter of the cartoon tricking the filter into thinking it's not following the pattern because of the abundance of duplicate frames? i get mainly in-pattern readings but i will get out-of-pattern in short spurts here and there.

[manono]

I already said it can't be IVTC'd successfully. The IVTCs first field match followed by removing the dupes. Here there are no fields to match, or relatively few. Here the blended fields are removed followed by the decimating.

[Amateur]

alright i have my first PAL source that i am working on. I got The Tick animated series from amazon uk because it has an episode missing from the U.S. release and also is supposedly the full show instead of the syndicated show. also all three seasons were released onto dvd instead of just the first two.

anyway, here is a sample source (http://www.mediafire.com/?122itsf2y8xj6zp).

i put info() into the script to see what it showed for the source and i see the following:
ColorSpace = YV12
Frames per second = 25
Fieldbased (Separated) Video = No
Parity = Top Field First

so, i tried Telecide(guide=2, post=1, show=true) ##i used guide=2 since source is PAL## but that gave mostly out-of-pattern results. i tried guide=3 since it is 25fps but i also got mostly out-of-pattern results. I tried using just Yadif but I end up with frames like so:

http://img51.imageshack.us/img51/6384/31096936.jpg
http://img3.imageshack.us/img3/2483/11014112.jpg
http://img684.imageshack.us/img684/7681/45608412.jpg
http://img855.imageshack.us/img855/8319/60650863.jpg
http://img202.imageshack.us/img202/1772/78912857.jpg

is it possible to get rid of these mixed frames (frames 2 and 4)?


edit: just noticed that this pattern doesn't happen through the entire episode. some parts seem progressive and are fine.

[manono]

The sample is field-blended from an NTSC source. You need to unblend it:

Yadif(Mode=1)#or the bobber of your choice
SRestore()

http://avisynth.org/mediawiki/Srestore
http://forum.doom9.org/showthread.php?t=95924

[Amateur]

i have a show here (http://www.mediafire.com/?4h8ne4no71e2inv) and i'm not sure how to treat it. I tried using the suggested Telecide(guide=1, post=1, show=true) to determine if it had 3:2 pulldown. it would flicker back and forth saying in-pattern and out-of-pattern. So, I decided to manually go through some chunks to see if there was a pattern and it was just getting tricked since it was a cartoon with many repeated frames. I would get chunks that appeared to be 3 progressive:2 interlaced but i would also run across some runs where each frame was different.

telecide provided this output
http://img233.imageshack.us/img233/985/57950712.png

yadif provided this output
http://img850.imageshack.us/img850/7836/85834613.png

is this not fixable? i would hate to think that a Spielberg cartoon was treated this badly for its DVD release.

[manono]

You're screwed. You've got 4 fields in a row, in every 10 field cycle where you have that ghosting. There's no way to get rid of it all.
i would hate to think that a Spielberg cartoon was treated this badly for its DVD release.
It's horrible. You've got interlaced content encoded as progressive. That'll wreak havoc with flag-reading DVD players outputting to progressive displays (HDTV sets). This is better than what you're showing us:

TFM(Chroma=True)
TDecimate()

But I had my best results with this, where the ghosts/double images seem more faint (unless it's just my imagination):

Yadif(Mode=1)#or the bobber of your choice
BlendBob()
TDecimate()# or Decimate()

[dvdboy]

I'm seeing more and more examples of this kind of PAL Pulldown:

http://www.mediafire.com/?96l95e6ramd41e6

I assume it is a variation on this:

Although the 4% speed increase has been standard since the early days of PAL and SECAM television, recently a new technique has gained popularity,[citation needed] and the resulting speed and pitch of the telecined presentation are identical to that of the original film.
This pulldown method[4] is sometimes used in order to convert 24 frame/s material to 25 frame/s. Usually, this involves a film to PAL transfer without the aforementioned 4% speedup. For film at 24 frame/s, there are 24 frames of film for every 25 frames of PAL video. In order to accommodate this mismatch in frame rate, 24 frames of film have to be distributed over 50 PAL fields. This can be accomplished by inserting a pulldown field every 12 frames, thus effectively spreading 12 frames of film over 25 fields (or “12.5 frames”) of PAL video. The method used is 2:2:2:2:2:2:2:2:2:2:2:3 (Euro) pulldown (see below).
This method was born out of a frustration with the faster, higher pitched soundtracks that traditionally accompanied films transferred for PAL and SECAM audiences. A few motion pictures are beginning to be telecined this way[citation needed]. It is particularly suited for films where the soundtrack is of special importance.

Can anyone suggest the 'smartest' way to de-interlace this?

Cheers!

[manono]

Where's that quotation from? Why are you showing us a lossless Lagarith when something from DVD would be much smaller in size. Is this a TV capture?

Anyway, yes, it roughly follows the method as described in the quotation. You add an additional frame every second in the form of a field every half second. So, every other half second of the video is interlaced. However, this one was done somewhat peculiarly and has a bunch of orphaned fields which wreak havoc on the IVTCs. The only way I could figure out to get a decent - decent but not perfect - 24fps video back out of it was to bob it first. The better the bobber, the better the results. I tested with Yadif because it's fast. Something like QTGMC will produce much better results at the expense of taking much more time:

Yadif(Order=1,Mode=1)
TDecimate(Mode=0,Cycle=50,CycleR=26)

The sample was TFF. You might have to change the Yadif field order setting for the whole video (but probably not). And maybe someone else has a better solution.

It was also probably a mistake to create an RGB32 AVI of it, although maybe you have your reasons.

[dvdboy]

Thanks Manono for the help - I'll give that a go when I'm back in the office tomorrow.

The sample was taken from a Digital Betacam, captured to CineformHD via Digital Rapids. I then trimmed out a short section with VirtualDub and outputted to Lagarith as I assumed that was the best mezzanine format that the community could work with.

The quote was taken from Wikipedia - not always the best source I know, but seemed to be describing the effect I was seeing.

[mizumi]

Hi!

I'm sorry it took me so long to answer. At the moment I didn't have enought knowledge about this things and got quite confused with everything I was asking. Now hopefully I have a better understanding of IVTC and avisynth :)

At http://www.doom9.org/ivtc-tut.htm, when they say "bad telecine" or when you say "a clean pulldown pattern", does it mean there has been telecine but it has been edited later and that IVTC is no longer possible or what do they mena?

There are other choices besides telecine and pure interlace.

By "other options" do you mean hybrid?

You said there were passages where every frame was different and I asked you to provide a sample of one such section. But you didn't. It has field-blending and no standard IVTC will work well with that kind.


Field-blending = interlacing?

This anime has little movement so I looked for a passage where a character kept moving in each frame. I thought that was what you meant when you said "every frame to be different". What kind of passage would be useful: you mean the background should be moving too and not only the character?



Try:

Yadif(Mode=1)#or the bobber of your choice
SRestore(FRate=23.976)

http://avisynth.org/mediawiki/Srestore

http://forum.doom9.org/showthread.php?t=95924

The better (and slower) the bobber used, the better the results. Yadif is an OK and very fast bobber, but there are better ones. Also, the video 'shakes' so you might also want to use Stab on it, or Depan:

http://avisynth.org/mediawiki/External_filters#Stabilization

Thank you, I will.
May I try Tdeint too? or it's not useful in this video?

[manono]

If it's field blended no IVTC or deinterlacer will work right. You can tell if it's field blended by separating the fields or bobbing it and looking for the blends/ghosts/double images.

By other choices I meant field blended. Real hybrids are pretty rare. You see them in documentaries sometimes, a mix of video and film sources. You also see them in those space TV shows where they might mix film and 30fps computer effects, either progressive or interlaced. I've also seen real hybrids in anime. Unless you're prepared to encode as VFR, there's no easy way to handle that kind of material.

A lot of film sources are hard telecined and edited as video. This happens a lot with TV shows. They don't care about the telecine pattern. This makes an IVTC maybe more difficult, but not impossible. Because of the cadence breaks you may get orphaned fields and this usually means the post-processor (the deinterlacer) has to kick in to prevent interlaced frames from getting through.

[mizumi]

If it's field blended no IVTC or deinterlacer will work right. You can tell if it's field blended by separating the fields or bobbing it and looking for the blends/ghosts/double images.


What would be bobbing it? (I know how to separate the fields :))


A lot of film sources are hard telecined and edited as video. This happens a lot with TV shows. They don't care about the telecine pattern. This makes an IVTC maybe more difficult, but not impossible. Because of the cadence breaks you may get orphaned fields and this usually means the post-processor (the deinterlacer) has to kick in to prevent interlaced frames from getting through.

Thank you for the explanation but I don't know the difference between hard and soft telecine and how it affects the telecine pattern. Or what "Because of the cadence breaks you may get orphaned fields" means. ^^" If it's easy to explain could you explain me or give me some link?

Why in the case of a bad telecine like this it's better using a deinterlacer (+deblender) rather than using IVTC? What would happen if we IVTC it?

Which brings me to the next question: How do I know if the telecine is good enough to do a IVTC or I'd rather deinterlace?

I have a PAL DVD which I want to deinterlace. It's also field blended as it's an anime >_<
In this case I want to leave the video at 25fps. I guess I would apply the script as with this video:

Bob filter
SRestore()

I did some tests and the one I like better is TDeint(mode=2, type=3) or TempGaussMC_beta1mod().

About SRestore: I read the instructions for SRestore but I still not sure how to configure it for a PAL video which I want to keep at 25fps. Or would it be better to turn it to 23.976fps? (I want to insert a PAL audio in it, that's why I'd keep it at 25fps).

[manono]

What would be bobbing it?
Bobbing takes a field and turns it into a full frame. Yadif(Mode=1) is one such bobber. The framerate is doubled as compared to the source video but the height remains the same (as opposed to being halved as when separating the fields).
Thank you for the explanation but I don't know the difference between hard and soft telecine
Hard telecine is when the telecine is encoded into the interlaced 29.97fps video. Soft telecine is when the video is encoded as 23.976fps progressive and told how to output interlaced 29.97fps with the proper RFF/TFF flags (which act as software, maybe).
Or what "Because of the cadence breaks you may get orphaned fields" means.
Ordinarily, with hard telecine you get 3 progressive and 2 interlaced frames in every 5-frame sequence. When it's edited, the people doing the editing don't take care about keeping the cadence and will make their cuts any old place. This can leave behind orphaned fields, fields without their proper match to make up a frame. This can make a proper IVTC difficult and also can make it difficult for DVD players to play it without stray interlacing being displayed on the TV set. That's no problem for interlaced CRT sets. It can be really annoying for progressive displays such as LCD HDTVs.
Why in the case of a bad telecine like this it's better using a deinterlacer (+deblender) rather than using IVTC? What would happen if we IVTC it?
You talking about that sample? It was never telecined. Therefore it can't be IVTC'd. It has to be unblended. If you IVTC it you'll get jerky, blurry playback because of missing and blended frames.
How do I know if the telecine is good enough to do a IVTC or I'd rather deinterlace?
You IVTC something that's been telecined. IVTC stands for InVerse TeleCine, or undoing the telecine. For any other kind of video an IVTC is the wrong thing to do.

If it was shot using video cameras with every frame interlaced (not that 3/2 telecine pattern I described earlier), then you might want to deinterlace it. If it's field-blended, you unblend it. To find out what you have, maybe the best way is to separate the fields or bob it and have a look.

In this case I want to leave the video at 25fps.
You'll get back the source framerate which, if it's a PAL video field-blended from a film source, isn't 25fps. If you use that script you showed, you'll get back 23.976fps (although I might make it for 24fps. In any event, the output length will be exactly the same as the input length, so you can use whatever audio came with it.

[mizumi]

First of all thank you very much for your help, Manono :)

How can I know if a DVD is soft or hard telecined in order to edit it properly (e.g. IVTC it in case it's a NTSC with film source)?

If the telecine has orphaned fields can I still IVTC or in some cases of very bad telecine you can't IVTC?: In that case do you deinterlace?

If a video is NTSC video + blended, like the one I uploaded and that you checked out, then I deinterlace + unblend, right?

To find out what you have, maybe the best way is to separate the fields or bob it and have a look.

I somewhat learnt how to tell if a video is interlaced using DGIndex and Telecide, as you might remember. I don't know how to do it the way you're explaining... So, after writing Separatefields() or Yadif(mode=1) what else do I have to do to know if a video is telecined or not? :)

In the case of the video sample I uploaded (or in any other case), does it matter if I apply a blender rather than a bobber: it's rather a personal choice or is there a rule for using one or another depending on the source?

I have a PAL DVD which I want to deinterlace. It's also interlaced. In this case I want to leave the video at 25fps.You'll get back the source framerate which, if it's a PAL video field-blended from a film source, isn't 25fps.

So SRestore gets back the source framerate.

If I apply the SRestore() to the video I upload, being NTSC source, after applying SRestore() it will remain at 29.97fps?

Let's say the NTSC video I uploaded was a PAL converted from NTSC - not film -, if I applied SRestore would the framerate go back to NTSC?

Is there any way to know if the source is film or NTSC video, or does it matter anyway in order to configure SRrestore()?



If you use that script you showed (of the PAL video), you'll get back 23.976fps (although I might make it for 24fps. In any event, the output length will be exactly the same as the input length, so you can use whatever audio came with it.


In either case, how would I configure SRestore(): SRestore(), SRestore(frate=25)?

Why does the audio length remain the same even changing the frame rate?

Here's a sample of the PAL video in case you need it: http://www.megaupload.com/?d=6P287A6F

[manono]

How can I know if a DVD is soft or hard telecined in order to edit it properly (e.g. IVTC it in case it's a NTSC with film source)?
Again, there are other choices besides hard or soft telecine. If it's field-blended, it's neither but unblending can still return it to progressive film. Anyway, run the Preview in DGIndex, If it says Film, set DGIndex for 'Forced Film' and make a 23.976fps D2V project file. If it's hard telecine it'll say NTSC. But besides hard telecine, it could also be pure interlace or field-blended. For all three cases you make the D2V with 'Honor Pulldown Flags'. After that you have a look at the unfiltered video and look for that 3/2 progressive/interlace pattern I mentioned earlier. You can also look at the bottom of the D2V file and it'll give the film percentage.

If the telecine has orphaned fields can I still IVTC
Yes. With any luck the deinterlacer will kick in and deinterlace any frames that can't be properly field-matched. Or add in Vinverse afterwards to clean up any residual interlacing.

If a video is NTSC video + blended, like the one I uploaded and that you checked out, then I deinterlace + unblend, right?

You bob and unblend. Although bobbing a video produces progressive frames, it's not a deinterlacer as such.

So SRestore gets back the source framerate.
Well, it has defaults. If from an NTSC source, it'll give you back 25fps; if from PAL you get back 23.976fps. However, there is such a thing as a film source being field-blended to 29.97fps and you'll have to set SRestore to return 23.976fps. And there are silent films at any framerate. You have to figure out and set the 'base' framerate, if it's not the default.

Is there any way to know if the source is film or NTSC video, or does it matter anyway in order to configure SRrestore()?

There are a variety of ways to determine the true framerate. You're not likely to run across any unusual framerates, though. But one way is to set SRestore to return 29.97fps and then count the duplicate frames (for NTSC sources). Sometimes that doesn't work so well and I have to use other methods.

In either case, how would I configure SRestore(): SRestore(), SRestore(frate=25)?
Why would you want to encode for something other than the real framerate? If for DVD encode for 23.976fps and then run DGPulldown afterwards set for 23.976->25 (if for PAL) to make it DVD compliant and ready for authoring. Or add an AssumeFPS(25) to your script, but that will require reencoding the audio to keep audio synch.

Why does the audio length remain the same even changing the frame rate?
Just check the length before and after (VDub(Mod) File->File Information). More frames at a higher framerate in the original video is the same length as fewer frames at a lower framerate after being IVTC'd or unblended. Adding an AssumeFPS to the script will change the length, though

Here's a sample of the PAL video in case you need it
I didn't check it. Is there a question about it?

[mizumi]

In this case I want to leave the video at 25fps.
You'll get back the source framerate which, if it's a PAL video field-blended from a film source, isn't 25fps. If you use that script you showed, you'll get back 23.976fps (although I might make it for 24fps. In any event, the output length will be exactly the same as the input length, so you can use whatever audio came with it.


What does it depend on that you decide to make the video 23.976fps or 24fps?


About SRestore:

Well, it has defaults. If from an NTSC source, it'll give you back 25fps; if from PAL you get back 23.976fps. However, there is such a thing as a film source being field-blended to 29.97fps and you'll have to set SRestore to return 23.976fps. And there are silent films at any framerate. You have to figure out and set the 'base' framerate, if it's not the default.


I've written down the frame rate settings I would write down. Can you correct me in case they're wrong, please?

field-blended NTSC video:
- if the source frame rate is NTSC: SRestore(frate=29.97)
- if the source frame rate is FILM: SRestore(frate=23.976/24)
- if the source frame rate is PAL: SRestore()?
field-blended PAL video:
- if the source frame rate is NTSC: SRestore(frate=29.97)
- if the source frame rate is PAL: SRestore(frate=25)
- if the source frame rate is FILM: SRestore() or SRestore(frate=24)?



There are a variety of ways to determine the true framerate. You're not likely to run across any unusual framerates, though. But one way is to set SRestore to return 29.97fps and then count the duplicate frames (for NTSC sources). Sometimes that doesn't work so well and I have to use other methods.


So to discard if the video's source frate is NTSC I count the duplicate frames and what else shall I do? I tried with the NSTC video I uploaded but wasn't sure what to observe. ^^"
How can I discard it being PAL or film?

For example, in the case of the PAL video sample I uploaded, I don't know if the source video framerate is film or NTSC.

Thank you!

[manono]

What does it depend on that you decide to make the video 23.976fps or 24fps?
I don't work much with PAL sources, field-blended or otherwise. However, I suspect most PAL sources field-blended from film are really 24fps. Better to be wrong at 24fps and have an extra frame in every 1000 than be wrong at 23.976fps and be missing a frame in every 1000.
I've written down the frame rate settings I would write down. Can you correct me in case they're wrong, please?
Some of that is wrong. After determining you have a field-blended source just use SRestore at default settings and then go over the result (step through a section with movement in VDub(Mod)). If you still have duplicate frames, or notice missing frames (the video might play a bit 'jumpy') or have way too many blends remaining, only then should you think about examining it more closely.
So to discard if the video's source frate is NTSC I count the duplicate frames and what else shall I do?
If there's a pattern (one duplicate in six, one in five, etc.), set SRestore for the framerate you'd get with those duplicate frames removed.
For example, in the case of the PAL video sample I uploaded, I don't know if the source video framerate is film or NTSC.
NTSC, as in interlaced 29.97fps video? Just hope you don't ever run across any like that.

[aNToK]

So if I understand all of the above correctly, if I use telecide with the show command and get 0.0% mismatches with telecide(guide=2,post=1, show=true)
and a visible repeat frame every 5 frames, the correct syntax should be:

Telecide(guide=2)
decimate(cycle=5)

Is this correct?

[manono]

Is this correct?
From the Decomb Reference Manual:

For NTSC 24fps->30fps telecine guidance, set guide=1.

I suppose, though, if it's a 20fps silent film on PAL DVD then you could use Guide=2. Is that what you have?

[aNToK]

No, it's a Malaysian dvd of a Korean movie from 2009. Megui couldn't detect the properties and DGIndex came up with Field/Frame at 29.970 FPS and except for 2 frames on a scene change in the beginning, it shows straight progressive video all the way through. Going frame by frame in various parts of the video, the 5th frame was a repeat consistently through all of the movie. Would it be more appropriate to just use decimate without Telicide?

Just checked the vob and the resulting encode with MediaInfo. The Vob reads as 29.970fps interlaced top field first and the encoded one reads as 23.976fps progressive. It plays smooth without the extra frames and audio's in sync, but if I read correctly earlier, Guide=2 was supposed to be for vids that had some kind of marker tab. It seemed to work, but since I've had issues with jittery dvd encodes with Megui-analyzed vids before I'd rather understand it and do it manually.

[manono]

If, when playing the Preview in DGIndex, it shows as Video Type=Film and Frame Type=Progressive, just make the D2V Project File with the Field Operation set for Forced Film. It'll be 23.976fps already and you won't have to worry about an IVTC at all.

[aNToK]

Video type shows as ntsc, interlaced in the DGIndex play window

[manono]

Oh, one of those. I misunderstood a bit about what you said earlier. Mostly every 5th frame is repeated but with at least two interlaced frames. In the off chance that there's more interlacing you haven't spotted, I think I'd make the D2V using 'Honor Pulldown Flags' and perform a full IVTC. To make the encoding go a bit faster you could use TFM(PP=0).TDecimate() which will turn off the post-processor, and which should be pretty safe. You could try using just Decimate() without also using Telecide, but you take the chance of getting some stray interlacing in the output.

And I would never trust MeGUI's analysis. Nothing beats your eyes.

[mizumi]

After determining you have a field-blended source just use SRestore at default settings and then go over the result (step through a section with movement in VDub(Mod)). If you still have duplicate frames, or notice missing frames (the video might play a bit 'jumpy') or have way too many blends remaining, only then should you think about examining it more closely.


This is what I did:
I used a shorter part of the video sample I used before trying to find a scene with more movement and that frames where different from each other:
http://www.yourfilelink.com/get.php?fid=697978

I bobbed the video using yadif(order=1, mode=1) and opened it in vdub. Then I started counting the frames. That was difficult because I didn't know if counting the blended ones as part of the current frame or the following frame. The pattern goes something like this:
2-6-9-5-6-4-7-7-6-6-13-12-5-5-5...

Then I wrote:
yadif(order=1, mode=1)
srestore(frate=24) and reloaded the script in vdub. Then the pattern was easier to account because there were way less blended frames:
2-3-5-2-3-2-3-3-3-3-3-6-6-2-2-2-1-1-1...

Which steps do you follow in order to know which is the proper framerate? Also, which pattern belongs to which framerate?
I know that if the pattern goes like 2-3-2-3 needs to be IVTCed but I have no clue for the rest of patterns, and which are pal and which are NTSC. Could you tell them to me, please?


If there's a pattern (one duplicate in six, one in five, etc.), set SRestore for the framerate you'd get with those duplicate frames removed.


This would be the same doubt as above.
Also I notice you say "one" duplicate in six while most frames I obtained where duplicated twice or more times XD