handling ntsc material originally from pal source

[Lebowsky]

I have an ntsc video from dvd. Original material was most likely PAL progressive and converted to NTSC dvd.

It is 30fps and it seems one frame out of 6 (or 12 if count both a and b frames each) is a duplicate. So if you handle it as progressive and leave it at 29.97fps, 1 frame out of 6 you'll see during playback is duplicate. All a frames are progressive and b interlaced.

How should one handle such a video? Somehow ivtc it to 24 or 25fps? What would be the best way to do this in avisynth? I've actually never encountered this, all PAL->NTSC converted material I had seen till now was "fully" interlaced.

[manono]

It might be best if you make available a short 10 second or so sample. Your description is a little bit confusing and one hates to guess. The most common way to do poor conversions from PAL to NTSC is with field-blending but you made no mention of that.

[Music Fan]

You can try
tfm().tdecimate()
both included in TIVTC ;
http://avisynth.nl/index.php/TIVTC

[Lebowsky]

It might be best if you make available a short 10 second or so sample. Your description is a little bit confusing and one hates to guess. The most common way to do poor conversions from PAL to NTSC is with field-blending but you made no mention of that.

alright, here's a sample. I realize my description is not very accurate. I used tmpgenc to check the pattern (don't laugh, but I still think this is the easiest tool to check) so I kind of mixed up between frames and fields...

anyway here's a sample: http://www.0wned.ch/tmp/sample_1.vob


You can try
tfm().tdecimate()
both included in TIVTC ;
http://avisynth.nl/index.php/TIVTC

yes exactly, but what I don't know is which parameters to use in such a case... especially in order to keep the audio in synch.

[johnmeyer]

It is just a simple PAL to NTSC 29.97 telecine, with every fifth frame repeated. They didn't even make the attempt to do the repeats with fields, so it probably looks pretty jerky when played on a TV.

The TFM / TDecimate solution already suggested should work perfectly to get you back to pure progressive 25 fps. If you instead want 23.976 progressive, you will need to add AssumeFPS(24000/1001) after the TFM/TDecimate lines. You'll then need to adjust the audio to fit.

[manono]

You can try
tfm().tdecimate()
both included in TIVTC
No.

It is just a simple PAL to NTSC 29.97 telecine, with every fifth frame repeated.

Yes, every sixth frame is a repeat of every fifth frame in each six-frame cycle. A standard IVTC won't work since it'll return 23.976fps (as in Music Fan's script above) and every second there will be a missing frame. It's progressive so there's no field matching to be done (unless I missed some interlacing) and only TDecimate is needed:

TDecimate(Mode=0,Cycle=6,CycleR=1)

...especially in order to keep the audio in synch.

The length doesn't change so the audio remains in synch. If you slow the video framerate as johnmeyer mentioned, only then will you have to slow the audio to match.

[StainlessS]

Don't you mean every sixth frame repeated?

No, every 5th frame is repeated, every 6th frame is a repeat.

[manono]

Damn, I edited it but you spotted the mistake before I finished. The point still stands, though, that the script recommended by two different people is incorrect.

[Music Fan]

A standard IVTC won't work since it'll return 23.976fps (as in Music Fan's script above) and every second there will be a missing frame. It's progressive so there's no field matching to be done (unless I missed some interlacing) and only TDecimate is needed:

TDecimate(Mode=0,Cycle=6,CycleR=1)
I don't understand why adding TFM before TDecimate would result in a 23.976fps video. TFM is supposed to create progressive frames but doesn't change framerate, TDecimate does.
But I didn't download the sample to make tests.

[manono]

I don't understand why adding TFM before TDecimate would result in a 23.976fps video
It doesn't. Since there are no fields to match, in this case TFM does nothing at all. It's TDecimate that, at default settings, removes one out of every five frames to give you 23.976fps. But for this video the 'base' framerate isn't 23.976fps, but 24.975fps (or maybe 25fps). You want one out of every six frames removed.

You could try it yourself and see.

You could read the TDecimate doc which also explains what TDecimate at the default settings does.

[Music Fan]

Ok, I misunderstood because I don't know very well these functions and as you said "only TDecimate is needed", I believed you meant that adding TFM would remove some frames each second.

[Lebowsky]

alright, so how to treat the duplicate frames is clear to me now, but if I only do TDecimate(Mode=0,Cycle=6,CycleR=1), what will happen to the framerate of the resulting video? will it stay at 29.97? or will it "automatically" be converted to 24.975/25?

and if there happens to be some interlaced frames here and there in the rest of the video, should I simply add tfm() first? (I didn't check the whole video)

thanks a lot!

[Lebowsky]

much easier than asking questions, I tried it myself. The whole thing comes out fine, and at 24.975 fps. However, the last scene is interlaced. How should this be handled? Here are the last 10 seconds: http://www.0wned.ch/tmp/sample_13.vob (interlacing starts after 5 secs)

[johnmeyer]

Sorry about forgetting the Cycle/CycleR settings. I was just doing a quick reply.

As for the most recent clip (post #13), it looks exactly the same to me as the first clip. I see no interlacing when I separatefields(). All upper/lower fields are from the same moment in time, the definition of progressive.

However, having said that, there are definitely some weird things happening in the last few seconds with the closeup of the girl in the back seat of the car. There is some sort of field reversal happening. I think this probably happened because they used slow motion on the ending. I found that merely adding TFM prior to TDecimate solved the problem. AFIK, there is no downside (other than processing time) to using TFM() prior to TDecimate, even in the case where the fields are already matched, and including it lets you deal with situations like this.

This is the script I used to test your clip. The final separatefields() was for testing purposes and should not be used for your final result.

assumeBFF()
tfm()
tdecimate(Cycle=6,CycleR=1)
separatefields()

[manono]

How should this be handled?

Yes, then you'll need TFM. However, that part with the woman in the cab is pretty messed up and not only is there field-matching going on but deinterlacing as well. And there are many more duplicate frames which won't all disappear when using TDecimate afterwards. Although you can have TFM use a different and better deinterlacer when necessary, easiest might be to just use:

TFM()
TDecimate(Mode=0,Cycle=6,CycleR=1)

Edit: johnmeyer beat me to it. :)

[johnmeyer]

And there are many more duplicate frames which won't all disappear when using TDecimate afterwards. But isn't that OK? Like I said in my previous post, I think this was the result of slow motion in the original film which, in this case, was done simply by repeating frames. I don't know if the TFM/TDecimate combo will get back to the exact cadence of the repeats in the original film, but my guess is that it will get pretty close.

[manono]

But isn't that OK? Like I said in my previous post, I think this was the result of slow motion in the original film which, in this case, was done simply by repeating frames. I don't know if the TFM/TDecimate combo will get back to the exact cadence of the repeats in the original film, but my guess is that it will get pretty close.
I agree. I'd use the script and leave the remaining dupe frames alone.

[Lebowsky]

so I tried tfm(), but on that last part in the cab it doesn't look too good... stilll looks interlaced even? or it has lots of interlaced artefacts... can I tweak tfm() to do this better?

I actually used a tmpgenc project which I left at 30fps but where I simply deinterlaced the last frames with double and then framweserved to avisynth using vfapi, and used tdecimate(Cycle=6,CycleR=1) on that and it looks like quite good... so any chance the deinterlacing can be done directly in tfm() or using another function? is it possible to do it only on the last frames? (or does it not matter if it's done on the whole video?)

I'm sorry, I've always worked with PAL before so this is really new to me...

[johnmeyer]

I saw no interlacing in the cab interior after using that script. How are you testing for interlacing? You can be fooled by screen grabs. Also, make sure you have the field order correct (use AssumeTFF or AssumeBFF, if necessary, at the beginning of the script). Since you are frameserving, it is easy to get the field order incorrect (at least that has been my experience).

[Lebowsky]

the tmpgence frameserving thing is where I'm getting a good result, it was just to do a comparison, I'd like to avoid that if possible :)

I have assumebff() in the beginning. I think it's not really interlaced, but maybe more like deinterlacing artefacts due to the method used by tfm()? I simply checked by watching the video. here are two screenshots, not the exact same frame but you can see what I mean I think:

using tfm():
http://www.0wned.ch/tmp/tfm.jpg

using double in the tmpgenc project:
http://www.0wned.ch/tmp/double.jpg

[manono]

You are right, and that frame is interlaced. Frame #157 at 29.97fps is one such. It's so slight that TFM didn't pick it up.

You have to lower the CThresh to make TFM more sensitive to it:

TFM(CThresh=3)
TDecimate(Mode=0,Cycle=6,CycleR=1)

However, with the CThresh so low this means other good frames might get deinterlaced, along with the ones needing it. Then you might want to use a better deinterlacer, better than TIVTC's TDeint. Here are a couple of ways to use different deinterlacers, the first with NNEDI3, the second with QTGMC (which slows the encoding considerably):

tdeintted = NNEDI3()
TFM(CThresh=3,clip2=tdeintted)
TDecimate(Mode=0,Cycle=6,CycleR=1)

tdeintted = QTGMC().SelectEven()
TFM(CThresh=3,clip2=tdeintted)
TDecimate(Mode=0,Cycle=6,CycleR=1)

Using NNEDI3 just requires NNEDI3 (http://avisynth.nl/index.php/Nnedi3). Using QTGMC requires QTGMC (http://avisynth.nl/index.php/QTGMC) and a bunch of other stuff. Unless you're familiar with its use, it will be easier to stick with NNEDI3 (or even TDeint).

Also, while using a VFAPI had its uses once upon a time, that time passed 15 years ago. It might not make much difference with your lousy source, but going the VFAPI route produces inferior results. In my opinion, you should stick with MPEG2Source and AVISynth filters.

[johnmeyer]

You are right, and that frame is interlaced. Frame #157 at 29.97fps is one such. It's so slight that TFM didn't pick it up.The OP might also want to take a look at TFM's "mode" and "pp" settings.

[Lebowsky]

You are right, and that frame is interlaced. Frame #157 at 29.97fps is one such. It's so slight that TFM didn't pick it up.

You have to lower the CThresh to make TFM more sensitive to it:

TFM(CThresh=3)
TDecimate(Mode=0,Cycle=6,CycleR=1)

However, with the CThresh so low this means other good frames might get deinterlaced, along with the ones needing it. Then you might want to use a better deinterlacer, better than TIVTC's TDeint. Here are a couple of ways to use different deinterlacers, the first with NNEDI3, the second with QTGMC (which slows the encoding considerably):

tdeintted = NNEDI3()
TFM(CThresh=3,clip2=tdeintted)
TDecimate(Mode=0,Cycle=6,CycleR=1)

tdeintted = QTGMC().SelectEven()
TFM(CThresh=3,clip2=tdeintted)
TDecimate(Mode=0,Cycle=6,CycleR=1)

Using NNEDI3 just requires NNEDI3 (http://avisynth.nl/index.php/Nnedi3). Using QTGMC requires QTGMC (http://avisynth.nl/index.php/QTGMC) and a bunch of other stuff. Unless you're familiar with its use, it will be easier to stick with NNEDI3 (or even TDeint).

Also, while using a VFAPI had its uses once upon a time, that time passed 15 years ago. It might not make much difference with your lousy source, but going the VFAPI route produces inferior results. In my opinion, you should stick with MPEG2Source and AVISynth filters.

thank you very much. I'll do some tests and report back.

regarding VFAPI, I know it's totally outdated, however tmpgenc is the only visual tool I know to use to check for the interlacing patterns. whats used nowadays then?

The OP might also want to take a look at TFM's "mode" and "pp" settings.

actually I did and tried a few combinations unsuccessfully before asking :)

[manono]

whats used nowadays then?

I use my eyes.

[Lebowsky]

I use my eyes.

yes if course :p but what utility do you use to browse a video frame by frame and/or field by field?

[Music Fan]

I believe most of us use Virtual Dub with Avisynth and the function separatefields().

[Lebowsky]

thank :)

back on topic though, I might be stupid, but I am not getting any better result using nnedi3 or gtgmc. and I know qtgmc is being used, because the encode take a lot more time.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\TIVTC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\nnedi3.dll")

mpeg2source("D:\sample.d2v")
assumebff()
deint = QTGMC(Preset="Slow").SelectEven()
tfm(CThresh=3,clip2=deint)
TDecimate(Mode=0,Cycle=6,CycleR=1)

am I missing something obvious?

[manono]

am I missing something obvious?
Dunno, worked for me. Okay, try this:

TFM()
Vinverse()
TDecimate(Mode=0,Cycle=6,CycleR=1)

Vinverse (http://avisynth.nl/index.php/Vinverse) cleans up residual interlacing.

[Lebowsky]

I'll try it, thanks!