29.97 -> 25 conversion with tdecimate

[Katie Boundary]

Mainframe Entertainment, creators of Reboot and several other great shows, are a bit unique in their PAL -> NTSC conversion process. Instead of slowing footage down to 24.975 fps and then duplicating one out of every 5 fields, they leave the speed and running time unchanged and duplicate 199 out of every 1001 fields. This has made things very complicated for anyone who has these shows on NTSC discs and wants to get a perfect conversion back to 25 FPS. Which of the following options is better?

tdecimate(cycler=199,cycle=1200,sdlim=5)

or

tdecimate(mode=2,rate=25,maxndl=6,mp2a=true)

?

Mode 2 doesn't do a perfect job of actually hitting the intended frame rate. However, using such a huge cycle length in mode 0 on hard-telecined footage can result in duplicate frames not being seen as duplicates due to having different compression artifacts, and non-duplicate frames from motionless scenes being seen as duplicates.

[manolito]

Being from PAL land I might be overstepping my boundaries here (no pun intended), but I believe you got some things wrong...

Instead of slowing footage down to 24.975 fps and then duplicating one out of every 5 fields

I believe you really mean "slowing down footage to 23.976 fps and then applying standard 3:2 NTSC pulldown". Am I correct?

they leave the speed and running time unchanged and duplicate 199 out of every 1001 fields.

So they apply the "irregular" 25 -> 29.97 pulldown, correct?

In both cases the fields get duplicated, not frames. The result will be "pseudo"-interlaced. You cannot just retrieve the original progressive frames by simply decimating the result. You need to match the fields first (TFM or Telecide).

After field matching you can decimate back to 25 fps, but of course not by using the default 1 in 5 decimation. With TDecimate I mostly get good results using "mode=7, rate=25", but you know how to tweak the TDecimate parameters... :D


Cheers
manolito

[johnmeyer]

It's really tough to know how to give advice on this without a sample since, as you point out, there are several variations on how this is done. However, I just looked through all the scripts I've written and apparently I was asked to deal with something similar. Here is the script I used.

# Script to return NTSC to PAL video, if original converion was done with 6:5 pulldown

loadPlugin("c:\Program Files\AviSynth 2.5\plugins\TIVTC.dll")

source=AVISource("e:\fs.avi")

AssumeBFF(source)
crop(92,0,-92,0)
separatefields()
tfm(display=false, mode=1)
tdecimate(display=false,mode=1,cycleR=2,cycle=12)
weave()
IResize(720,576)
AssumeFPS(25)Don't forget that you'll probably have to do a resize as part of this, as I did above. I assume the crop was part of that, although I didn't keep any notes on this job, so I don't know.

[wonkey_monkey]

I believe you really mean "slowing down footage to 23.976 fps and then applying standard 3:2 NTSC pulldown". Am I correct?

You probably aren't correct - "slowing down to 24.975" and repeating every fifth field/frame results in 29.97fps, so seems a perfectly logical strategy (albeit a weirdly described one, here) since it hardly changes the running time - and be warned, you will probably get your head bitten off for daring to assume that Katie Boundary may have made an error.

In any case, she's using that as an example of what they have not done, so it's not relevant.

I can only recommend that people never ever assume anything about what Katie Boundary says, and never discuss anything that isn't directly relevant to the desired solution, because you'll probably get nothing but spite in reply for your trouble.

---

Personally in this case I would see if I could determine if there was a regular pattern in these "199 duplicates out of 1001". Is it a case of "1 out of 5" and then a change in cadence every 1001 frames?

[hello_hello]

Slowing PAL down to 24.975fps for converting to NTSC does seem to be a thing, although I haven't seen many examples. It's mentioned in the PAL to NTSC paragraph here. http://foro.doom9.org/ivtc-tut.htm#IVTCForNTSC
Or here if the site starts working again. http://www.doom9.org/ivtc-tut.htm#IVTCForNTSC

For converting from 25fps to NTSC, 199/1200 sounds right, although I didn't try to work it out.

For that sort of thing I generally go in assuming my assumption regarding the correct frame rate will be wrong, so I'd run an analysis pass to create metrics files for TFM and TDecimate. Then you can switch to "2nd pass mode", adjust the output rate for TDecimate and check the result without having to wait a week for mode=2 to look at a huge number of frames each time. I'm not sure maxndl=6 is necessarily a good idea. Telling TDecimate it's limited to a maximum of 6 non duplicate frames in a row might be a little errr..... limiting.

If you look at the metrics file there's often a pattern of duplicate frames to be seen somewhere, along with how regularly they occur. And I recall Display=true can be informative, but it's all easier with metrics files.

Not that I spent more than a couple of seconds looking, but the internet says Reboot was a Canadian production. Wasn't Canada an NTSC land?

[wonkey_monkey]

for converting from 25fps to ntsc, 199/1200 sounds right, although i didn't try to work it out.

25*(1200)/(1200-199) = 29.97002997... (30000*1000/1001)

[StainlessS]

(30000*1000/1001)
Maybe 1 too many zeros.

To below
Edit: Maybe three too many zeroes?
Yep, sounds more like it :)

[Frank62]

Even TDecimate is not quite perfect when it comes to bigger cycles (after my experience), play with the parameters as much as you like. You always get longer parts where everything seems fine, but occasionally horrible jerking parts. Also with the alternative mode with an explicit framerate. Sometimes - depends on the source - I even had better results with SmartDecimate(!), but I fear in this context it will also fail.
If this conversion has been made especially for the DVDs (no further edits) then the chance is high that you can convert 100% correctly "by hand", with a correct SelectEvery (or I am sure there are more simple ways to do select manually, which I hadn't the time to explore yet...).
We do a lot of NTSC-PAL conversions this way "by hand" - often just half an hour more work but with the result: not a single dup or jump.

Edit: Maybe three too many zeroes? ;)

[Katie Boundary]

I believe you really mean "slowing down footage to 23.976 fps and then applying standard 3:2 NTSC pulldown". Am I correct?

No. Mainframe renders their stuff at 25 frames per second. A slowdown to 23.976 would make the audio noticeably deeper and serve no real purpose. Are there really people who convert PAL content to NTSC that way?

So they apply the "irregular" 25 -> 29.97 pulldown, correct?

Yes. It's a 1,001-field-long pulldown pattern that ChiDragon first noticed when trying to clean up Reboot footage.

In both cases the fields get duplicated, not frames. The result will be "pseudo"-interlaced. You cannot just retrieve the original progressive frames by simply decimating the result. You need to match the fields first

Yeah, that's all old news and not really related to what I was asking.

(TFM or Telecide).

Because there are no orphaned fields, Uncomb works pretty well too ;)

After field matching you can decimate back to 25 fps, but of course not by using the default 1 in 5 decimation.

I said all of that in the OP. Thanks for paying attention ;)


# Script to return NTSC to PAL video, if original converion was done with 6:5 pulldown

loadPlugin("c:\Program Files\AviSynth 2.5\plugins\TIVTC.dll")

source=AVISource("e:\fs.avi")

AssumeBFF(source)
crop(92,0,-92,0)
separatefields()
tfm(display=false, mode=1)
tdecimate(display=false,mode=1,cycleR=2,cycle=12)
weave()
IResize(720,576)
AssumeFPS(25)

That's REALLY interesting. Following separatefields with TFM sounds like something I'd try to do in order to clean up early seasons of South Park, and I've never seen "Iresize".

I mean, it's not helpful at the moment, but it's super interesting.

Don't forget that you'll probably have to do a resize as part of this, as I did above.

I do resize to 352x288 because I'm exporting to VCD-compliant MPEG-1, but that's not part of the decimation strategy.

If this conversion has been made especially for the DVDs (no further edits)

Well... it was done especially for NTSC televisions. I'm not sure what DVD has to do with it.

then the chance is high that you can convert 100% correctly "by hand", with a correct SelectEvery

I'm not autistic enough or OCD enough to use a 1,001-argument-long Selectevery, especially if I'd have to figure out a new set of arguments for each of 16 episodes of Reboot III, 26 episodes of War Planets, 52 episodes of Transformers: Beast Wars, and 26 episodes of Transformers: Beast Machines. The Force isn't THAT strong with me :D

[StainlessS]

never seen "Iresize".


function IResize(clip Clip, int NewWidth, int NewHeight) {
# http://forum.doom9.org/showthread.php?p=1185790#post1185790
Clip
SeparateFields()
Shift=(GetParity() ? -0.25 : 0.25) * (Height()/Float(NewHeight/2)-1.0)
E = SelectEven().Spline36resize(NewWidth, NewHeight/2, 0, Shift)
O = SelectOdd( ).Spline36resize(NewWidth, NewHeight/2, 0, -Shift)
Ec = SelectEven().Spline36Resize(NewWidth, NewHeight/2, 0, 2*Shift)
Oc = SelectOdd( ).Spline36Resize(NewWidth, NewHeight/2, 0, -2*shift)
Interleave(E, O)
IsYV12() ? MergeChroma(Interleave(Ec, Oc)) : Last
Weave()
}

[Katie Boundary]

function IResize(clip Clip, int NewWidth, int NewHeight) {
# http://forum.doom9.org/showthread.php?p=1185790#post1185790
Clip
SeparateFields()
Shift=(GetParity() ? -0.25 : 0.25) * (Height()/Float(NewHeight/2)-1.0)
E = SelectEven().Spline36resize(NewWidth, NewHeight/2, 0, Shift)
O = SelectOdd( ).Spline36resize(NewWidth, NewHeight/2, 0, -Shift)
Ec = SelectEven().Spline36Resize(NewWidth, NewHeight/2, 0, 2*Shift)
Oc = SelectOdd( ).Spline36Resize(NewWidth, NewHeight/2, 0, -2*shift)
Interleave(E, O)
IsYV12() ? MergeChroma(Interleave(Ec, Oc)) : Last
Weave()
}


Now I'm just confused instead of curious.

[Frank62]

I'm not autistic enough or OCD enough to use a 1,001-argument-long Selectevery, ...
Yes, I wouldn't do so either. We use it for plain 29.97 to 23.976 with a SELECTEVERY.
But as I said, I am sure there are other ways for decimating by hand, I just never used it.

The clue with "especially for the DVDs" was, that if the conversion has been made in more recent times and no editions were made afterwards, that you might need only one pattern for one episode.
Your decision.

[johnmeyer]

IResize first appeared ten years ago in this thread:

Deinterlace should preserve original pixels? (https://forum.doom9.org/showthread.php?t=140233)

It is worth looking at this because it gets into some very esoteric stuff about how Bob() works, but then dives into some really interesting stuff about how chroma is handled in interlaced video.

The very confusing "shift" in the IResize function resulted from the conversations in this post.

[Katie Boundary]

From the linked thread:

Since Bob uses a Bicubic (aka Mitchell-Netravali) reconstruction filter, it will keep the original pixels when that filter acts as a pure interpolator, ie if and only if b=0 (regardless of the value of c). Hence the default values (b=c=1/3) do not preserve the input data. As the documentation states, (b=0, c=1) will do so, but so will b=0 and any other value of c


HAH! You know, this is something that I've been curious about for a LONG time, and wanted to test. I just never got around to it.

All the stuff about incorrect chroma interpolation is neat too. But I think we're drifting a tad off-topic.

The clue with "especially for the DVDs" was, that if the conversion has been made in more recent times and no editions were made afterwards, that you might need only one pattern for one episode.

It's a straight pulldown of 25 fps progressive content to 29.97 fps. The pulldown pattern is implemented by Mainframe, not by distributors, so it should be the same on the NTSC TV broadcasts as it is on the NTSC DVDs. Why should it matter how recently it was done or if any other "editions" have been made since then?

[Frank62]

Sorry, that's again my English. I, of course, meant Edits, not Editions... ;)

Recent Pulldowns are - after my experience - very much more straight than older from the ninetees or even more old. Might have to do with former hardware, and how these things are done today.

But I am out at this point - didn't want to compete in smartness, just to help. :)