2:2:2:2:2:2:2:2:2:2:2:3 pulldown or PAL IVTC?

[odyssey]

Hello

I'm converting my Disney Treasure series (mostly PAL) into it's original 24 footage. Problem is that with the PAL release, they decided to use what I believe is called a 2:2:2:2:2:2:2:2:2:2:2:3 pulldown or perhaps PAL IVTC?

I tried browsing for the proper way to treat this, but couldn't find anything useful. Perhaps most people don't care, or this is just a rare case.

I tried Telecide and TDecimate with various parameters, and some work, but selects wrong frames (would perhaps work better with real footage, rather than cartoons).

It would be most convenient with an automated solution, since it would take ages determining offsets for each cartoon (and there are quite a lot).

Anyway, to chop this up, the field(/true frame) order is this (E/O for even/odd-fields):

EO.EO.EO.EO.EO.EO.EO.EO.EO.EO.EOE|OE.OE.OE.OE.OE.OE.OE.OE.OE.OE.OE.OEO|...(repeat)

How could I deal with this, even in a manual way? Ideally the redundant 23. field should just be deleted, but then the field order should be reversed for the remaining set before weaving it all back into original progressive frames.

I hope you can help. I find it hard to believe I should be the first with this issue. Maybe I just searched for the wrong terms?

[odyssey]

I just traversed the footage and of course it's not even regular.

2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:2:2:3:
2:2:2:2:2:2:2:2:2:3:

I didn't look further to examine if this pattern is repetitive. I guess it's just random, and even if it's not, it's certainly not possible to for me to determine it for hundreds of shorts.

This looks pretty hopeless, unless someone has a magic formular?

I uploaded a short sample here, if anyone wants to have a go at it:
http://s000.tinyupload.com/index.php?file_id=00117952962885514019

[kuchikirukia]

Have you tried 2-pass?

http://s000.tinyupload.com/?file_id=06095348869741574811

[manono]

Have you tried TIVTC on it:

TFM()
TDecimate(Mode=1,Cycle=25,CycleR=1)

[Emulgator]

Just in case you want to try the hard way: the manual way:
#[*#________________Start of Manual encoded Hard Euro-Pulldown-Removal 25i -> 24p (25 pattern offset variations)_____________
LWLibavVideoSource("source.avi")
#SwapFields()
#ComplementParity # ComplementParity only changes the assumed field order, not the physical field order !
#ReverseFieldDominance(shiftup=true)#(PAL Only ! YUY2 oder RGB only) ...Like CCE's shift lines by 1
#RevFieldDom: PAL DV is expected to be sampled bottom field first.
#In case of a device sampling top field first this filter can reverse field dominance
#by simply shifting each line up (or down) by one line and duplicating the bottom (respectively top) line.
#AssumeBFF()
SeparateFields()
shift=0 # testwise vary shift to shift source under pattern or choose a different pattern
Blankclip(last, length=shift) ++ last
SelectEvery(25, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 0 von 25
#SelectEvery(25, 0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 1 von 25
#SelectEvery(25, 0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 2 von 25
#SelectEvery(25, 0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 3 von 25
#SelectEvery(25, 0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 4 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 5 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 6 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 7 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 8 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 9 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 10 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 11 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 12 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 13 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 14 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 15 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 23, 24) #Decimation: 16 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24) #Decimation: 17 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24) #Decimation: 18 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 24) #Decimation: 19 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24) #Decimation: 20 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24) #Decimation: 21 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24) #Decimation: 22 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24) #Decimation: 23 von 25
#SelectEvery(25, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23) #Decimation: 24 von 25
Weave()
#*]#______________End of Manual encoded Hard Euro-Pulldown-Removal 25i -> 24p (25 pattern offset variations)___________________

[odyssey]

Have you tried TIVTC on it:

TFM()
TDecimate(Mode=1,Cycle=25,CycleR=1)

Yes I did try that, and it looks good, but it's not precise, perhaps due to the odd pattern I determined.

F.ex. it drifts about half a second towards the end of the entire cartoon, and some frames are not deleted properly. F.eks. the last frame of the scene with huey louie and dewey walking with their ice cream, is doubled. If you inspect the source, you can see that there are just 3 fields with this frame, but the redundant field is not deleted as it should be. I suspect this is leading to the drift.

I compare it to the NTSC version, which is easier to IVTC properly.

I imagine that it would probably be possible to make a proper script to take care of this, with this ruleset:

- If frame -2 compared to the current frame is completely identical, the field should be deleted, and the field order swapped. It might need to look ahead or behind to avoid deleting intentional animated duplicate frames. Perhaps this could be done, but only deleting a field if the group of similar fields totals to an odd number, but that may be harder to know since comparing an even and odd field may not look too identical.

- Only delete one field in a series of max 25 (perhaps more?) and minimum 21 fields.

What I don't understand is, why it looks like the pattern is duplicating more than 1 field per 25 fields (sometimes just 18 fields between the 3 duplicated fields). That would make it faster than 24fps (and even 23,976 - right?), when they are deleted, wouldn't it?

[ChiDragon]

Have you tried TIVTC on it:

TFM()
TDecimate(Mode=1,Cycle=25,CycleR=1)
TDecimate of course picks the wrong frames to remove now and then since it's animation, but more importantly 24.000 isn't the correct frame rate for this sample. It's something closer to 23.976, if not exactly that.

It's been a long time since I dealt with something like this so I'm just going by a quick forum search for the numbers (http://forum.doom9.org/showthread.php?t=141013) involved in 23.976 -> 25: the pattern is likely 1001 frames long with 41 duplicates.

I don't suppose there are any parts of this animation that feature camera movements over 1000 frames long so we could check easily...

Hopefully this image works:
http://s14.postimg.org/619chykdd/odyssey_Disney.jpg

If the cycle were 25, frame 90 would be a duplicate rather than frame 88.

Using mode 2 with the frame rate set to 23.976 may work out well.

[ChiDragon]

the last frame of the scene with huey louie and dewey walking with their ice cream, is doubled.
I think this frame (301) is actually a pattern break caused by a video edit at that scene change. Same with at least one other scene change.

I don't like this sample anymore. :)

[manono]

F.ex. it drifts about half a second towards the end of the entire cartoon...
What are you saying? That the resulting video is longer or shorter by half a second than is the source? If so, that's simply not true. An IVTC, even with the wrong framerate, doesn't change the length of a video. If you're saying there might be 12 extra frames than there should be after the IVTC, then yes, I'd agree if the framerate is supposed to be 23.976fps rather than 24fps.

As for ChiDragon's suggestion that it might be more accurately be 23.976fps, that's something I considered but using Mode=2 and defining a framerate is often less accurate than using Mode=0. And, since it's animation, having an extra duplicate frame every 40 seconds or so isn't such a big deal, in my opinion.

If you want to make sure a duplicate is removed before a maximum number of frames have gone by, you can use the maxndl parameter. To have it examine more than 100 frames before deciding which to remove, you can set the m2PA parameter.

[odyssey]

They really messed up the video beyond recovery. Seems impossible to determine which frames are supposed to be there twice and which are not. I found a place with 3 fields, which are actually 2 original frames.

Anyway, the suggestion to use TFM().TDecimate(Mode=2,Cycle=24) is pretty accurate! It retains the correct length throughout the animation, and seems to be the one with the fewest frame displacements. I'll try it, and it's probably not visible anyway.

Thanks a LOT for the help guys. You are amazing!!

ChiDragon: How do you get that extended info displayed?