Field Blended MasterFile (Midnight Texas)

[FranceBB]

Hi,
I just got a field blended MasterFile from a production studio and I have to encode it in XDCAM50 PAL to air it via our playout (Omneon Ports).
When they said that they were going to deliver an Apple Prores, I expected a nice 23.976fps progressive 10bit to speed it up to 25fps and apply a pitch adjustment.
Unfortunately, that didn't happen...
They did send a ProRes, but...


General
Complete name : C:\Encoding\MidnightTexas.mov
Format profile : QuickTime
Codec ID : qt 2005.03 (qt )
Overall bit rate mode : Variable
Overall bit rate : 199 Mb/s
Writing library : Apple QuickTime

Video
ID : 1
Format : ProRes
Format version : Version 0
Format profile : 422 HQ
Codec ID : apch
Bit rate mode : Variable
Bit rate : 197 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 29.970 (29970/1000) FPS
Color space : YUV
Chroma subsampling : 4:2:2
Scan type : Interlaced
Scan type, store method : Interleaved fields
Scan order : Top Field First
Bits/(Pixel*Frame) : 3.178
Writing library : any0
Language : English
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
matrix_coefficients_Original : BT.709

Audio
ID : 2
Format : PCM
Format settings, Endianness : Little
Format settings, Sign : Signed
Codec ID : sowt
Bit rate mode : Constant
Bit rate : 1 536 kb/s
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 kHz
Bit depth : 16 bits
Language : English

Other
ID : 3
Type : Time code
Format : QuickTime TC
Time code of first frame : 11:23:35;00
Time code, striped : Yes
Language : English
Default : No




When I saw "29.970fps interlaced TFF" I thought "fair enough, it's 23.976fps telecined".
Nope...

It's field-blended with progressive parts on a regular pattern.
They basically recorded at 23.976fps progressive, edited it while progressive and they exported it field-blending it in order to air it in NTSC.


#Index original 10bit source
FFMpegSource2("MidnightTexas.mov", atrack=-1)

#Dithering down to 8bit with Floyd-Steinberg error diffusion
ConvertBits(bits=8, dither=1)

#Converting from 4:2:2 planar to 4:2:2 interleave
ConverttoYUY2(interlaced=true)

#Deinterlacing the field-blended parts
tdeint(mode=2, order=-1, field=-1, mthreshL=6, mthreshC=6, map=0, type=2, debug=false, mtnmode=1, sharp=true, cthresh=6, blockx=16, blocky=16, chroma=true, MI=64, tryWeave=true, link=1, denoise=true, slow=2, opt=4)

#Trying to restore the original progressive frames
SRestore(frate=23.976)

#Speed-up +4% with pitch adjustment
AssumeFPS(25, 1, true)
ResampleAudio(48000)

#Converting from 4:2:2 interleave to 4:2:2 planar
Converttoyv16(interlaced=false)


Unfortunately, the resulting 25fps PAL file still had some of the field-blended frames, 'cause SRestore didn't remove the right blended frames.
I tried with TDecimate(mode=2, rate=23.976) but it didn't work either 'cause TDecimate it's supposed to remove duplicated frames, not the blended ones.

I then said "whatever, I'm gonna blend it to 50 and make a interlaced" like:


#Index original 10bit source
FFMpegSource2("MidnightTexas.mov", atrack=-1)

#Dithering down to 8bit with Floyd-Steinberg error diffusion
ConvertBits(bits=8, dither=1)

#Converting from 4:2:2 planar to 4:2:2 interleave
ConverttoYUY2(interlaced=true)

#Deinterlacing the field-blended parts
tdeint(mode=2, order=-1, field=-1, mthreshL=6, mthreshC=6, map=0, type=2, debug=false, mtnmode=1, sharp=true, cthresh=6, blockx=16, blocky=16, chroma=true, MI=64, tryWeave=true, link=1, denoise=true, slow=2, opt=4)

#Blending the resulting 29.970 to 50fps
ConvertFPS(50)

#Diving in field to get 25i
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()

#Converting from 4:2:2 interleave to 4:2:2 planar
Converttoyv16(interlaced=true)


The result was awful and I didn't have the guts to send it to QC.

In other words, I have to get back the original 23.976fps progressive to apply the speed-up.
Luckily enough, the field-blended frames follow a regular pattern through the whole video.

The pattern is:
0 = Progressive
1 = Progressive
2 = Field Blended
3 = Progressive
4 = Progressive
5 = Progressive
6 = Progressive
7 = Field Blended
8 = Progressive
9 = Progressive
10 = Progressive
11 = Progressive
12 = Field Blended
13 = Progressive
14 = Progressive
15 = Progressive
16 = Progressive
17 = Field Blended
18 = Progressive
19 = Progressive
20 = Progressive
21 = Progressive
22 = Field Blended
23 = Progressive
24 = Progressive
25 = Progressive
26 = Progressive
27 = Field Blended
28 = Progressive
29 = Progressive
30 = Progressive
31 = Progressive
32 = Field Blended
33 = Progressive
34 = Progressive
35 = Progressive
36 = Progressive
37 = Field Blended


I think I could achieve this via SelectEvery but I don't know how to set the pattern.

Last but not least, I'm afraid I can't upload a sample as it's protected copyright material that we have the rights to air on our channels, but I can't redistribute it here.

Thank you in advance,
Frank.

[poisondeathray]

Luckily enough, the field-blended frames follow a regular pattern through the whole video.

I think I could achieve this via SelectEvery but I don't know how to set the pattern.


If that's true, then it would be the same as you posted . Selecting the progressive ones, omitting the blended ones

selectevery(30, 0,1,3,4,5,6,8,9,10,11,13,14,15,16,18,19,20,21,23,24,25,26,28,29)


Or get on the phone and tell them to send you a proper version

[Frank62]

Can you upload a short test file?

Ooops. Just read, that this is not possible.

[Cary Knoop]

Or get on the phone and tell them to send you a proper version
^ This!

[johnmeyer]

I would read through all the SRestore documentation, and look at the SRestore main thread. My goal in doing this would be to see if you can do the entire recovery within SRestore, without trying to do any decimation or manipulation prior to SRestore. I've done some complicated screwed-up video and SRestore was able to handle it, but I had to set a lot of parameters by hand that are normally left at their default settings.

TFM can also be useful for helping you automatically track the patterns and adapt as needed. The problem with anything that uses "SelectEvery()" is that the pattern inevitably hiccups at some point and you then have to readjust. This can happen dozens of times in a relatively short time.

[FranceBB]

If that's true, then it would be the same as you posted . Selecting the progressive ones, omitting the blended ones

selectevery(30, 0,1,3,4,5,6,8,9,10,11,13,14,15,16,18,19,20,21,23,24,25,26,28,29)

Alright, I just tried it and it worked.
I just encoded the whole episode in XDCAM and the pattern didn't change, so it was actually fine.
I'm sending it to QC right now.


I would read through all the SRestore documentation, and look at the SRestore main thread. My goal in doing this would be to see if you can do the entire recovery within SRestore, without trying to do any decimation or manipulation prior to SRestore. I've done some complicated screwed-up video and SRestore was able to handle it, but I had to set a lot of parameters by hand that are normally left at their default settings.

By testing various parameters of SRestore maybe I could have achieved the same result.
I gotta say that the result outputted by SRestore by default wasn't bad, though, but it kept some blended frames, that's why I just took a look at the pattern and I decided to use SelectEvery.
In other words, SRestore is definitely useful, but whenever I have a constant pattern that doesn't change through the whole video, SelectEvery comes in handy.

Thank you all. ^_^

[johnmeyer]

If the pattern is constant, then SelectEvery() is definitely the way to go.

Poisondeathray comes through again.