Interlaced Studio DVDs

[magic144]

Forgive me for asking what may be an oversimplification,
but can anyone tell me why on the verge of 2007 - with so many people continuing to usher in a new hardware era of progressive monitors, film studios continue to release feature films (and I mean films here, not made-for-TV movies) on DVD in an interlaced format - at least here in North America

(and most disgustingly of all, in an interlaced format in a letterboxed widescreen frame in a 4:3 aspect ratio!!! - but 4:3 ratio is a separate gripe)

(and before anybody says "ALL DVDs ARE INTERLACED" - this much I do know! - but DVDs as we know, can and very often do have the picture field information stored with pulldown flags to allow reconstruction of original/source progressive film frames)

surely most feature films (if not all) are actually filmed (and indeed still released on FILM) with full-frames

I can only imagine there must be some budgetary or time-constraint reason for doing this, but perhaps somebody in the know here can fill in the gaps in my comprehension :-)

thanks for this great site!

m

[zambelli]

Forgive me for asking what may be an oversimplification,
but can anyone tell me why on the verge of 2007 - with so many people continuing to usher in a new hardware era of progressive monitors, film studios continue to release feature films (and I mean films here, not made-for-TV movies) on DVD in an interlaced format - at least here in North America
Can you give an example of a movie that was recently released on DVD and was hardcoded telecined?

[magic144]

- Stay Alive

- The World's Fastest Indian (well, the Canadian DVD version only, as far as I know - NOT the US disc - for reasons unknown, the Canadian DVD was released with a 2.35:1 ratio in 4:3 Letterbox, interlaced/hard-telecined... whereas the US disc was 16:9 anamorphic/progressive)

- Down In The Valley

- March of the Penguins - again, another CDN DVD version that was hard-telecined, whereas I believe the US was not (WHY!)

...are 4 recent releases that come to mind...

in fact, a recent film on DVD called Loverboy (dir. by Kevin Bacon) started hard-telecined, finished hard-telecined, and had true progressive video for the meat of the middle of the movie!

and "An American Haunting" was one of those infuriating 4:3 letterboxed interlaced numbers... grrrrr!

[Revgen]

^Hollywood studios hate Canadians. This is proof.;)

[Shinigami-Sama]

^Hollywood studios hate Canadians. This is proof.;)

we're a land of igloos dogsleds and beer
we dont have any of these fancy widescreen TVs
they wont fit in the igloos, and they're to heavy for the dogs to pull across our baren snow filled plains
:rolleyes:

[magic144]

well, the possible xenophobia of Hollywood aside,

is there a practical or financial reason why such hard-interlacing would ever be used in the authoring/production process???

[zambelli]

well, the possible xenophobia of Hollywood aside,
is there a practical or financial reason why such hard-interlacing would ever be used in the authoring/production process???
I'm guessing the reason is purely technical. The film masters are always delivered on tape, which means they're interlaced. The good encoders (used by the big studios and production houses) are able to detect 3:2 patterns and encodes the frames progressive when appropriate. Bad encoders just encode the whole thing as interlaced.

[Mug Funky]

all i can say is "suck it up" :p

we may be ushering a new era, but DVD is standard def. that means interlaced.

now if the encode monkey simply forgets to hit the "inverse telecine" button, does a 4 hour encode (and on decks that expensive, time is money), they wont bother to do the whole thing again if the tester says it looks alright (btw, the testers will be using pretty sweet setups. CRTs, but bloody good ones that go all the way up to 1080i).

just count yourself lucky you're not in PAL land. it's a bit of a lottery whether you get a good conversion, a bad conversion, or a native PAL transfer. where i work it's a real mix - some tapes come in already converted (it breaks my heart, because the current state of avisynth and it's plugins is better than 99% of the hardware converters out there), some tapes come in as NTSC, and some come in as native PAL transfers that look really good.

please bear in mind that just because a film was produced in hollywood with billions of dollars of experience and equipment behind it, the DVDs are usually outsourced to places with much less in the way of resources and equipment.

besides which, interlaced looks fine on a TV. it's only if you're backing up the disc that you run into problems, and after all DVD was never intended for that purpose - they're made to be watched, not re-encoded into a progressive-only, low bitrate codec and kept on a hard disk.

with that said, i agree that there should be more care taken in DVD production - but not every DVD house has people like us in it :), and from a QC point of view if it looks fine it is fine.

[magic144]

oh I'm sucking... and sucking hard!

thanks for the feedback guys...

1) movies are delivered on tape in a field-based format? wow - I'd just assumed they would be frame-based at least, since the original films must surely be frame based (progressive frames I mean!) - so the film studios interlace the movies themselves first before delivering them to the authoring houses?

2) I had a brief play around with IVTC using Donald Graft's IVTC software and AVISynth, but it always seemed to me that it wasn't guaranteed that the film would be PERFECTLY IVTC-able - that is, it never seemed to exactly follow the pattern ALL THE TIME, and sometimes Donald's software would highlight problems - was I doing something wrong, or will that generally be the case?? If so, what's the procedure for handling fields that don't "fit into the pattern"?? It would be awesome if you could point me at some good reading material for how to handle this stuff...

curious you mention PAL - I'm in the process of sourcing some PAL DVDs just because the UK seems to be the only place I can get a couple of films in Anamorphic Widescreen! The North American versions of said films are hideous fullscreen botches. One of the films in question is From Dusk Till Dawn - it's an American film for gawd's sake!!

as far as interlaced material on a TV goes - yes, perfectly watchable, never had an issue... the real problem I have with these discs is when I started to look at attempting to upscale some 4:3 interlaced material for use on my 16:9 TV - it's a darned nightmare of deinterlacing hell, which basically just seems to require hours of processing (especially if using any of Scharfis' top quality deint algorithms) for a frankly so-so result... it just left me wondering how my HD-upscaling DVD player managed to do such a good job "on the fly" with interlaced material and not produce any noticeable artifacts

[zambelli]

1) movies are delivered on tape in a field-based format? wow - I'd just assumed they would be frame-based at least, since the original films must surely be frame based (progressive frames I mean!) - so the film studios interlace the movies themselves first before delivering them to the authoring houses?
Mug can correct me if I'm wrong, but I believe the films are digitally scanned (progressive, of course) and then exported to D1 or D5 (http://en.wikipedia.org/wiki/D5_HD) tapes (interlaced) before they're shipped off to authoring houses for encoding. The encoders (often hardware encoders) take the interlaced signal and do IVTC in the encoder as they see fit.

It's a completely different process than what you and I would do on a PC with Avisynth and HC. :)

[celtic_druid]

From Dusk Till Dawn SE is about $13 here at the moment.

[magic144]

thanks again mates,

@zambelli
"do IVTC in the encoder as they see fit" - what does "as they see fit" mean exactly...?!

@celtic
is that PAL in Aus too... it seems weird to me (as a lot of things seem to!) that the only place you can get an anamorphic transfer of an NTSC film is in PAL land... :-)

@general
as for these tape originals
why would DG's IVTC see out-of-pattern fields anyway - if the video came from a progressive source, wouldn't/shouldn't the fields have a perfect 3:2 cadence, so that IVTC should be achievable in a perfectly reversible manner to recover the progressive original???

[Shinigami-Sama]

thanks again mates,
@general
as for these tape originals
why would DG's IVTC see out-of-pattern fields anyway - if the video came from a progressive source, wouldn't/shouldn't the fields have a perfect 3:2 cadence, so that IVTC should be achievable in a perfectly reversible manner to recover the progressive original???

nope

interlace
edit
fx
export


basic list of what they do
mug can correct me, he'll know more about it

[Guest]

as for these tape originals
why would DG's IVTC see out-of-pattern fields anyway - if the video came from a progressive source, wouldn't/shouldn't the fields have a perfect 3:2 cadence, so that IVTC should be achievable in a perfectly reversible manner to recover the progressive original??? Three things:

1. Source doesn't always have perfect 3:2 cadence. There is hybrid material and there is irregular pulldown.

2. Field matching, which is the first stage of the IVTC process that Decomb uses, is a heuristic process; it is not perfect. It can be confused by noise, jitter (from the telecine process), tape head switching noise, tape splices, line 21 data, and numerous other problems in the video.

3. There are often phase changes in the 3:2 pattern (due to edits). Decomb's pattern matching has to try to recognize these and distinguish them from the effects described in 2 above. That too is a heuristic process. It's hard to lock onto a pattern in a static scene because the only things the matcher sees are the artifacts just described. So the pattern locker tries to pretend that it still sees the previously locked pattern. The pattern matcher has to decide between trying to continue the current pattern or changing to the new pattern phase.

I'm sure that dashes your hopes for perfect IVTC (except for soft telecining). It's lucky for us that it performs as well as it does! The postprocessing to catch combs that sneak through is the final safety net. Even that is heuristic because you have to decide whether a frame is combed, and that requires distinguishing combing from real image detail at the spatial frequency of combing.

Read my 2003 journal here for discussion of heuristics used by Decomb:

http://neuron2.net/journal/journal2003.html

[magic144]

aye carumba!

is the situation any different in HD-land (Blu-Ray/HD-DVD) - or are we still looking at interlaced there too? I get the impression that these new discs are 1080p24 encoded

I'm still a bit confused though - I mean, to make a film, there MUST be a TRULY progressive original (i.e the film's projection frames) - so surely there must be a regular/non-hybrid/non-digitally-post-edited source of this stuff which is
a) what was shown in the cinema
b) what COULD be perfectly cadence-encoded and therefore not require a heuristic process to recover

I guess there must be a bunch of edit "fiddling" in between showing the film in public and preparing it for video release...

Out of interest, for the majority of new DVDs that ARE progressively encoded, are THESE stored with perfectly regular 3:2 cadence, or does the cadence shift around within these films too (the flags just keep adjusting for the frames' reconstruction?)

Thanks so much for the explanations - even though they hurt my head!

I guess the DecombTutorial in the Decomb package explains most of the blend/interpolate options and consequences - I guess that in the absence of "perfectly regular" cadence, reconstruction of a progressive stream is going to involve some sort of compromise/guesswork...

thanks you guys!

[celtic_druid]

Yep PAL. A lot of stuff here is R4/R2 so I guess the same transfer as the UK version. Just different packaging.

Details:
http://www.ezydvd.com.au/item.zml/220500
Works out to about $16 Canadian including shipping. Not sure what it would cost to get from the UK and often ezydvd aren't the cheapest here.

[magic144]

@celtic_druid
cheers celtic - yeah, it's actually about $16Cdn (equivalent) from amazon UK too - for the 2-disc collector's edition, with the "Full Tilt Boogie" feature-length documentary on disc2 to boot, so I'll probably go with that next time I order (it's getting ridiculously frequent lately - customs and excise must love me and my Visa card!!)

[Mug Funky]

@ zambelli:

it's usually Digibeta, sometimes beta SP (yuck!), and sometimes even 2Vuy mov files on hard disks, or DV tapes, or HDV. anything we can't handle here gets sent out and dubbed to digibeta at a duplication house.

i can't speak for other places though - some have much higher budgets :)

@ magic144:

most broadcast tape formats are interlace agnostic - the colour sampling (usually 4:2:2) is such that you can put either on it without problems (unlike the 4:2:0 on DVDs). it's not so much that they're interlaced in the transfer process - it really can be either.

the IVTC (if it happens) happens inside the hardware encoder usually - it'll simply compare the current field with the previous+1 field, and if they're the same it'll store the current field as a "repeat first field" flag. this means the bits per frame goes up considerably while the actual bit rate doesn't change (hence the quality is better). also it'll be a more efficient and cleaner encoding.

i don't know of any hardware encoders that do this for PAL though... on the MPX3000 the IVTC button is greyed out in PAL mode (and wouldn't be all that useful anyway).

[Guest]

I'm still a bit confused though - I mean, to make a film, there MUST be a TRULY progressive original (i.e the film's projection frames) - so surely there must be a regular/non-hybrid/non-digitally-post-edited source of this stuff which is
a) what was shown in the cinema
b) what COULD be perfectly cadence-encoded and therefore not require a heuristic process to recover The telecine process involving stepping the film frames through a physical mechanism that has mechanical tolerances, so there is jitter in the vertical alignment. Then the result is often stored on analog tape, which is subject to noise. Also analog broadcasting and analog capture add noise. Even digital scanning has noise. Then there are the splices, line 21 data, etc., that I mentioned. Sometimes you get a DVD with perfect hard telecining, because the whole process is digital, but it's rare because if you're staying digital throughout, you may as well use soft telecining. In those rare cases, IVTC can be flawless (and you'd turn off postprocessing). I'm sure Mug Funky could regale us with horror stories about how clean film source material is degraded before it reaches our hard drives.

It's just a plain fact of life that hard-pulled-down field "duplicates" are usually not truly identical. If they were, then perfect IVTC would be possible (for a pure 3:2 sequence).

I guess there must be a bunch of edit "fiddling" in between showing the film in public and preparing it for video release... That too in addition to the above.

[zambelli]

is the situation any different in HD-land (Blu-Ray/HD-DVD) - or are we still looking at interlaced there too? I get the impression that these new discs are 1080p24 encoded
Yes. The process for encoding HD-DVDs has been different. Microsoft has supported major studios in their encoding of all the North American HD-DVD titles released so far. The VC-1 encoder provided to them is a x86 software-only encoder that takes raw YUV videos as input. The source videos are transferred on D5 tapes and captured to the PC as 1920x1080 10-bit 4:2:2 YUV files. This is where the pre-processing is done: scaling if necessary and dithering to 8-bit 4:2:0 YUV. D5 supports 1080p/24PsF so the sources can already be progressive. If they are 1080i/30, IVTC is performed. The video is then finally encoded and flagged for telecine in the encoder, much like you would do with HC or TMPGEnc yourself.

I'm still a bit confused though - I mean, to make a film, there MUST be a TRULY progressive original (i.e the film's projection frames) - so surely there must be a regular/non-hybrid/non-digitally-post-edited source of this stuff which is
a) what was shown in the cinema
Right. The celluloid film is progressive, of course. However, Early telecine machines existed entirely to covert films to broadcast TV formats. All the SD video editing equipment was also designed for broadcast TV, and so on... tape decks, mixing boards, etc. The whole end-to-end solution was interlaced NTSC or PAL (or SECAM).

[magic144]

@all
thankyou all for such an illuminating discussion

@zambelli
I was reading something yesterday about the PS3's Blu-Ray capabilities and had not appreciated that early Blu-Ray discs used MPEG-2 whereas HD-DVD was using VC-1

There certainly seems to be a lot of chatter on forums about the superiority of VC-1, some even saying they would never buy Blu-Ray in case they ended up with MPEG-2!! Others were clamoring for Blu-Ray discs to specify on the packaging which codec was used (reminds me of the whole anamorphic widescreen labelling debacle which STILL is not mandated on DVD packaging)

Does HD-DVD NOT use MPEG-2 then? (or does it just choose not to)? Is Blu-Ray shifting towards using VC-1 as a standard/matter-of-course? Looking at the spec for HD-DVD on Wikipedia, it shows MPEG-2 as an option here too - why are people so sure it won't be used on an HD-DVD release??

Finally, I'm not sure from your answer, but presumably you could still encode a letterboxed 4:3 widescreen interlaced (hard- or truly-interlaced) video onto a HD/Blu-Ray release (you really shouldn't, since HD specs are for widescreen display devices!) - it's an awful prospect though...

[SeeMoreDigital]

You raise an interesting point Magic....

As things stand with good 'old DVD. We all know (by default) that the video will be MPEG-2. And a quick look at the back of the DVD case informs us of the audio stream formats...

From what I understand with Blu-ray and HD-DVD, we'll still be able to look at the back of the case to determine the audio stream formats.... But NOT THE VIDEO stream format (which could be any one of three competing HD formats) - Which I think is inexcusable :devil:

[Inventive Software]

BluRay won't use VC-1 cause Microsoft's supporting HD-DVD and VC-1's Microsoft's bitch. AVC is supported by Toshiba, who also back HD-DVD. So in order to appeal to their shareholders they use the bloatable MPEG-2 codec!

[magic144]

@Inventive

apparently they already are...???
http://www.engadgethd.com/2006/09/07/blu-ray-movies-in-vc-1-up-to-par-with-hd-dvd/

@SeeMoreDigital
I heartily agree - I think the consumer has a right to know ALL the relevent info before they lay down their ca$h - movie houses really do need to take some pretty basic steps to engender some good faith with the consumers in areas like this one

btw - don't want to go waaaaaay off topic now and into a discussion on the future of HD discs - no doubt that belongs in a separate/existing thread...!!!

[Morte66]

Has anybody seen comparisons of Japanese HD DVDs using h264 to VC-1?

[zambelli]

Does HD-DVD NOT use MPEG-2 then? (or does it just choose not to)? Is Blu-Ray shifting towards using VC-1 as a standard/matter-of-course?
Studios releasing BluRay titles have so far chosen to use MPEG-2 for the majority of their releases. Sony has a very tight grip on its own studios (Disney, Sony Pictures, MGM, etc.), and has so far preferred MPEG-2 on their BD releases for reasons that I won't go into. However, there have been several VC-1 encoded BD releases too, so VC-1 is making progress there too. It's ultimately up to the studios who own the movies to decide which format they want to go with. The ones that released on both HD-DVD and BD have gone with VC-1 because it allowed them to encode the movies only once and use the same encode twice.

Looking at the spec for HD-DVD on Wikipedia, it shows MPEG-2 as an option here too - why are people so sure it won't be used on an HD-DVD release??
I think it's fair to say that major HD-DVD releases will never use MPEG-2 due to space constraints. It's simply not efficient for them to use a codec that takes up twice as much space than VC-1 and H.264.

[SeeMoreDigital]

....However, there have been several VC-1 encoded BD releases too, so VC-1 is making progress there too. Are you able to tell us what those releases are please?

[zambelli]

Are you able to tell us what those releases are please?
Magic already linked the article - check above.

[SeeMoreDigital]

Thanks... I missed that link :o