Working Utility for Hybrid-framerate MPEG-2 (using RFF/TFF) in TMPGEnc

[dbloom]

http://davidbloom.home.mchsi.com/
(both the download and the tutorial with many screenshots are on that page)

This is a utility I made (and a tutorial on how to use it) that allows you to create a manual IVTC in TMPGEnc without having to pay attention to audio sync - so you can use a pattern like 10100 on 24fps parts, 10 on 30fps parts, and so on. You can then pass your TMPGEnc project file through my utility, and it generates a force-picture-type file that sets bitrates in TMPGEnc to compensate for the speedup in parts above 24fps, so that you don't go above your maximum bitrate. After encoding your file, you run the m2v through my utility, and it will apply the proper RFF/TFF flags to the MPEG-2 video headers. Then, multiplex in the audio and enjoy mixed-framerate MPEG-2! :)

Because it uses RFF/TFF, the output is fully MPEG-2 compliant (except for the simple profile, which is only used for some older DVB systems) and is fully compliant with SVCD/DVD standards.

My utility is basically a proof of concept, and it has some limitations. It will probably max out on files over 18 minutes long, it can crash occasionally, it requires a manual TMPGEnc IVTC (so you can't use an automatic system such as Telecide), and it's slow. Really, I made this program (in VB6, I might add) purely out of neccesity, and it's not designed to be a pernament utility to do this sort of thing. Ideally, an experienced programmer can code a (better) replacement for this utility, or I'm going to start studying more C++ :D. If anybody volunteers to code it, I'd be glad to help out with the specifics in the TMPG project/MPEG-2 file formats, etc. I'll even send my source code, but I don't think you'll like it :devil:. Hopefully then we'll have something that can work with automatic IVTC too ;).

Video streams run the utility work about as reliably as regular SVCD and DVD content. MPEG-2 video streams run through this utility have worked flawlessly on the dozen or so DVD players/software video players I have tested on, in both SVCD and DVD formats (this doesn't work with VCD because MPEG-1 doesn't support the pulldown codes this method requires).

I spent a lot of time researching/reverse engineering/testing/wandering around aimlessly/coding to make this (I had never messed with the inner structure of an MPEG-2 video file before), and nobody in other forums was patient enough to deal with TMPG manual IVTC to try it out :(. If you do use it, let me know what you think - I've invested a lot of time into this and feedback is my only reward.

Good source materials: anime (obviously), NTSC music videos

If you want a sample MPEG-2 file patched by this program, PM me (http://forum.doom9.org/private.php?s=&action=newmessage&userid=75500) your email address, and I'll YouSendIt (http://www.yousendit.com/) it to you.

[dbloom]

Apparently this is a popular request among people who make DVD backups: the ability to copy the pulldown from another m2v file.

I'll see if maybe I can make a m2v-pulldown --> TMPG-project-file IVTC converter or something when I get home :).

[dragongodz]

enjoy mixed-framerate MPEG-2!
you have to be kidding. hybrid framerates are one of the stupidist things invented. thank god we in PAL countries dont have to suffer such rubbish.

[dbloom]

Originally posted by dragongodz
you have to be kidding. hybrid framerates are one of the stupidist things invented. thank god we in PAL countries dont have to suffer such rubbish.

Yeah you do. It's just converted/mangled from NTSC to PAL first :).

[dbloom]

Look, I don't want to bump my post just for attention, but I put a LOT of hours into this and don't want it to go to waste. If it doesn't work for you guys, what does it need? At least give me some suggestions, I know there's got to be some good in it :).

[dragongodz]

Yeah you do. It's just converted/mangled from NTSC to PAL first
you know what i meant.

I know there's got to be some good in it
nope. there is nothing "good" about hybrid framerates no matter how good your util is. ;)

[dbloom]

Originally posted by dragongodz
you know what i meant.


nope. there is nothing "good" about hybrid framerates no matter how good your util is. ;)

This isn't about using hybrid framerates because you can, it's about using them because they are used a LOT in NTSC stuff. Basically, either you do this or you drop frames. DVD backups, TV encodes, etc could all benefit from this.

[dragongodz]

it's about using them because they are used a LOT in NTSC stuff.
doesnt make them good. far from it. :D

Basically, either you do this or you drop frames. DVD backups, TV encodes, etc could all benefit from this
yes i know. IMHO you are better off dropping frames than continuing a bad practise. each to their own though. :)

by tv encodes do you mean tv captures ? if yes then they shouldnt be hybrid anyway.

[dbloom]

Originally posted by dragongodz
doesnt make them good. far from it. :D


yes i know. IMHO you are better off dropping frames than continuing a bad practise. each to their own though. :)

by tv encodes do you mean tv captures ? if yes then they shouldnt be hybrid anyway.

Hybrid is seen very often with music videos and anime. I don't think they're going to stop doing it for a few mpeg-2 hackers who cut out their commercials :).

By the way, in the TV case, the signal is interlaced, but you can't really IVTC it right because when you do it's hybrid. I don't mean the signal's hybrid to begin with - it's just IVTC = higher quality, especially on a computer or progressive scan TV.

[dragongodz]

Hybrid is seen very often with music videos and anime. I don't think they're going to stop doing it for a few mpeg-2 hackers who cut out their commercials
never said they would. i just think its not a good idea to continue it at home. as i said though, each to their own.

EDIT:
i wasnt going to comment on the second part but i do have to say that sounds more like a self inflicted wound. ;)

[dbloom]

Originally posted by dragongodz
never said they would. i just think its not a good idea to continue it at home. as i said though, each to their own.

EDIT:
i wasnt going to comment on the second part but i do have to say that sounds more like a self inflicted wound. ;)

Who said being a MPEG-2 hacker is bad?

[makes a scale with his hands] Money-obsessed copyright lawyer...MPEG2 hacker...copyright-obsessed...MPEG-2....

I'll be the MPEG-2 hacker.

[dragongodz]

Who said being a MPEG-2 hacker is bad?
huh ? let me expand what i said a little so there is no confusion.

i just think its not a good idea to continue such a practise as hybrid framerates at home just because commercial companies use it. as i said though, each to their own.

[dbloom]

Because nobody has taken interest in doing the job for me, I'm working on coding this again, the right way ;).

I'm still using Visual Basic, but it will be much more optimized and flexible. Also, it will have a wizard-like (step by step) interface to make things simpler.

Here's what I think would be nice to be able to extract a pulldown *from*:
an MPEG-2 m2v file
a DVD2AVI D2V file
a TMPGEnc project file (for manual IVTC)
debugging information from Telecide or Decimate created when using the debug (http://neuron2.net/decomb/DecombReferenceManual.html#advanced) parameter in either (is anyone familiar with what kind of debug info Telecide and/or Decimate outputs?)

Then, once a pulldown is loaded:
Add/remove frames as needed to make it match the MPEG-2 REPAT_FIRST_FIELD capabilities (since we are able to make a frame last either 2 or 3 fields, and we may need to add copied frames/remove frames to make it match that)
Apply it to an m2v file
Set the IVTC pattern in a TMPGEnc project file to the pulldown (for manual tweaking to an automatic IVTC maybe, or to avoid having to do the AVISynth thing below:)

Also, perhaps 60fps (with copy frames) IVTC pattern export, so then you can output to AVI in TMPG using drop frames to achieve the pulldown in an AVI file.


Generate an AVISynth script that uses conditions or something to output each unique frame without regard to framerate, etc, so we can apply the pulldown to it once it's MPEG-2 encoded.
Generate a TMPGEnc forced-bitrate text file so the bitrate doesn't go up above the format restrictions on higher frame-rate parts (this would unfortunately force us into CBR)
I think CCE has advanced VBR capabilities that include outputting/inputting a file defining bitrate (I've never used CCE so I'm not sure). Anyway, that could be loaded, adjusted to compensate for the mixed framerates of the pulldown, then exported back to CCE.

Any other ideas? Or help :)?

I've already coded a basic rich text display system for the instruction text in the wizard so I don't have to include any OCX's or anything. The program will display instructions (with many helpful screenshots of the other programs the user has to work with) for every step of the process. The EXE for this should work alone as long as the VB6 runtimes are installed (VB6 runtimes come with Windows XP, and I think they come with Windows 2000 as well. The instructions are stored as resources so they are built into the EXE as well.).

Hopefully the new features and ease of use will make this tool more useful to people working with DVD backups, so that they don't have to do blending or anything for hybrid DVD sources. Also, the automatic IVTC (you all are so lazy...) import will make it good for TV show recordings and such.