ATI MMC v9.02 DVD-compliant MPEG-2 captured files – problem editing (but MMC v8.5 OK)

[Internaut]

Greetings to you all.

I have an ATI AIW 9800 Pro video capture card which I am very happy with. I record a lot of TV programmes off-air directly using the ATI’s MMC software supplied with the card, and after some basic editing/trimming with Womble Mpeg Video Wizard (mainly to cut out commercials), I use Adobe Encore to author the mpegs to DVD’s.

I have found this choice of software to be optimal because :-

1) ATI MMC is the best and only package which fully supports the 9800 Pro card (not surprising as it is supplied by the card vendor !) … InterVideo WinDVD Recorder works OK but results not as good as MMC.

2) Womble MVW – best bang-for-the-buck MPEG editor that I am aware of – I believe M2-Edit is probably better but a bit outside my price range. Womble has proved to be very easy and reliable and does all that I need at the moment.

3) Adobe Encore DVD is a very versatile DVD authoring package, allowing virtually limitless flexibility, albeit at the expense of a bit of a learning curve. I have also tried most of the other well-known authoring packages (TMPEGEnc DVD Author, DVDLab, InterVideo WinDVD Creator, Ulead etc) …. Nothing wrong with any of them, but some of these tend to be one-click “pretty” apps and being an Engineer myself I like the nuts-and-bolts philosophy of Encore, which gives you total control of the authored result.

As the final destination for the recorded material is DVD, I capture it with MMC as MPEG-2 DVD-compliant files right from the start. I do not want to waste time transcoding at ANY step of the process ie after capture, editing or during authoring. This works well for me, and I can author DVD’s very quickly, having eliminated the need for any transcoding from the process altogether.

Until recently, I have been using MMC v8.5 to do all my MPEG-2 captures (this was the MMC version which was originally supplied with my 9800 card). However, the current version of MMC is now v9.02, as per the ATI support site, and there have been several updates in between. I have just upgraded to the latest MMC v9.02, following all the installation notes and manuals very carefully - everything works fine.

Now for my problem :-

When recording DVD-compliant MPEG-2 files with MMC v8.5, both the original unedited and Womble-edited versions of these files remain DVD-compliant, as indicated when importing into Encore, which allocates a Transcode Status of “Don’t Transcode” to all the files. HOWEVER ….now using MMC v9.02 with the identical settings as before with MMC v8.5, when capturing DVD-compliant MPEG-2 files, although the ORIGINAL MPEG-2 captured file is accepted by Encore as DVD-compliant, SOME of the edited pieces from Womble are not; these get the Transcode Status of “Untranscoded”, which indicates that Encore regards these as non DVD-compliant, and will transcode these prior to authoring the final DVD file set.

• So far we have one variable ie the MMC version (8.5 or 9.02)

• A second variable is the version of Womble MVW. I have used builds 12-31-2003, 02-06-2004, 04-23-2004 and the current 09-24-2004, and the above behaviour is unchanged.

• A third variable is Encore. I have used Encore v1.0 (this was the first release), the v1.0.1 update, and the current v1.5, and the above behaviour is again unchanged. I have also cross-checked by using a second authoring package ie TMPGEnc DVD Author, and it also has problems with the same files that Encore does.

This suggests that MMC v9.02 could be the culprit. It would seem that MMC v8.5 produces DVD-compliant files, which remain compliant even after editing with Womble, which is the desired situation. However, MMC v9.02 produces DVD-compliant captures which in some way become non-compliant after they have been edited with Womble.

As a newcomer to this excellent forum, I lurked for a while before posting, and was interested to learn that Encore is based on a Sonic SDK, “most probably following Sonic DVDit!'s long, infamous, history of input-file-fussiness.” This may have some bearing on the problem.

I attach a snapshot of the Encore Project Manager window. Files 1.mpg, 2.mpg and 3.mpg are the original MPEG-2 DVD-compliant files captured with MMC v9.02. I then used Womble to chop these three files into 4, 3 and 2 parts respectively, resulting in files 1a, 1b and 1c, 2a etc. Note that of the 4+3+2 = 9 files, Encore sees 5 of these as compliant (marked as “Don’t Transcode / N/A”), and the remaining 4 as requiring transcoding (marked as “Untranscoded / Automatic”). Furthermore, note that in the “Bitrate” column, the value for ALL the non-compliant files is 8.6 Mbps. The original files 1, 2 and 3 were all recorded at VBR nominal 4Mbps to maximum 5Mbps. So this reported bitrate of 8.6 Mbps seems spurious to me. The compliant files all report plausible bitrates, knowing from where they came.

Repeating the above exercise using MMC 8.5 results in a complete set of DVD-compliant files, and this is how I have been successfully operating prior to upgrading to MMC v9.02.

My own investigations thus far :-

• LordSmurf’s article “Capturing MPEG with an ATI card” at http://www.digitalfaq.com/capture/atimpeg/atimpeg.htm is interesting but unfortunately does not solve my problem – I tried his suggested MMC settings, taking note of his comment about closed GOP’s and Sonic Solutions, on which Adobe Encore is based.
• As mentioned above, I cross-chcked with TMPGEnc DVD Author (v1.6.26.73) to see if it would accept the files that Encore would not, and in most cases it reported the error “The video sequence header is incorrect”. I discovered that this is a well-documented error message on the forums, so ……
• VideoHelp Forum has an article “Inserting MPEG Sequence Headers with TMPGEnc” at http://www.videohelp.com/forum/userguides/126292.php which describes how to use TMPGEnc Encoder (not DVD Author) to fix this problem with MPEG-1 VCD files; I tried it with my non-DVD compliant MMC v9.02 Womble-edited MPEG-2 files and it fixes the problem – Encore imports them as DVD-compliant ie no transcoding required. In the snapshot, this is the file 1b (fixed).mpg
• There are plenty of references to the importance of GOP sequence headers in DVD-compliant MPEG-2 streams, and how not all MPEG-2 encoders strictly comply (maybe MMC v8.5 does, but MMC v9.02 dies not ?)
• Using GSpot to examine the unedited MMC v8.5 and v9.02 captures, (the latest version supports MPEG now – nice !), the bitrate of the v9.02 files is always reported as 10080 kbps, which I recognize as the maximum combined audio/video Program Stream rate for DVD-compliant MPEG-2. It seems like GSpot is looking for a value, not finding it, and so defaulting to the maximum value. The v8.5 captures, however, are reported at the correct bitrates.

Other relevant pieces of information :-

• When using Womble to save the edited MPEG-2 file, I use the “Automatic” setting, which streams the edited file to disk without retranscoding it ie a “don’t-touch” mode … this is a very nice feature of Womble !
• I am in PAL-land (South Africa), but I get the same problem editing NTSC (740 x 480) DVD-compliant files from MMC 9.2 with Womble.
• I can use Womble to edit decrypted VOB files (using DVD Decryptor) straight off a commercial DVD (which by definition are DVD-compliant !) and, as I would expect, both Encore and TMPEGEnc DVD Author accept all the edited pieces as DVD-compliant. This moves the focus away from Womble as the culprit.
• The settings I use for MMC v9.02 are identical to those I originally used for MMC v8.5 (during upgrade of MMC, it asks if you want to retain your old settings to which I reply “Yes”.)
• Some time ago, I contacted Womble technical support, who could not shed any light on the problem – as noted above, I don’t think it’s a problem with MVW. I have not contacted ATI as I believe they just do not reply to technical queries – even if you live in Canada ! I have not contacted Adobe, but I searched the Encore forum exhaustively with no joy.

My gutfeel :-

• MMC v9.2 captures DVD-compliant MPEG-2 files which in some way have a sensitivity to editing that those from MMC v8.5 do not
• Could there be problems in the header or GOP seqeces of the non-compliant edited MPEG-2 pieces ? If so – what ? Are there any tools which could detect and/or fix this ?

My PC environment :-

Intel “Brownsville” M/B with Intel P4 2.4G CPU, 512MByte RAM, Promise Ultra 133 card, 4 x 120G plus 1 x 80G Seagates, Plextor PX-708 DVD writer, Plextor PX-W4012A CD writer, Pioneer DVD-105 DVD drive

Windows XP Pro with SP1a installed

ATI MMC settings :-

720x576 25fps MPEG I Layer 2 48kHz 16-bit mono audio @ 128kbps, video VBR 4 to 5 Mbps
These settings are PAL DVD compliant

Although MMC v8.5 gives me a working solution, I would like to move on with MMC v9+ which has some other nice features such as MPEG-4 capture which is nice.

Any pointers – this problem has been driving me nuts ! Any help to solve the problem would be greatly appreciated !

[Sulik]

It's sounds like this is really a bug in womble, which has some problems cutting some of the files captured with MMC 9.02.

You don't have any more details on the compliance error ?

The 10.08Mbps is the program_mux_rate (not the video stream bitrate), which is normal for DVD-compliant. I doubt womble would have a problem with that.

I noticed that MMC 9.02 has automatic scene changes detection, causing GOP length to vary, whereas it was fixed in 8.5 (easier for a cutting tool).
One possibility is that it automatically inserts chapter points in the stream at scene changes (closed GOPs), where B-frames use backward-only motion, even when the closed-gop option is off (selecting closed_gop forces all GOPs to be closed, not just scene changes). If the cutting tool doesn't handle this properly, it may generate an incorrect GOP. You could verify if this is the problem by modifying the preset to not use B-frames at all (P=14, B=0 instead of P=4, B=2).

[Internaut]

Sulik,

Thanks very much for the reply, which pointed me in the right direction.

I have isolated the problem, and have sent a detailed report to Womble, attached here for perusal.

The root cause of the problem was the way MVW handles the automatic scene detection in the newer MMC v9.02, which was not present in the previous MMC v8.5. MMC v9.02 implements the automatic scene detection in one of two ways :-

A. Truncating the current GOP and starting a new one
B. I-frame insertion in the current GOP

A and B occur throughout the MMC v9.02 capture file; MVW handles A properly, but with B it appears to be generating extra GOP's without proper sequence headers; this is why the authoring software rejects the MVW-edited file.

I explain in detail in the attached report.

How did you know about the scene detection in MMC v9.02 ? I can't find it documented snywhere in the MMC Release Notes or anywhere else on the ATI site. Now that I understand more about this, I guess you could pick this up by observation with Bitrate Viewer ?

Regards,

Internaut

[Internaut]

Follow up :-

Womble have come back to me and fixed the problem ! They sent me a pre-release version with the bug fix, and I have tested it and it works.

I and probably thousands of other ATI MMC users who also use MVW can expect trouble-free DVD-compliant edited captures with the next release of MVW.

I am still a bit baffled that other people had not picked up the problem also !

Regards to all,

Internaut

[Sulik]

Great news!

I work with MPEG encoding, so I have a bunch of tools at my disposition to analyze MPEG files.
I'm guessing Womble used to assume that a new I-frame means a new GOP, so they insert a new GOP header when rewriting the bitstream, even though it is perfectly legal to have an I-frame in the middle of a GOP (though it is uncommon, it can be slightly more efficient).

This was also 'legal' for Womble to do this (from the MPEG's spec point of view), but unfortunately not compliant with DVD specs, which require a sequence header before every GOP.

Note that I'm now using MMC 9.03 and I noticed that this version always starts a new GOP at scene changes (probably for improved compatibility with Womble and possibly other editors).

[Internaut]

Thanks Sulik.

I downloaded MMC v9.03 a few days ago, but have as yet not installed it, so I will give it a shot and confirm that it now starts a new GOP at every scene change, and not only at some of them. This should make even the unfixed Womble version work OK.

Once again I am reminded about the differences between MPEG-2 compliance and DVD-compliance ! Womble were MPEG-2 compliant in what they were doing, but not DVD-compliant, as you mention. DVD puts you into quite a straightjacket, whereas MPEG-2 is very flexible by comparison.

You raise some questions in my mind about which I am not too clear :-

1) Surely a GOP, by definition, can only contain ONE I-frame ! Doesn't an I-frame in the stream mark the beginning of a new GOP. as per the definition of a GOP ie "a Group Of Pictures, commencing with an I-frame, and terminating with the frame immediately prior to the next I-frame" ? I have read up on this somewhat, and this is my current understanding of a GOP, so a "GOP with two or more I-frames" is a contradiction ?
2) What is the difference between a GOP header and a sequence header, if any ? (I probably need to do some reading on this one).
3) Are you sure that the DVD spec calls for a sequence header per GOP ? I am under the impression that this is only a recommendation, since chapter points on a DVD may only exist at I-frame boundaries with sequence headers, hence the recommendation which maximises chapter point positional accuracy.

I attach an updated version of the the spreadsheet file "MMC v9.02.pdf" which I previously posted, which has two more tables added :-

i) TABLE 3, which shows how TMGEnc was able to fix the file from the current version of MVW, as mentioned in my Bug Report "Other relevant notes 3)"
ii) TABLE 4, which demonstrates Womble's bug fix. This is the same analysis on the same input file run through the fixed pre-release version of MVW. Note how all the "No's" under the "Has SeqHdr" column in TABLE 2 have become "Yes" (indicated in bold) in TABLE 4, which now is essentially the same as TABLE 3.

[Sulik]

1. The first decoded frame in a GOP must be an I-frame. There is no other restriction for picture types within a GOP. In some [rare] cases, if images are very different, I-frame coding may be more efficient than P-frame or B-frame coding. Not starting a new GOP in this case avoids putting additional sequence and GOP headers before these frames (hence increasing compression efficiency, though by a minimal amount). The GOP header itself doesn't indicate how many frames are in the GOP. All frames belong to the last GOP header, until the next GOP header is found in the bitstream. The short answer is multiple I-frames are allowed in a GOP (there is no contradiction).

2. The GOP header includes very little information. It is only here to indicate GOP boundaries, and it includes the timecode of the first frame, weither the GOP is closed (ie: if the first B-frames in the GOP can be decoded without any dependencies on the previous GOP).
On the other hand, the sequence header includes important information such as the picture size, aspect ratio, coding parameters etc. The bitstream cannot be decoded until the sequence header has been parsed.

3. The reason why the DVD spec requires a sequence header before every GOP is so that the decoder can easily seek to any GOP and start decoding immediately without having to search for the previous sequence header (combined with other DVD restrictions, this also guarantees at least 2 seek points per second, ie: 16x fast-forwarding possible by decoding only the first I-frame of every GOP).
Chapter points have additional restrictions that are outside of the MPEG-2 spec (DVD navigation layer has additional information for chapters)

[Internaut]

Thanks Sulik, you have explained these points very well. It all makes a lot more sense now, particularly embedded I-frames within a GOP, which, although probably not the norm, are quite legal. I do recall having seen such GOP's in commercial DVD's, though not that often.

What Womble were doing was truncating the current GOP at the embedded I-frame, presumably generating a new GOP header (which is how BRV identified that there is a new GOP), but not generating a GOP sequence header as required by the DVD spec.

I guess that with DVD's requiring a new sequence header for every GOP, this gives you a degree of freedom to change some of the parameters dynamically during playback.

Regards,

Internaut