shortened GOP

[lovelove]

Hi. Is there a way to fix "shortened GOP"s in MPEG2 video?
Or is this an irrecoverable condition?

[lovelove]

Background:
I was muxing m2v->mkv with MKVmerge, the process finished but there were warnings ("Shortened GOP detected. Some frames have been dropped. You may want to fix the MPEG2 video stream before attempting to multiplex it.")

Both the DVD and the extracted m2v play without any errors in VLC and MPC-HC. The converted mkv file plays fine too basically, but there are a few instances where - if you pay very close attention - the scene cut is not clean, but you can see what looks like a Powerpoint checkerbox transition effect for a few milliseconds (as if a few frames were missing, which corresponds to the MKVmerge warning). The DVD disc is not physically damaged (I made a bit for bit verification).

[lovelove]

OK, I did my research in the meanwhile.

Wiki says:
Maximum frames per GOP: 18 (NTSC) / 15 (PAL), i.e. 0.6 seconds both

It seems there is an upper limit for frames per GOP, but no lower limit. Therefore I don't really understand why a GOP could be "too short".

Especially so since the GOP layer doesn't even seem to be mandatory:

The GOP layer is optional, i.e. it's not mandatory to put any GOP header in the bitstream.
[...]
the GOP length is the period (often expressed in frames) by which an Intra frame occurs. It must be noticed that such a value cannot be found in the bitstream and it is unnecessary to the decoding process.

this is confirmed Dan ROsen (author of VideoReDo):

This begs the question what is the purpose of the GOP timecodes? We don't use them at all in VideoReDo and neither do most PC software players or DVD players. GOPs are actually an "optional feature" in an MPEG stream. So, we don't really get concerned about the difference of a frame or two in the GOP calculations by some programs.

[setarip_old]

In your second post to this thread, you quote a warning as, specifically, "Shortened GOP detected. Some frames have been dropped."

Then, in your third post to this thread, although in quotes, you modify the condition as follows, "I don't really understand why a GOP could be "too short"."

I believe you've misinterpreted the warning - It likely means that a GOP has been shortened as a consequence of some frames having been dropped.

The user's concern should likely be about replacing the missing frames, not the GOP...

BTW - There's no reason that your first two posts (10 minutes apart) couldn't have been combined by simply editing the first one ;>}

[lovelove]

Thanks for commenting. Well you know that I know not a whole lot about the subject (otherwise I wouldn't have quoted wiki). And no, I didn't "modify the condition":
Post #2 clearly quotes the error ("shortened GOP") and sums up what happened (that's what you call "the condition").
Post #3 is only theoretical research and my guesswork. (Hence the interest to separate the posts, because they are thematically different.)

Actually I intended to steer a possible discussion onto a generic level focusing on the question what shortened GOPs exactly are and whether or not they can be fixed if occuring in MPEG2 video. I kept this is the vital part in post #1 separate from the dispensable information in post #2 and #3 to signal to the reader that #1 is all he needs to know to reply and the rest is just there as a bonus if he is interested.
But this discussion is off-topic.


With regards to your reply: to me "shortened" and "too short" is the same thing. If MKVmerge issues a WARNING about a shortened GOP, it is saying that the GOP is "too short" to be acceptable (otherwise it wouldn't issue a WARNING, right?).

I believe you've misinterpreted the warning
My intention was not to interpret it, but to ask what it means exactly.

The user's concern should likely be about replacing the missing frames, not the GOP...
ok, so you mean that the frames are not there at all, i.e. the image data is not in the mpeg stream at all for these frames? (as opposed to: the frames are there, but not or badly referenced - e.g. in a malformed GOP - with the result that they are dropped while muxing ... because this is EXACTLY what MKVmerge's warning says: "Some frames have been dropped."!!)


What I don't quite understand:

The point is, MKVmerge drops frames and the reason it gives for this is "shortened GOP". Now given that the GOP layer is not mandatory as pointed out in #3 (and thus not needed for MKVmerge muxing), then an error in the GOP layer cannot legitimately be a reason for dropping frames, can it?

[setarip_old]

With regards to your reply: to me "shortened" and "too short" is the same thing. If MKVmerge issues a WARNING about a shortened GOP, it is saying that the GOP is "too short" to be acceptable (otherwise it wouldn't issue a WARNING, right?).

No, that is not correct - The informational warning advises that that a GOP has been shortened (NOT necessarily "too short") as a consequence of missing frames.

e.g. Although my tailor shortened my pants, he did not make them too short...

[lovelove]

I understand the difference in principle, but not in this specific case. Can you explain it in terms of GOPs and frames instead of pants, particularly with regards to the last § in #5 ? Rememer, I am not knowledgeable about these terms.

[Guest]

Consider a GOP in encode order. The first field is the picture type and the second is the temporal reference.

I 2
B 0
B 1
P 5
B 3
B 4
P 8
B 6
B 7
P 11
B 9
B 10

That is a complete GOP and all pictures in decode order can be decoded and displayed in their correct display order positions.

Now suppose that due to a compressed-domain cut or edit, the last two B frames are not present:

I 2
B 0
B 1
P 5
B 3
B 4
P 8
B 6
B 7
P 11

It looks like this GOP contains 10 pictures. But only pictures 0-8 can be delivered correctly. Although the last P picture can be decoded, it would be incorrect to deliver it out of its correct position in display order. Therefore, an intelligent decoder/muxer will discard P 11. That is a "shortened GOP". If it happens once at the end of the file, just ignore it. If you have it multiple times in the middle of a stream, then you should cut/edit more carefully. If VideoReDo is outputting such GOPS then it is a bug that should be corrected.

Note that this is not limited to MPEG2 video. Sometimes I get a query about why DGDecNV is losing some pictures at the end of an AVC stream. In all such cases that I have looked at, we have a case analogous to this one, but with POC replacing temporal reference.

Your point about GOP headers being optional is not germane. For our purposes when actual GOP headers are not present, we consider "GOPs" to be delimited by the I pictures.

Finally, to answer the question of your first post: You cannot "fix" a GOP that is missing pictures (unless you add the pictures back). mkvmerge is doing the best it can with your input.

[lovelove]

excellent :goodpost: and very helpful. BIG thanks! makes sense to me now.