How to get a single I-Frame??

[jeanl]

Guys,
I have a technical question for you. I'm looking into ways to "shrink" a motion menu cell into a still cell, and ideally I'd like to only keep the first GOP (ok maybe not the first one) and shrink it down into a single I-Frame (by keeping the first I-frame for example and dumping all the subequent B/P frames).

I've done tests with this idea (using vobblanker to replace the original menu cell with one that I extract as described above), and I'm finding that things work find if I extract a full GOP (tested on a standalone), but not if I only keep the first I-frame (the menu frame only partially displays, but the buttons highlights are still there, and working, it's quite a treat! ;) ).

So, does anyone know what I'm probably missing?
- Is it possible at all without re-encoding?
- Do I need to adjust the GOP structure somewhere in the VOB since I'm dumping many B/P frames? (didn't see where)
- In my tests, I kept the I-frame, but the end of it also had the beginning of a B frame, which I didn't remove (in other words, part of the last LBA also has the beginning of a B frame). That HAS to be bad, could that be the cause of my problem? Should I simply pad the LBA?


Thanks for any info.
Jeanl

[mpucoder]

Well, I've done it (to test Muxman) using a demuxed m2v and a hex editor. If it's not the first GOP remove everything before the sequence header (00 00 01 b3). Then remove everything from the second picture header (00 00 01 00) on. The last thing I had to do (to satisfy the verifier and caused no problem with playback) was change the temporal sequence number to 0 (it was from a closed gop).

My guess would be the decoder doesn't like a partial B frame - it will start to decode it. Replace the picture header and following data with a padding packet.

[jeanl]

Thanks a bunch mpucoder, I was hoping you'd be reading this!!!
Great, I know what to do next! ;)
Jeanl

[jsoto]

Mmmm, very, very useful info. Thanks to both of you.

jsoto

[jeanl]

Well, thanks to the hints from mpucoder, I was able to do exactly what I wanted: create a new cell from an existing one, keeping only the first I-frame and the subpics (that's to replace a motion menu by a still one). The "trick" was indeed to get rid of the beginning of the B/P frame in the last LBA of the I-frame (replace it with a padding packet).

The cell can then replace the original one, (vobblanker can do that easily) and the result seems to play fine on my standalone DVD player.

Success!!! :D

Thanks mpucoder for the hint. I'm finishing a small .exe that does the whole thing automatically (well, just the cell extraction, not the replacing which has to be done with vobblanker). I'll keep you guys posted.

@jsoto: vobblanker seem to have a NavPack pointer problem when cell replacing is used. I'll post about that in your thread.

Jeanl

[jeanl]

OK, more info, more problems!!!
- With lots of help from blutach, I found that it was not good to reset the highlight start time to 0, when extracting a still GOP from a motion menu. Apparently the highlight start time has to match the value in the subpic packs, or the buttons are not displayed.
- Now blutach also encountered a very strange case: He's got a DVD with a VOB cell that starts with a non-closed GOP! I didn't think that was possible!
mpucoder, is this bad authoring, or is it legit: the first GOP at the start of a cell that's non-closed.

Jeanl

[mpucoder]

legit and common. Someone started an urban legend that all GOPs in DVD should be closed, but it's not true (except in multi-angle).

Just because it is not closed does not mean that it requires a prior reference frame. The encoding sequence of IPBBPBBPBB is self-contained, but not closed.

[jsoto]

So, how can you know if a GOP needs a prior reference frame?

jsoto

[jeanl]

This one would for example, right:

BPIPBBPBBP

(is it legit?). The B frame refers to the last I-frame of the previous GOP. Right?
Jeanl

[jsoto]

:eek: Can a GOP start in a P or B frame instead of an I one?

[jeanl]

Originally posted by mpucoder
legit and common. Someone started an urban legend that all GOPs in DVD should be closed, but it's not true (except in multi-angle).

Just because it is not closed does not mean that it requires a prior reference frame. The encoding sequence of IPBBPBBPBB is self-contained, but not closed.
mpucoder, I'm a bit confused!
- Is it true that a non-closed GOP is one that needs a reference from another GOP (either forward or backward)?
- Why are you saying that IPBBPBBPBB is "self-contained"? It still needs the next GOP to be decoded right (because of the P frames EDIT: sorry B frames!) but it does not need the previous GOP, right?

Jeanl

[mpucoder]

Originally posted by jeanl
mpucoder, I'm a bit confused!
- Is it true that a non-closed GOP is one that needs a reference from another GOP (either forward or backward)?
no, it may use a reference frame from the previous GOP, which would look like:
IBBPBBPBB
Or it may be self-contained, which looks like:
IPBBPBBPBB
The key is how many reference (I or P) frames were encoded prior to the B. In the first case only one, so the B may or may not require a frame from the previous GOP. That is what the closed-GOP flag tells us, closed means the first B frames do not require 2 reference frames.

- Why are you saying that IPBBPBBPBB is "self-contained"? It still needs the next GOP to be decoded right (because of the P frames EDIT: sorry B frames!) but it does not need the previous GOP, right?
Jeanl
A GOP never needs the following GOP. Maybe you are getting display order and encoding order mixed up. We are talking about encoding order, the way it appears in the file. The reference frames are always encoded before the B frames (likewise the P frames rely only on the previously encoded I frame).
Also, a GOP must start with an I frame as the first encoded frame. That does not mean it is displayed first.

[jeanl]

mpucoder, thanks a bunch for taking the time to straighten me up, I was seriously wrong!!! And indeed, I was mixing up encoding order and display order (indicated by the temporal sequence number I guess).

If I'm understanding you correctly, you can't always tell whether a GOP is closed or open from just looking at the sequence of IBP frames. That's why we have the flag, to tell us if the B frames rely on a previously encoded reference frame.

OK, thanks again. :)
Jeanl

[jsoto]

Thanks also from my side. I was also wrong in some concepts.

jsoto

[mpucoder]

jean, that's true when there is only one reference frame in the GOP preceeding the first B frame. Otherwise the flag should not be set.

Then there's the broken flag...
It means a "normal" GOP, ie IBBPBBPBB which was not closed (so therefore relied on the previous GOP) has been orphaned by editing (previous GOP removed). This can be avoided / corrected by removing the first group of B frames and adjusting the temporal sequence numbers of the remaining frames downward by the number of B frames removed. Now you will see a "self-contained" GOP emerge - IPBBPBB
Broken and closed should never be set at the same time, if the GOP was closed you can not break it.

[jsoto]

Hey, I didn't ask about broken GOP flag but I really was wondering about its meaning...( mpucoder, do you also have a crystal ball? ).

One question more... If the broken flag is properly set (when cutting the VOB in a non-closed GOP)... Does the decoder (the player) have to honor the flag and skip the B frames?.

jsoto

[jeanl]

Originally posted by mpucoder
jean, that's true when there is only one reference frame in the GOP preceeding the first B frame. Otherwise the flag should not be set.

AH! OK. That's because a B frame can reference a reference frame up to 2 frames before? If there are 2 ref frames (IP for example) before the B, then we know that it's closed because the B frame cannot reference 3 frames back. If there's only 1 I/P frame, then it may or may not be closed. I'm paraphrasing you to make sure I understand. Am I right?


Then there's the broken flag...
It means a "normal" GOP, ie IBBPBBPBB which was not closed (so therefore relied on the previous GOP) has been orphaned by editing (previous GOP removed). This can be avoided / corrected by removing the first group of B frames and adjusting the temporal sequence numbers of the remaining frames downward by the number of B frames removed. Now you will see a "self-contained" GOP emerge - IPBBPBB
Broken and closed should never be set at the same time, if the GOP was closed you can not break it.

OK I understand that now. I had no idea what the broken flag meant!!! Your explanation is very clear.

Again, a million thanks for taking the time to enlighten us. There's probably a place where all this is fully explained (your site perhaps?), but so far I haven't been able to find it!

Jeanl

[mpucoder]

They should not try to decode the frames, but repeat the previous successfully decoded frame. But some decoders have clever ways of masking the errors.

And don't feel bad about being confused, I didn't get it at first, and the author of Scenarist didn't get it either. This is why Scenarist, and almost all authoring programs since, insist on closed GOP for multi-angle. Because the user can switch between angles they thought only closed GOP would do that without problems. But a self-contained GOP will do just as well.

[jeanl]

mpucoder,
I'm still confused!!!
Why do you say that IPBBPBBPBB is self-contained, but not closed?

The first 2 frames are reference frames, so why is it not closed? I does not depend on the previous GOP (as you said, it's self-contained), but why is it not closed????


Jean

[mpucoder]

Just semantics - closed means the closed flag is set and there is only the I frame before the first B. Self-contained means there is an I and P before the first B. Neither one depends on the previous GOP.

[jsoto]

"Self-contained means there is an I and P before the first B"
Err, a self-contained does have the closed GOP flag set or not?

jsoto

[mpucoder]

No, it does not. That flag is only to be used for IBB... to make it clear that the first B frames use no forward prediction (ie need only one reference frame).

[jeanl]

mpucoder, then you can have a frame like this
IPBBPBBPBB
and its closed flag won't be set! The closed flag is only used if a B frame is in second position, then.
So closed flag does not indicate the GOP is self contained.
If it's set, then it certainly is self-contained.
But you can be self-contained with the flag set to 0 as in:
IPBBPBBPBB

Am I understanding you right?
Jeanl

[mpucoder]

You got it!

[jsoto]

I'm going to ask still more...

How can you know where the I-frame ends?. You can trust in Vobu first reference frame end block, but how to find the byte number?. And, if you do not want to trust in vobu first reference end block?

jsoto

[mpucoder]

You look for the next picture header - 00 00 01 00. It will be byte aligned. If you are parsing yourself (not trusting the pointer) make sure you are inside the video stream - ie do not look within the pack and pes header, and make sure it is a video pack.

[jsoto]

So, picture header - 00 00 01 00, cannot be simulated in any case by picture data?

Many thanks again for all the info/clarifications.

jsoto

[jeanl]

Originally posted by mpucoder
You look for the next picture header - 00 00 01 00. It will be byte aligned. If you are parsing yourself (not trusting the pointer) make sure you are inside the video stream - ie do not look within the pack and pes header, and make sure it is a video pack.
jsoto, that's exactly what I do in vobcut: you have to search the full mpeg frame (if it's a video pack) for the pic header. If you find one, that indicates the end of your I-frame. The header is only byte aligned, so you have to look for it at every byte (or do a "smart search").
If you're only going to keep the I-frame, then you have to replace the rest with a padding pack (email me if you want the details for that).
Jeanl

[jsoto]

Thanks, probably I'm going to trust the pointer...

Well, I've to logout until Monday...

See you,
jsoto

[jsoto]

Eeeps, a last question...
Can the picture header be between two different packs?, let say 00 00 in one pack and 01 00 in the next one?

jsoto

[jeanl]

man I didn't think of that!!! That's wicked! I think it can, because otherwise there would be a need to pack the previous I-frame!!!! What the heck! I'm not too sure though! I certainly don't check for that.
Jeanl

[mpucoder]

Yes, it can - welcome to mpeg.
And it can end without enough room for a filler stream (in which case 0 fill is OK, not preferred, but it will work)

[mpucoder]

A suggestion - a form of state processing. Use one function to scan the video payload itself and return when it reaches the end or a full header. Return value = number of header bytes that matched. Use an outer function to handle the program stream, calling the video data scan function when in video data.

[blutach]

Gents,

I have been reading this thread with plent of interest.

Re the split header. Surely, in the first GOP, the I frame can hardly have a header spanning 2 LBAs. That doesn't seem sensible.

I mean, the only thing before it is a navpack. That ends and the I frame of a GOP starts. So where's the chance for the header to span 2 sectors?

Regards

[jeanl]

Blutach, it's not the I-frame header, it's the header of the next video frame (i.e. a B or a P frame). That's smack in the middle of the video payload, and it can traddle a sector...
Jeanl

[mpucoder]

They are looking for the end of the I frame, ie second picture header.

[jeanl]

mpucoder, while you're still around :) (sorry for imposing!)

I"m having problem with the following thing:
- I'm keeping a single I frame (or a full self-contained GOP) extracted from a motion menu, along with all the audio packs (and the nav packs). Again, the idea being to turn a motion menu into a still menu (possibly with audio).

I would expect the resulting VOB to show the pic, and play the audio.
Instead, in PowerDVD, it shows a black frame, plays the audio, AND AT THE END displays the I-frame for a brief momen (I'm talking about playing the VOB, as in drag/drop into PowerDVD).

Any idea what could be wrong? I can do the same thing with a "real" still menu with audio (from a DVD) and PowerDVD shows the image first while it plays the audio. I'm having trouble pinpointing the difference...

If you can think of something obvious right off the top of your head, that would help tremendously.

A million thanks in advance! I promise I won't ask anything after that!
Jeanl

EDIT: I think I've got it: The PTS of the video does not match the PTS of the first audio pack, whereas it does in the "good" still frame with audio... Could that be it?

[mpucoder]

You need an end_of_sequence header (00 00 01 b7) after the single frame. You also need to set vobu_se_e_ptm to vobu_s_ptm + frame_duration.

[jeanl]

Thanks a bunch man! More stuff to look into! :)
Jeanl

[Sir Didymus]

Originally posted by mpucoder
Yes, it can - welcome to mpeg.
And it can end without enough room for a filler stream (in which case 0 fill is OK, not preferred, but it will work)

It can and it happens...
Observed (just few times on three or four titles, but it happens). Viewed while debugging a little mpeg parser written for testing purposes...

By the way want to propose this thread for the "best thread of the month" prize... :)

Cheers,
SD

[mpucoder]

I came across one pack that contained just one byte of the video stream. Made me wonder if the transcoders take advantage of the chunkiness. Here was a case where shortening the video by only one byte would save 2048 bytes overall.

[Sir Didymus]

Observed something similar... (packets smaller than 6 bytes)...
Thinking at the way the packetization process of a mpeg stream, its clear it may happen...

What I've NEVER seen is the presence of 00 00 01 00 inside an mpeg stream...

Sometimes I wonder what happens in case of purposely patching with these values an m2v file, and remuxing it...

By the way, just for the sake of curiousity, do you actually check this type of events into MuxMan ?

SD

[blutach]

Originally posted by jeanl
Blutach, it's not the I-frame header, it's the header of the next video frame (i.e. a B or a P frame). That's smack in the middle of the video payload, and it can traddle a sector...
Jeanl I understand now. But, as you have seen with the cut we have been doing, the cut to full GOP option works fine, so if there is a header issue, it's not affecting this method - only the single I frame one.

@mpucoder - Looking at this particular VOBU that jean is working with, it's stored as I BBP BBP ... I thought the storage order was usually I PBB PBB PBB ..., with the decoder then displaying the GOP as I BBP BBP. Is this normal that in this cas storage and display order are the same?

Regards

[mpucoder]

If the standard is followed to the letter it's not possible to do what they call "start code emulation" Everything is designed to prevent there from being 23 or more zero bits in a row (the trigger for a start code) - this is why marker bits exist. BUT simple mistakes, like padding a pes header with 0 bytes instead of 0xff, can cause problems.

I'm not sure which events you mean, but Muxman does need information from every sequence, sequence_extension, sequence_display_extension, picture, picture_coding_extension, and sequence_end header. And you know what damage a false interpretation can do :)

[mpucoder]

Originally posted by blutach
Looking at this particular VOBU that jean is working with, it's stored as I BBP BBP ... I thought the storage order was usually I PBB PBB PBB ..., with the decoder then displaying the GOP as I BBP BBP. Is this normal that in this cas storage and display order are the same?
Regards
If it's encoded as I BBP BBP .. then it displays as BBI BBP ... (regardless of what the tsn may say!)
Here's the rule:
You decode each frame in the encoded order, paced by the display.
If you just decoded a reference frame (I or P) put it aside and display the previous reference frame (except when starting up, in which case display nothing yet)
If you just decoded a B frame display it immediately.

[jeanl]

mpucoder, you RULE! (but that's no news!).
I added the end-of-sequence header, and that fixed it!
2 Mega-thanks.
Jeanl

[blutach]

@jeanl

You got a new version for me to test. It's been a couple of hours buddy!!!!! :D

:rofl:

[Dimad]

Originally posted by Sir Didymus
By the way want to propose this thread for the "best thread of the month" prize... :)

:) Indeed, very interesting thread.

[jeanl]

OK, I've made great progress thanks to mpucoder's hints.
I still have 1 problem, which I can't seem to find the solution too, so I'm back with a question!

- I'm now trying to turn a motion menu cell into a single-I-Frame cell with audio. In other words, I'm keeping all audio and subpic packs, compressing the first GOP into the first single I-Frame, and dropping all other GOPs in the cell.

- I've modified the first NavPack, and the I-frame pack in the following way:

In the first NavPack, the second and third references frame end blocks are set to 0 (since there's no P frame to follow).
In the first NavPack VOBU_SRI, I set the Next VOBU with video to 0xbfffffff (since there's no more video in the cell).
In the I-frame GOP flag, I set the Closed GOP to 1, and broken GOP to 0.
In the I-frame picture header, I set the temporal sequence number to 0.


After these modifications, I put back the cell in the VOB, adjust the IFO (mock strip), and test.
Things work quite well in lots of cases, BUT I have this one menu where the I-frame won't show only on my standalone (it plays fine in PowerDVD, WindDVD and IFOEdit Play DVD). On my standalone I can hear the audio, see the button highlights, but the I-frame isn't displayed.

I'm wondering what it could be that I'm missing.

- Could it be a presentation time problem? I'm not adjusting the PTS of the I-frame, could that be a problem?
- Could it be related to the fact that in the I-Frame's extension start code, the Top Field First is 0 (and the progressive frame flag is 0)???

Any pointer would be greatly appreciated.

Thanks a bunch in advance!

Jeanl

[mpucoder]

The tff should be 1 for the first frame in a sequence, that could be a problem.

pts should be the same as vobu_s_ptm (and as a double check, one frame later than dts).

vobu_se_e_ptm should be one frame duration later than vobu_s_ptm. (and there should be an end_of_sequence).

vobu_e_ptm should remain unchanged and match the next NAV pack vobu_s_ptm.

SRI won't mess up normal play, but if you want to get strict for all of the pointers not set to bfffffff the top bit should be cleared (it indicates video present in the referenced vobu)

frame duration: NTSC 3003, PAL 3600

single frame GOP is not closed, that flag is only for 2 or more frames, and second is B.

And for other things neither one of us thought of, you could demux after modifying then multiplex with Muxman and compare.