Splitting cells at open GOPs

[jsoto]

I've been working a little bit on splitting, and I found the following problem:
If the split is done in one GOP which is self-contained or closed, not problem at all.

But if the split is done just in an open GOP, the options are:
a) Mark the GOP as broken
b) Keep the GOP as open
c) Mark the GOP as broken only if the new cell is going to be a new program/chapter

In the case of a), the normal playback of the PGC is affected, and when the player is just in the cell transition, a small jump can be noticed (the settop skips two frames, I suppose..).

In the case of b), I suposse (I cannot see it in a SW player) a jump to the chapter starting with an open GOP will produce two bad frames. Or may be the settop is smart enough to detect the open GOP and skips the two frames...

c) is not a good solution. Playback will be affected, and programs/chapters could be modified in a further step...

So, I think the best option is to leave the GOP as it is (open), and trust the settop is smart enough.

Any comments?

jsoto

[bigotti5]

Every authoringsoft keeps the GOP as it is (open) in case of creating chapters/cells/programs and I have never noticed bad frames...

[jsoto]

I agree...

What about the cuts in open GOPs? I think it is better to mark the cut as a broken GOP, because, in this case, the cell transition is always marked as non-seamless.

What I'm going to do is to completely remove the warnings about open GOPs and to modify the GOP to broken only in cuts.

jsoto

[blutach]

Maybe good to leave a warning in that VobBlanker (http://jsoto.posunplugged.com/vobblanker.htm) is doing that, so users can perhaps search for a nearby closed GOP?

Regards

[jsoto]

I've shown in the dialog the GOP type. So the user will know in advance which kind of GOP is going to be used as first one in a cell.

http://forum.doom9.org/showthread.php?t=120140

jsoto

[r0lZ]

It's exactly what I would have suggested! Thanks Jsoto!

Also, a new option to jump to the first closed GOP to the left or right of the current position would be nice. Maybe we could use the current arrows buttons together with the control or shift key?

[jsoto]

Also, a new option to jump to the first closed GOP to the left or right of the current position would be nice.I've seen many DVDs where the only non-open GOPs are the first ones in the cell. So I think it has no sense to put a function to find them.
Maybe we could use the current arrows buttons together with the control or shift key?
The sector (lba) displayed in "Go to" is the one reflected in Time and Gop type. These fields are updated every time the sector changes. You can move with "prev", "next" and the slide.
Note the original DVD2AVI controls (the ones at the bottom) do not modify the data in "Go to" area
BTW, there is a minor bug in b1. Changing the cell in PGC preview, these data are not updated, and it should.

As a side note, before you ask:
If a GOP is closed, it is easily detected in the first video pack. But if it isn't, it is needed to verify the next frame, in order to decide if the GOP is self-contained or open. It has to be done detecting the end seq of the MPEG video, which is 4 bytes long. If these four bytes are not located in the same pack (say part in a pack and the other part in the next one), VobBlanker (currrently) fails to detect it, and mark the GOP type as unknown. I'm too lazy to modify the code to include this case...

jsoto

[blutach]

I've seen many DVDs where the only non-open GOPs are the first ones in the cell. So I think it has no sense to put a function to find them.Me too - you can search for ages for a closed GOP and only find it at the next cell. Short of re-encoding it yourself, you're stuck with an Open one in most cases.

Regards

[jsoto]

As a side note, before you ask:
If a GOP is closed, it is easily detected in the first video pack. But if it isn't, it is needed to verify the next frame, in order to decide if the GOP is self-contained or open. It has to be done detecting the end seq of the MPEG video, which is 4 bytes long. If these four bytes are not located in the same pack (say part in a pack and the other part in the next one), VobBlanker (currrently) fails to detect it, and mark the GOP type as unknown. I'm too lazy to modify the code to include this case...OK, I've also coded this case (not so difficult, after all). I hope there won't be more unknown GOP types (if the source is not corrupted).
jsoto

[Sir Didymus]

Just speculating...

Do you think there are any benefits in just re-encoding the single first GOP at the new cell boundary ?

What's your opinion (useful ?) about the possibility of creating seamless joined cell splits ?

Apart its usefulness, is it feasible at all ?

[jsoto]

@Sir Didymus
What the hell does you have inside your head? You're always making things as much complex as possible! :D :D

Answering...
joined cell splits are seamless, if you do not modify the GOP flags (as currently VB is doing). What could happen (but I've never seen it in a SW player) is a couple of frames bad decoded in a direct jump to a program/chapter if its first cell starts in an open GOP
So, I'm convinced this is the right option. (and the one used in all other authoring programs..)
Just thinking a little more: What can do the player if it jumps to an open GOP. Obvioulsy it knows it needs the two previous frames, so it only has two options:
- be stupid, and use the two previous frames in memory (from other chapter) so you'll see something wrong in the screen.
- be smart, and knowing the two previous frames in memory are not valid anymore, discard them and skip the frames needing this info, so you won't see nothing wrong in the screen. (well, the chapter will start playing two frames after the selected one).

Feasible?
Yes, it is for sure...
Just write an avisynth script, trim the frames of this particular GOP (better the whole VOBU, sometimes it has two GOPs), and encode it with a bitrate lower enough to fit in the space you have (to avoid a complete remux).
Finally, doing it with care, replace the video in the VOBU, using the same SCR values in the packs

But, forget it, I'm NOT going to code this...still not so crazy :D :D

jsoto

[r0lZ]

OK, I'm committed. :)
You are certainly right, Jsoto. Anyway, you are the specialist in VOBs, not me! Thanks for your answer.

[jsoto]

Just write an avisynth script, trim the frames of this particular GOP (better the whole VOBU, sometimes it has two GOPs), and encode it with a bitrate lower enough to fit in the space you have (to avoid a complete remux).
Finally, doing it with care, replace the video in the VOBU, using the same SCR values in the packs
To be honest, I have to confess I had similar idea in the past (before DVD-RB). Just passing the encoder all the VOBUs timestaps as "chapters". The encoder should produce a video with the GOPs in the same time positions, so the new video could be replaced easily in the VOB...The drawback is you'll loose the advantage of bitrate reduction when using open GOPs.
jsoto

[Sir Didymus]

@Sir Didymus
What the hell does you have inside your head? You're always making things as much complex as possible! :D :D
...
But, forget it, I'm NOT going to code this...still not so crazy :D :D

jsoto

:p
Good to know...
Everyone here is confident you avoid doing crazy things with VobBlanker, leaving to fool people like the writer the job of flying with the fantasy... :rolleyes:


So, I'm convinced this is the right option. (and the one used in all other authoring programs..)


Me too...
I mean, I am fully convinced it is the right option!

Please do not take my comment as something different than what I stated in the premise - so let me say again... I was just speculating...

What I wanted to point out is that this option is the one used by all authoring programs when the GOP is open.

If the GOP is closed there is no need to rely on the behaviour of the player, since no frames will be skipped in any case...

I mean, do you know what encoders do when including some chapter list as option ?

I am not sure (never checked) but I think, other than placing an I frame at the chapter point, they set the starting GOP of the new chapter as "closed"... Right ?

Edit: OOPS... Posted before reading your last message... Your clarification is OK!
Cheers,
SD

[blutach]

Just a question - anyone know why open GOPs were invented in the first place? I suppose it more efficient to encode this way. However, they do seem to be a bit troublesome.

Regards

[jsoto]

Just a question - anyone know why open GOPs were invented in the first place? I suppose it more efficient to encode this way. However, they do seem to be a bit troublesome.

No idea... but I'm not sure they increase the efficiency too much... I'm going to test it, just curious...

What for sure increase the efficiency is to have large GOPs. IIRC, in DVD-standard the GOPs are limited to 15 (PAL) /18 (NTSC) frames (36/30 fields), which means 0.6 sec. That means one large I-frame each 15/18 frames. I've seen in some avi encodes up to 300 (even more) frames per I frame. This is toooo much, but may be 15 frames is too short.

The advantage of having short GOPS is clear. You can jump to whatever time and you’ll find very close an I-frame, so you can start the decoding very close to the selected time.

Following could be OT, but IMHO it is interesting…
Short GOPs are very important in IPTV distribution. In this case, simplifying the scenario, the access node is receiving all the TV channels in multicast streams, but a single user has a limited bandwidth in his DSL line. When the user “zaps” from one channel to other one, what he is doing is to leave a multicast channel and join another one. “Leave” means the access node has to stop sending the channel towards the DSL line and join means it has to start sending the new one . There are some tricks to leave the multicast channel quickly in the access node, but what is clear is that the user is joining a multicast channel in a random way, so the first bit of the GOP received is not controlled. The user’s settop has to wait the first I-frame to start the decoding process. As a consequence, using large GOPs means a very slow channel zapping, producing an undesirable effect from the user point of view.
Finally, there is also a trick (owned by MicroSoft, BTW) to increase the zapping speed, using unicast streams during a while, but this overloads the DSL line during this time and needs more powerful video servers as well as more bandwidth in the core network. Also, the channel can be delayed too much (so the user can hear !!GOAL!! in a football match a couple of seconds or more before seeing it in the TV, because other people is listening to the radio, or watching a different, may be no-IPTV, TV channel).

jsoto

[jsoto]

I'm not sure they increase the efficiency too much... I'm going to test it, just curious...
Here are the results of my tests.
I've created two clips (scenes from two different DVD originals) and passed them to CCE, with exactly the same settings (the normal ones when encoding DVDs) and changing the number of pictures in the GOP (between 12 and 15) and the GOP types (closed/open). I've used constant quantization factor Q (which means, in theory, the same quality) and I've recorded the file sizes.
http://download.videohelp.com/jsoto/images/GOP_size_differences.jpg
As you can see, the encoder does take advantage from the open GOPs, even more than from increasing the pictures from 12 to 15. This advantage is more noticeable when the encoder has to work harder, compressing more (higher Qs).
The results are quite similar in both scenes so we can conclude that to use open GOPs is a good practice, mainly when re-encoding a DVD-9 to fit in a DVD-5

jsoto

[BigCondor]

Thanks, so that's why open GOPs are used.
The thread itself is very educational.

[mpucoder]

Just a question - anyone know why open GOPs were invented in the first place? I suppose it more efficient to encode this way. However, they do seem to be a bit troublesome.

Regards
Actually it was the closed GOP that got invented. MPEG was designed to be a streaming media with no beginning or end, but then someone asked "how do we switch streams or edit stored content without glitches" and along came the closed GOP to allow cuts and edits, and also the broken GOP to inform decoders of a cut made at a point where it was not planned in advance (ie on an open GOP)
DVD players are designed when starting a chapter (or any other seek/skip/trick-play operation) to discard orphaned B pictures

[Susana]

Sorry for el off-topic, but I want to take advantage of your presence in this topic.

I'd like to know if there is a way to know if a DVD was transcoded (using any of the one click solutions) or reencoded (with dvdrebuilder for example), looking at the mpeg stream inside vobs. So, is there any appreciable difference between them, at bit level?.
I'd like to make a program for this.

Thanks.

[blutach]

Thanks jsoto and mpucoder for that.

Regards

[jsoto]

I'd like to know if there is a way to know if a DVD was transcoded (using any of the one click solutions) or reencoded (with dvdrebuilder for example), looking at the mpeg stream inside vobs. I suppose the question is directed to mpucoder, as he is the MPEG expert. But I think I can give you a trick to detect shrink, w/o looking into the MPEG video

Usually the SCR of the packs follows the sequence:
{
scr+=146;
scr_ext+=86;
if (scr_ext > 299) {scr++; scr_ext-=300;}
}

Shrink, keeps untouched the SCR of all non-video packs, so it has to "distribute" the compressed video along the file using less packs than the original.
Many times one VOBU starts with the NAV followed by a lot video packs (the second pack in the VOBU must be video, but the third can be any kind of pack). Well, in these VOBUs, a "normal" muxer (including DVD-RB) follow the sequence of 146.086 in scr, but shrink breaks the scr sequence just between the first video pack and the second video pack. It is a "dvdshrink signature"

jsoto

EDIT:
BTW, DVDRemake modifies the PROVIDER_ID in the IFO (although it could be changed after..)

[Susana]

Thank you very much, I'll have a look at it, gracias.

[Sir Didymus]

DVD players are designed when starting a chapter (or any other seek/skip/trick-play operation) to discard orphaned B pictures


Wow... Nice to know... It seems the most natural way to handle the situation, but never been sure if this discarding was accepted (or even described) as a standard behaviour...


...
It is a "dvdshrink signature"

Double Wow... That's very interesting... It's always possible to extract the elementary assets from the shrinked output and rehautor the whole in order to produce a clean mux, but nobody does this additional unuseful step after shrinking... So, very nice to know...

I totally agree with BigCondor: the thread is very educational.

[jsoto]

It's always possible to extract the elementary assets from the shrinked output and rehautor the whole in order to produce a clean mux, I'm not saying shrink output is not OK. It is just curious, but IMHO not a bad authoring/muxing at all.

jsoto

[Sir Didymus]

Of course, of course...

I used the term "clean", but it was not my intention to designate as "dirty" the mux produced by Shrink. Maybe the term "continuous" would have been more appropriated than "clean"... I just intended to point out that this "Shrink" signature can be easily removed, if this is what the user want...

I am not sure about the implications of creating packs spaced more than 146 in the SCRs - or skipping some packs - in the output mux. AFAIK it is totally compliant. Also, DTS and PTS of the individual pictures in the video stream are left unchanged - I think, so also the VBV model should be used in the same way as it is in the original: less data is entering in the buffer, but also less data is exiting, and the timing relationship between this in and out events is the same...

Edit: not sure about; maybe just the efficiency of usage of the buffer is reduced, but - in any case - this should have no drawbacks, IMHO...

[jsoto]

146 is the minimum space between pack, and it is related with the maximum bitrate supported by the DVD drive as specs (1x). Packs have to be spaced more than 146 in some points of the VOB, unless the video and audio were recorded at the maximum bitrate all the time.
But the algorithm to decide when the SCR sequence is broken and the packs are spaced more is something in the internal logic of the muxer, and should be related with the buffer efficiency, as you say.
We are running OT...this thead was about open GOPs.

jsoto