Chapters timing problem.

Jeffster · 30th April 2006, 17:58

I've been watching/checking a DVD after compressing it with DVD Rebuilder and was struck by the fact that the chapters don't start at the same place as they do on the original. As best as I can figure, all the chapters appear to begin 2 frames too early.

It's readily noticeable on this DVD because many of the chapters begin on a definate scenechange, so when you skip chapters or go to one from the menu, it plays a split second of the previous scene as the chapter begins, making it stand out from the original.

It's very weird and had me wondering as to the cause. I've compared the cell times and durations of the rebuilt DVD and they're all identical to the original, and the trim ranges in the avs files match these also, but if I open and view an avs file for a particular chapter/cell it actually starts at wrong place. It's almost as if the decoder (DGDecode.dll) is feeding the incorrect frames and yet the total number of frames is correct.

Anyway, this has me stumped, and I've searched the forum and haven't seen anyone else mention it, although somehow I don't think this is an isolated case. Maybe someone might have some thoughts on this?

Oh yeah, it's a PAL disc, if that makes any difference (and using DVD-RB 0.97)

Jeffster · 30th April 2006, 19:20

Okay I think I have this figured out now.
At least I have a theory as to what's happening... but it's very, very late here and I need to get some sleep, so will give a follow up in the morning.

jdobbs · 30th April 2006, 21:09

Any preprocessing? No reports of anything like that in a long, long time.

Jeffster · 1st May 2006, 02:47

There was no pre-processing but it is a rather obscure DVD (R5), so I thought I'd test my theory on a readily available title.

So I hauled out my Oliver Twist PAL R2 DVD, and found the exact same thing occurred, although the incorrect chapter start times would never be noticed on this DVD and probably on many others that fit the same criteria.

Like I said, I have a theory as to what's happening and it all boils down to encoded frames and displayed frames.

As you know, DGIndex originally began light because the old DVD2AVI lost frames, and in particular if the first GOP is not closed it thows those frames away but DGIndex keeps them, thus returning the correct total number of frames.

As a result, any inital frames that weren't decodable before are decodable now and therefore in the new re-encoded DVD are also decodable. You could say that the new rebuilt DVD is improved compared to the original but the consequence of this is it throws off the displayed frames by that same number of frames recovered and is why the chapters start at a different place on the rebuilt DVD.

Anyway, I think this is the problem but I may be totally wrong.

Unfotunately on the DVD above which brought this to my attention, it's not only noticeable but annoying.

Some further information which I hope won't confuse things... but with respect to both those DVD's, if I were to rip the movie only in IFO mode with DVDDecrypter it reports an audio delay of -80ms (2 frames in PAL), whereas DGIndex reports an adjusted audio delay of 0ms. This phenomenom will be familiar to anyone doing DivX rips. If I do the same thing again afterwards with the rebuilt disc, DVDDecrypter now reports an audio delay of 0ms. Probably useless information but I believe one way to identify a candidate for the problem.

jdobbs · 1st May 2006, 02:57

I don't think so. I'm not sure what you might be seeing, but I've done a lot of testing to ensure frames are exact matches to the original -- and I haven't found a mismatch in a long, long time (definitely not since DGDECODE was released to replace MPEG2DECODE3. Furthermore, I do a comparison of the SCR/PTS times at the start of each cell while I'm rebuilding to make sure I keep continuity -- and except for a few rare exceptions (under very unique circumstances, e.g. where I might adjust for buffering in the SCR) the values are exactly the same as the original. It couldn't do that with missing or added frames.

Are you using any filters, etc?

Jeffster · 1st May 2006, 03:20

No, I never use any filters.

I agree there are no missing or added frames... I spent hours comparing the rebuilt DVD to the original, and as I said in my first post, all the cells times and durations and frames are identical.

Unfortunately I don't understand SCR/PTS and technical things like that. I still believe it may have something to do with encoded frames vs displayed frames though, and on movies where the first GOP is open this problem will present itself (if anyone wants to check) but I don't know how to explain what I mean any better than this, sorry.

jdobbs · 1st May 2006, 03:35

The fact that a GOP is open or closed really shouldn't make a difference when frame serving -- unless of course you have orphaned frames due to a bad cut. DGDECODE does forward or backward referencing and decodes it -- then is presented as a frame to AVISYNTH.

As for encoded v.s. displayed -- I'm not sure exactly what you mean... but I assume you're talking about the temporal order in the MPEG stream as opposed to the order in which they are physically stored. But that also would be handled by DGDECODE. They are presented to the frame server in temporal order and the fact that the source was MPEG is invisible to the program using the frames that are served...

Jeffster · 1st May 2006, 04:17

Quote:

Originally Posted by jdobbs

As for encoded v.s. displayed -- I'm not sure exactly what you mean... but I assume you're talking about the temporal order in the MPEG stream as opposed to the order in which they are physically stored.

Well, when I was searching for an explanation I read a post by mpucoder which mentioned the word temporal (as well as encoded and displayed frames), so yes I think that's what I mean... but like I said I don't have a great understanding of the underlying technical and structural things that you're talking about, so this is the best I can do. It's entirely possible I'm completely wrong, in which case I have absolutely no explanation whatsoever.

Anyway, all I can do now is post 2 screenshots from Oliver Twist...
one was taken by opening the avs file for Chapter9 and taking a screenshot of the first frame, the second by opening the DVD in PgcEdit and previewing VOBID:1 CELLID:9 and taking a screenshot of the first frame there. As you can see they're different.

Apart from that nothing about this makes sense to me...

Jeffster · 1st May 2006, 04:41

Oh, one last thing, here is the trim range from one of the avs files, chosen at random, for the first DVD that I noticed the problem(R5 one) and a short extract from Parse D2V on the original DVD and rebuilt one, if it means anything to you.

Code:

trim(83307,87257)

Original DVD:

Code:

83306 [P]: 83306,83306..........2
[GOP]
83307 [B]: 83307,83307..........2
83308 [B]: 83308,83308..........2
83309 [I]: 83309,83309..........2
83310 [B]: 83310,83310..........2
83311 [B]: 83311,83311..........2
83312 [P]: 83312,83312..........2
83313 [B]: 83313,83313..........2
83314 [B]: 83314,83314..........2
83315 [P]: 83315,83315..........2
83316 [B]: 83316,83316..........2
83317 [B]: 83317,83317..........2
83318 [P]: 83318,83318..........2
[GOP]
83319 [B]: 83319,83319..........2

Rebuilt DVD:

Code:

83306 [P]: 83306,83306..........2
[GOP]
83307 [I]: 83307,83307..........2
83308 [B]: 83308,83308..........2
83309 [P]: 83309,83309..........2
83310 [P]: 83310,83310..........2
83311 [B]: 83311,83311..........2
83312 [B]: 83312,83312..........2
83313 [P]: 83313,83313..........2
83314 [P]: 83314,83314..........2
83315 [B]: 83315,83315..........2
83316 [P]: 83316,83316..........2
[GOP]
83317 [B]: 83317,83317..........2

By my reckoning, this chapter began at frame 83309 on the original DVD (chapters can only be on keyframes, right?) whereas on the rebuilt DVD it starts at 83307...

mpucoder · 1st May 2006, 17:13

Quote:

Originally Posted by Jeffster

chapters can only be on keyframes, right?

No, they can only be at the beginning of a GOP (which is then multiplexed to be the first GOP of the first VOBU of a cell). If the I picture is not temporal sequence number 0, as in the original, the I picture is not the first displayed frame, a B picture is. The chapter begins at the first displayed frame.
The rebuilt video has a different GOP structure - but both start display at frame 83307.
A lot of preview programs (the one in MuxMan included) will show the I picture when positioned to a GOP as it requires less decoding.

SAPSTAR · 1st May 2006, 17:48

Quote:

Originally Posted by Jeffster

Oh, one last thing, here is the trim range from one of the avs files, chosen at random, for the first DVD that I noticed the problem(R5 one) and a short extract from Parse D2V on the original DVD and rebuilt one, if it means anything to you.

Code:

trim(83307,87257)

Original DVD:

Code:

83306 [P]: 83306,83306..........2
[GOP]
83307 [B]: 83307,83307..........2
83308 [B]: 83308,83308..........2
83309 [I]: 83309,83309..........2
83310 [B]: 83310,83310..........2
83311 [B]: 83311,83311..........2
83312 [P]: 83312,83312..........2
83313 [B]: 83313,83313..........2
83314 [B]: 83314,83314..........2
83315 [P]: 83315,83315..........2
83316 [B]: 83316,83316..........2
83317 [B]: 83317,83317..........2
83318 [P]: 83318,83318..........2
[GOP]
83319 [B]: 83319,83319..........2

Rebuilt DVD:

Code:

83306 [P]: 83306,83306..........2
[GOP]
83307 [I]: 83307,83307..........2
83308 [B]: 83308,83308..........2
83309 [P]: 83309,83309..........2
83310 [P]: 83310,83310..........2
83311 [B]: 83311,83311..........2
83312 [B]: 83312,83312..........2
83313 [P]: 83313,83313..........2
83314 [P]: 83314,83314..........2
83315 [B]: 83315,83315..........2
83316 [P]: 83316,83316..........2
[GOP]
83317 [B]: 83317,83317..........2

By my reckoning, this chapter began at frame 83309 on the original DVD (chapters can only be on keyframes, right?) whereas on the rebuilt DVD it starts at 83307...

Hmmm something seems wrong in your original structure. You CANT have a chapter starting on a B/P frame. It must be an I frame. (BTW a GOP can only start by an I frame).

jdobbs · 1st May 2006, 17:53

Quote:

Hmmm something seems wrong in your original structure. You CANT have a chapter starting on a B/P frame. It must be an I frame. (BTW a GOP can only start by an I frame)

I think his list was showing temporal order rather that stored order. So the B frame is only first temporally -- it is second (behind the I Frame) physically in the stream.

Quote:

A lot of preview programs (the one in MuxMan included) will show the I picture when positioned to a GOP as it requires less decoding

Which explains why he's seeing a picture that is two frames off... but and stream is correct.

Jeffster · 1st May 2006, 19:22

@mpucoder
Thanks for the explanation. So the B frame is the first displayed frame of that chapter, on the original DVD. I must admit I don't know what the temporal sequence number means (and probably don't need to) but can I duduce from what you've said that this B frame would be number 0, since you said the I frame is not number 0?

Anyway, it kinda has me more confused now because I was just reading another thread in which you and others said, if I understand it correctly, it's best to sort display order based on picture type and not temporal order... or have I misunderstood things again?

@jdobbs

Quote:

Quote:A lot of preview programs (the one in MuxMan included) will show the I picture when positioned to a GOP as it requires less decoding

Which explains why he's seeing a picture that is two frames off... but and stream is correct.

Yep it does, and good to know... but it doesn't explain why the chapters start at different places when playing back the new DVD, that's still a mystery to me and may remain one.

mpucoder · 2nd May 2006, 00:56

IIRC that thread explains how to deduce the temporal sequence number by the picture type. And the mpeg standards even have rules for decoding and displaying the pictures that make this number redundant.
A simple rule is, from the encoded order, display all B pictures which follow an I or P before the I or P, all others display in the same order. So IBB displays as BBI, while IPBB displays as IBBP. Either order can be found in closed and open GOPs although only IBB needs the flag to indicate whether cutting at this GOP will orphan the B pictures (ie remove the reference frame which resides in the preceeding GOP)

Jeffster · 2nd May 2006, 02:56

@mpucoder
Thanks for the clarification, I appreciate it.

Quote:

Originally Posted by mpucoder

A simple rule is, from the encoded order, display all B pictures which follow an I or P before the I or P, all others display in the same order. So IBB displays as BBI, while IPBB displays as IBBP.

So in the extract I posted above from ParseD2v for the original DVD, the GOP begins IBB but is actually displayed as BBI... that's the temporal sequence and why you said in your previous post that the B picture is displayed first and not the I picture, at 83307?

So temperal sequence also means display order?

Sorry if I sound retarded by asking for confirmation like this.

Quote:

Either order can be found in closed and open GOPs although only IBB needs the flag to indicate whether cutting at this GOP will orphan the B pictures (ie remove the reference frame which resides in the preceeding GOP)

What is the flag that indicates if cutting at this GOP will orphan the B pictures? Do you mean "closed GOP:0"?

If cutting at a GOP with this flag will orphan the B pictures since the reference frame resides in the preceeding GOP, does it also follow that during random access, like for example skipping to a chapter point, these B pictures won't be able to refer to the reference frame in the preceeding GOP?

If that is correct, then would the next I picture be displayed instead?

I hope the answer is yes, because it's the only thing I can think of that explains this anomaly...

mpucoder · 2nd May 2006, 03:39

Temporal sequence is the display order within a GOP.
Yes, I'm referring to the closed GOP flag. If set it means the GOP does not need the preceeding GOP for a reference frame. So the GOP can be cut without any problem.
I'd say random seeking like that would vary among players, but the easiest thing to do would be start decoding at the I frame, and discard the B's - so probably a lot of players do. (for all I know it could be the spec)

Question is, is the original a closed or open GOP? The new video has frame 83307 as an I frame, so the chapter will surely start there.

I think I see what is happening if the original used an open GOP. While technically the chapter begins at the ptm of the first vobu in it, the display is being delayed until the first I picture because the GOP is open. The original author probably was not even aware of that, though, as all he had to do was tell the encoder to force an I picture at frame 83309, and the authoring app to start a chapter at that same time.

Jeffster · 2nd May 2006, 04:36

Quote:

Originally Posted by mpucoder

Question is, is the original a closed or open GOP? The new video has frame 83307 as an I frame, so the chapter will surely start there.

It is an open GOP... I opened the original VOB in VobEdit (now that I know what I'm looking for) and checked this particular GOP. It has the "closed GOP: 0" flag set.

Well both my standalone players (and also PowerDVD) display the I picture when you skip to the chapter on the original DVD, and it is certainly what the authors would have intended because it is a new scene and if frame 83307 were displayed, as in the new DVD, it's not only 2 frames early but a totally different scene so there is a jolt (to the viewer) as it plays for a split second, before the new scene begins. I don't think the authors would have intended that.

I haven't checked all the other chapters but I imagine they would also be open GOP's because there is the same jolt when skipping to almost every chapter on the new DVD.

Thanks mpucoder!

It looks like I finally know why I'm seeing what I'm seeing (and I think maybe I learnt a little bit in the process too).

mpucoder · 2nd May 2006, 05:04

I think I read somewhere that discarding the first B pictures in an open GOP is the accepted way to begin decoding. It is certainly a lot easier than decoding all the I and P pictures of the preceeding GOP just to form the last reference frame - a lot of work for 2 frames.
Of course to fix it you need to adjust the chapter points as well as the forced I frames.

Jeffster · 2nd May 2006, 06:23

Yep adjusted, burnt and everything is fine now. Thanks!

mpucoder · 4th May 2006, 01:39

This discussion was good for me, too, in that it made me think about this situation. There will be some changes made to MuxMan as a result to account for open gop's and cell/program/chapter placement.

30th April 2006, 17:58	#1 \| Link
Jeffster Registered User Join Date: Jan 2003 Location: Land of the Long White Cloud Posts: 179	Chapters timing problem. I've been watching/checking a DVD after compressing it with DVD Rebuilder and was struck by the fact that the chapters don't start at the same place as they do on the original. As best as I can figure, all the chapters appear to begin 2 frames too early. It's readily noticeable on this DVD because many of the chapters begin on a definate scenechange, so when you skip chapters or go to one from the menu, it plays a split second of the previous scene as the chapter begins, making it stand out from the original. It's very weird and had me wondering as to the cause. I've compared the cell times and durations of the rebuilt DVD and they're all identical to the original, and the trim ranges in the avs files match these also, but if I open and view an avs file for a particular chapter/cell it actually starts at wrong place. It's almost as if the decoder (DGDecode.dll) is feeding the incorrect frames and yet the total number of frames is correct. Anyway, this has me stumped, and I've searched the forum and haven't seen anyone else mention it, although somehow I don't think this is an isolated case. Maybe someone might have some thoughts on this? Oh yeah, it's a PAL disc, if that makes any difference (and using DVD-RB 0.97) Last edited by Jeffster; 30th April 2006 at 18:11.

30th April 2006, 21:09	#3 \| Link
jdobbs Moderator Join Date: Oct 2001 Posts: 12,058	Any preprocessing? No reports of anything like that in a long, long time. __________________ "I wouldn't be so paranoid if everyone wasn't out to get me." (Frank Burns) Help with development of new apps: Donations. Website: www.jdobbs.com

1st May 2006, 02:57	#5 \| Link
jdobbs Moderator Join Date: Oct 2001 Posts: 12,058	I don't think so. I'm not sure what you might be seeing, but I've done a lot of testing to ensure frames are exact matches to the original -- and I haven't found a mismatch in a long, long time (definitely not since DGDECODE was released to replace MPEG2DECODE3. Furthermore, I do a comparison of the SCR/PTS times at the start of each cell while I'm rebuilding to make sure I keep continuity -- and except for a few rare exceptions (under very unique circumstances, e.g. where I might adjust for buffering in the SCR) the values are exactly the same as the original. It couldn't do that with missing or added frames. Are you using any filters, etc? __________________ "I wouldn't be so paranoid if everyone wasn't out to get me." (Frank Burns) Help with development of new apps: Donations. Website: www.jdobbs.com Last edited by jdobbs; 1st May 2006 at 03:04.

1st May 2006, 03:35	#7 \| Link
jdobbs Moderator Join Date: Oct 2001 Posts: 12,058	The fact that a GOP is open or closed really shouldn't make a difference when frame serving -- unless of course you have orphaned frames due to a bad cut. DGDECODE does forward or backward referencing and decodes it -- then is presented as a frame to AVISYNTH. As for encoded v.s. displayed -- I'm not sure exactly what you mean... but I assume you're talking about the temporal order in the MPEG stream as opposed to the order in which they are physically stored. But that also would be handled by DGDECODE. They are presented to the frame server in temporal order and the fact that the source was MPEG is invisible to the program using the frames that are served... __________________ "I wouldn't be so paranoid if everyone wasn't out to get me." (Frank Burns) Help with development of new apps: Donations. Website: www.jdobbs.com

2nd May 2006, 06:23	#19 \| Link
Jeffster Registered User Join Date: Jan 2003 Location: Land of the Long White Cloud Posts: 179	Yep adjusted, burnt and everything is fine now. Thanks! Last edited by Jeffster; 2nd May 2006 at 06:40.

30th April 2006, 19:20	#2 \| Link
Jeffster Registered User Join Date: Jan 2003 Location: Land of the Long White Cloud Posts: 179	Okay I think I have this figured out now. At least I have a theory as to what's happening... but it's very, very late here and I need to get some sleep, so will give a follow up in the morning.

1st May 2006, 02:47	#4 \| Link
Jeffster Registered User Join Date: Jan 2003 Location: Land of the Long White Cloud Posts: 179	There was no pre-processing but it is a rather obscure DVD (R5), so I thought I'd test my theory on a readily available title. So I hauled out my Oliver Twist PAL R2 DVD, and found the exact same thing occurred, although the incorrect chapter start times would never be noticed on this DVD and probably on many others that fit the same criteria. Like I said, I have a theory as to what's happening and it all boils down to encoded frames and displayed frames. As you know, DGIndex originally began light because the old DVD2AVI lost frames, and in particular if the first GOP is not closed it thows those frames away but DGIndex keeps them, thus returning the correct total number of frames. As a result, any inital frames that weren't decodable before are decodable now and therefore in the new re-encoded DVD are also decodable. You could say that the new rebuilt DVD is improved compared to the original but the consequence of this is it throws off the displayed frames by that same number of frames recovered and is why the chapters start at a different place on the rebuilt DVD. Anyway, I think this is the problem but I may be totally wrong. Unfotunately on the DVD above which brought this to my attention, it's not only noticeable but annoying. Some further information which I hope won't confuse things... but with respect to both those DVD's, if I were to rip the movie only in IFO mode with DVDDecrypter it reports an audio delay of -80ms (2 frames in PAL), whereas DGIndex reports an adjusted audio delay of 0ms. This phenomenom will be familiar to anyone doing DivX rips. If I do the same thing again afterwards with the rebuilt disc, DVDDecrypter now reports an audio delay of 0ms. Probably useless information but I believe one way to identify a candidate for the problem.

1st May 2006, 03:20	#6 \| Link
Jeffster Registered User Join Date: Jan 2003 Location: Land of the Long White Cloud Posts: 179	No, I never use any filters. I agree there are no missing or added frames... I spent hours comparing the rebuilt DVD to the original, and as I said in my first post, all the cells times and durations and frames are identical. Unfortunately I don't understand SCR/PTS and technical things like that. I still believe it may have something to do with encoded frames vs displayed frames though, and on movies where the first GOP is open this problem will present itself (if anyone wants to check) but I don't know how to explain what I mean any better than this, sorry.

2nd May 2006, 00:56	#14 \| Link
mpucoder Moderator Join Date: Oct 2001 Posts: 3,405	IIRC that thread explains how to deduce the temporal sequence number by the picture type. And the mpeg standards even have rules for decoding and displaying the pictures that make this number redundant. A simple rule is, from the encoded order, display all B pictures which follow an I or P before the I or P, all others display in the same order. So IBB displays as BBI, while IPBB displays as IBBP. Either order can be found in closed and open GOPs although only IBB needs the flag to indicate whether cutting at this GOP will orphan the B pictures (ie remove the reference frame which resides in the preceeding GOP)

2nd May 2006, 03:39	#16 \| Link
mpucoder Moderator Join Date: Oct 2001 Posts: 3,405	Temporal sequence is the display order within a GOP. Yes, I'm referring to the closed GOP flag. If set it means the GOP does not need the preceeding GOP for a reference frame. So the GOP can be cut without any problem. I'd say random seeking like that would vary among players, but the easiest thing to do would be start decoding at the I frame, and discard the B's - so probably a lot of players do. (for all I know it could be the spec) Question is, is the original a closed or open GOP? The new video has frame 83307 as an I frame, so the chapter will surely start there. I think I see what is happening if the original used an open GOP. While technically the chapter begins at the ptm of the first vobu in it, the display is being delayed until the first I picture because the GOP is open. The original author probably was not even aware of that, though, as all he had to do was tell the encoder to force an I picture at frame 83309, and the authoring app to start a chapter at that same time.

2nd May 2006, 05:04	#18 \| Link
mpucoder Moderator Join Date: Oct 2001 Posts: 3,405	I think I read somewhere that discarding the first B pictures in an open GOP is the accepted way to begin decoding. It is certainly a lot easier than decoding all the I and P pictures of the preceeding GOP just to form the last reference frame - a lot of work for 2 frames. Of course to fix it you need to adjust the chapter points as well as the forced I frames.

4th May 2006, 01:39	#20 \| Link
mpucoder Moderator Join Date: Oct 2001 Posts: 3,405	This discussion was good for me, too, in that it made me think about this situation. There will be some changes made to MuxMan as a result to account for open gop's and cell/program/chapter placement.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode