PDA

View Full Version : Dropouts. jdobbs: don't bother reading this for now...


Sir Didymus
21st October 2004, 17:36
First of all sorry for the total lack of synthesis. Simply close this window when you are tired of reading... :)
You will lose nothing really relevant, believe me...

Ok. The reason for posting is to share some preliminar findings on the matter, or better, on subjects that may have some relationship with the matter, with people that could be interested on it.

I am asking Jdobbs not to bother with it just because nothing new is included, at least for the moment, on my side, on the subject.

Since the only material available to me which is by sure affected by the problem has been kindly provided by Fritzdis, I'd better stating clearly that the the rest of this post is biased by the fact that the origin of this material is synthetic, and that some tuning (using RB-OPT) has been applied in order to "simulate" the behaviour of DVD-RB on original titles.

I have a copy of "Paycheck", PAL, R2, which is one of the titles reported (by DvdManiac) as suffering from dropouts, but my release is Italian, so I can not say by sure it actually have drops or not. If DvdManiac or somebody else will find the way of making downloadable a cell (original and rebuilt) of some movie which is having BY SURE dropouts in it, I will be happy of doing the same simulation on this material...

I have also to say that the included pictures are showing systematic overflow conditions; in normal titles (including the release I have of "Paycheck") these conditions are still present, even though they are much less frequent...

The first figure (http://img96.exs.cx/img96/7972/Original.gif) is showing the STD buffer occupancy of the video stream contained in the original material. The graphics has been produced with Excel, and the data have been acquired using the Philips DVD-Video Verifier. The X axis is the time scale (in microseconds, so 45.000.000 means the first 45 seconds of the video stream...). The material has been produced using Scenarist, and the video stram is encoded at constant bitrate. See this thread for further information about the Fritzdis tests and the generation of the original:
http://forum.doom9.org/showthread.php?s=&threadid=82485&perpage=20&highlight=fritzdis%20test&pagenumber=1

The size of buffer is 237568 bytes. As it is possible to see, in no parts the real time simulation the buffer underflows or overflows.

The second picture (http://img96.exs.cx/img96/8012/DVD-RB065Reduct085.gif) has been obtained using DVD-RB 065, all standard options, three click mode, using RB-OPT for applying the 85% of reduction to the size of the original (which is already smaller than a DVD5). Let's call this material "DVD-RB-85%". As it could be seen, since the encoding has been carried out with a variable bitrate, when the original video contains black parts [0...5s.], [30...35s.], [40...50s.] the buffer occupancy is very low, but in any case no underflow conditions are present. Overflows are present in all the rest of the video, which is basically composed by different framed patterns of white noise...

The third picture (http://img96.exs.cx/img96/3131/DVD-RB065MaxBR5000.gif) is showing the result of using DVD-RB 065, all standard options, three click mode, using RB-OPT for encoding the cell with an average bitrate of 4200 Kbit/s and a maximum bitrate of only 5000 Kbit/s. The reason for doing the test is to show that a low bitrate, in itself, is not guaranteeing to solve the problem (I mean to solve the problem of the STD buffer overflows). It is possible to see that the situation is improved a lot (no more overflows exceeding 300.000 bytes), but are still present sistematic conditions of overflows exceeding 250.000 bytes.

The fourth picture (http://img96.exs.cx/img96/6087/IfoEdit.gif) is related to a test based on extracting the m2v video file and the ac2 audio from "DVD-RB-85%" and using Ifoedit fo re-authoring a new title. It's possible to see that some "special" care should be present into the code of Ifoedit for preventing underflows and overflows. Despite of this, frequent overflow situations are present, a little bit over 250.000 bytes...

The fifth picture (http://img91.exs.cx/img91/9996/Mplex.gif) is obtained using mplex for remuxing the m2v and the ac3 streams. The result is very similar to the one of Ifoedit, even though Ifoedit is a little bit better, since the overflow speaks are less deep...

The sixt picture (http://img96.exs.cx/img96/1597/DVDLab13.gif) is obtained using DVD-Lab 1.3 for re-authoring the m2v and the ac3 streams. Here the situation is "quite" good: just four overflow have been detected...

The seventh picture (http://img27.exs.cx/img27/1084/DVDLabProB16.gif) has been produced with DVD-Lab Pro (beta 16 release). Finally the authored stream is fully compliant. No overflows, no underflows. Only drawback is that the average usage of the STD buffer is below 150.000. Nothing to worry about, but keeping this value higher, helps in preventing underflows...

The eight picture (http://img96.exs.cx/img96/911/Scenarist30.gif) has been produced using Scenarist 3.0 for re-authoring the video and audio assets. Other than being fully compliant, the usage of the STD buffer is very very good...

Ok. That's all. Hope not being censored due to the very long post...
And sorry if you feel exausted after reading all this stuff...
Maybe you may understand how I feel after doing so much testings...
:scared:

Cheers,
SD

robot1
21st October 2004, 22:02
Originally posted by Sir Didymus

And sorry if you feel exausted after reading all this stuff...
Maybe you may understand how I feel after doing so much testings...
:scared:

Cheers,
SD Great researching works, as always, SD.
This could be the culprit of the few last rebuilder's issues... but as you can see, no one of the free muxer (and neither dvdlab...) can mux with a perfect standard adherence.
I hope jdobbs has the resources to check this.
Anyway, check your PM for further analysis, if you're not too tired!

Fr4nz
23rd October 2004, 08:57
Keep this post up! It's very interesting.

Trahald
23rd October 2004, 20:43
Good work, Sir Didymus

Sir Didymus
26th October 2004, 18:26
Sorry again for the long post...

Well, even though this exercise is just showing "effects" and not "causes" (I mean, no information or suggestion is given for understanding WHAT is causing these overflows), some curious guys asked, via PM, to know what is the behaviour using a transcoder (Shrink or Rejig) in the place of CCE, and some others asked to check with other Authoring tools (e.g. Maestro).

Regarding this last point, I have to say (apart that I don't hold Maestro), that I am afraid I have been misunderstood, since it was no my intention to make any comparison among authoring packages. I just want to carry out some testings with DVD-RB...

Regarding the transcoders, it should be stated that the function of a transcoder is easier, in general, due to the GOP structure is left untouched by them, so they have some advantages in working on the timings for the production of the reauthored stream.

I did two simple (and different...) more tests, with Rejig and Shrink. Rejig has the capability of reauthoring from m2v and ac3 assets, so this picture (http://img9.exs.cx/img9/9418/Rejig.gif) shows its behaviour for reauthoring a new title using the m2v video file and the ac2 audio extracted from "DVD-RB-85%". In this test Rejig is just used as a further reauthoring tool... Results are very very similar to the ones obtained using IfoEdit or Mplex for carrying out the same function.

On the other hand, here is a simulation using Shrink, starting from the original cell (that was encoded at CBR) and applying the 80.1% of reduction, in order to produce rougly the same file size of "DVD-RB-85%". This picture (http://img89.exs.cx/img89/2522/ShrinkReduct801.gif) is showing the obtained result. It is possible to see that the diagram is almost perfect (concerning the overflows, at least VISUALLY). Infact on many single points, looking at the values evidenced some overflow situations. These anomalies are in all cases of few hundred bytes (very limited in size), and concentrated on single time instants (very limited in time). I did not remind precisely, but it seems to me it was reported that shrinked material played almost well. That is not surprising...

This last picture (http://img87.exs.cx/img87/5705/KillBill2-Umainthetomb.gif) is quite interesting. It has been obtained applying DVD-RB to an original title (Kill Bill2, the nice scene when Uma exits from the tomb...).

I am actually using this cell for doing some further tests, where it seems to me something strange is happening in the DVD-RD timings, so I'd like to pose the following questions:

Let's assume, for a given title cell, to consider this cell both in the DVD-Rebuilded VOB file and in another VOB file, produced using the same m2v, audio and subtitles, extracted from the rebuild and authored by means of Scenarist or another HQ authoring package.

Q1: since they are both based on the same m2v asset, they should have exactely the same GOP structure, right ?

Q2: the presentation time stamp (PTS) is encoding the time a given frame is transferred to the display for presentation, using the system clock reference (SCR) as a local time basis. It means (neglecting the speed tolerances of the two asynchronous clocks) that for each I frame in the two cells the difference PTS - SCR/300 should be almost the same. Or in other terms, if the SCRs in the two cells start with the same value, the PTS values in the two cells should be the same for every I frame. Right ?

Q3: the decoding time stamp (DTS) is encoding the time a given frame should be decoded for handling in the correct way the frame decoding and interleaving in the MPEG2 stream, also using the system clock reference (SCR) as a local time basis. Since the GOP structure in the two cells is strictly the same (assuming the answer to Q1 is "yes"...), it means even for the DTS values that for each I frame in the two cells the difference DTS - SCR/300 should be almost the same (and if the SCRs in the two cells start with the same value, the DTS values in the two cells should match). Right ?

Cheers,
SD

robot1
26th October 2004, 18:41
Originally posted by Sir Didymus
[B]
I did two simple (and different...) more tests, with Rejig and Shrink. Rejig has the capability of reauthoring from m2v and ac3 assets, so this picture (http://img9.exs.cx/img9/9418/Rejig.gif) shows its behaviour for reauthoring a new title using the m2v video file and the ac2 audio extracted from "DVD-RB-85%". In this test Rejig is just used as a further reauthoring tool... Results are very very similar to the ones obtained using IfoEdit or Mplex for carrying out the same function.

Great work, again.
Could the graph you showed for DVD-RB cause the dropouts someone report?
In my player I've never noticed one, but if it's a bit out of standard, could it lead to this problems for less forgiving players?

Second:
Isn't there any other free authoring program? (better if with source...) It seems no one of the free ones (probably all mplex based?) can do a perfect job.

I hope your tests will lead to a perfect DVD-RB ;)

Trahald
27th October 2004, 17:01
Question.. I would think the peaks would be underflows (since usually we talk in terms of how the data effects playback) and not overflows? I may be wrong and forgive me if so.. ;)

And underflows/overflows can cause problems with playback on very compliant players

Sir Didymus
27th October 2004, 19:07
@Robot1:

-----------------------------------------------------------------
Could the graph you showed for DVD-RB cause the dropouts someone report?
-----------------------------------------------------------------
I don't know. Really I don't know...
And I am really worried I could induce expectations on this subject: for the moment I would like to discuss the matter with people (like you) that could give contributions to clarify; it's a matter (on my side) of doing tests and reporting a situation that seems to me anomalous [I may be wrong, but it seems to me THERE IS REALLY something anomalous in this...]...

-----------------------------------------------------------------
In my player I've never noticed one, but if it's a bit out of standard, could it lead to this problems for less forgiving players?
-----------------------------------------------------------------
Me too...
I've never seen a dropout since release 0.40 or something like that...
And this could be a good reason for me to stop generating noise...

-----------------------------------------------------------------
Isn't there any other free authoring program? (better if with source...)
-----------------------------------------------------------------
Again. Really I dont'know...
I think the subject is kept as a very precious know how of sw houses and authors of the DVD production packages...


@Trahald:
You are right (or better... you are wrong...)...
I mean, the peaks in the diagrams are all showing overflows, that is situations where (the simulation of) the decoding buffer is exceeding the value of 237568 bytes. In no one of the simulations underflow situations (i.e. some data are requested from the decoder, but none are available in the buffer) have been detected.

Usually in the DVD arena everybody is strongly concerned about underflows, because in the MPEG2 specifications they are in some way admitted and in such circumstances it is expected that the decoder just repeat the last available frame for presentation, generating in this manner very bad situations of video freezing and stuttering. This may happen for MPEG2 streams where the "low_delay" parameter is set to 1. In the DVD "low_delay" is specifyed to be 0, so it is mandatory that the video buffer is never underflow...

Nobody takes care about overflows simply because they are not admitted at all..., so nobody knows what it is supposed to be the right behaviour in such cases... I think that most player just have a decoding memory much bigger than 237568 bytes, so they may have no trouble in this situations, but these are just hypotheses...

Cheers,
SD

Sir Didymus
1st November 2004, 21:27
Ok. I have a last post to share on the subject before closing this thread. The reason for closing it is to involve EVERYBODY on the subject... :) . I think I have actually a good understanding (and a model) showing how the buffer overflows (presented in all this thread) are generated.

I think the overflows are produced by SCR values that are totally legal and valid from the point of view of the compliance with the DVD standard, but they are in some way too "close" each other, causing the rapid fill up of the STD buffer (more quickly than its voiding, that is ruled by the DTS values).

More details in the "STD Buffer Overflow" Thread...