Bitrate exceeding limits.

[GOPPER]

Aagin with 1.03 and using CCE and PROCODER I noticed the Bitrate from the original movie was exceeded ( used bitrate viewer, despite it could have 1/24 calibration error because of stuffed frames, but this was an original 25 frames/second disk.

The original source had a peak value of 7807 kbps in start of VOB1.

The compressed VOB1 had even peak of 8610 with CCE and peak of 87xx with Procoder.

That shouldn't be happening as there is 'Dynamic Peak bitrate anayses'.

That should force the processed version never to go over the original bitrate peak bitrate for a certain segment.

Clearly this happens and what's more worse it exceedes DVD-compliancy rates....

I did a movie-only backup of DIE ANOTHER DAY (james bond) and had ALL audio in.

SO:

DTS: 768
5.1 : 448
2 : 192
2 : 192
-----------------
+ 1600 Kbps....


Now we count 8610+1600=10210 kbps, way over spec. (10080 kbps).


Why do I experience this ?????

I do have to look at the ranges CCE get's specified, (you can easily see these while encoding is running) , but something is not ok here.

Anybody any suggestions ???

Can CCE and Procoder overshoot ?

Or is there maybe a bug ?

Somebody else looked at comparisions of source and output video bitrate ?

Thanx.

Addition: I used Mpegvalid and it also reports this peak.
Addition: I used Bitrate viewer of DVD-LAb-PRO and it reports peak 9188, which isn't anywhere and the highest peak in de graphics is below 8000kbps (1000 based).

I'm lost. However DVD-RB-PRO sends a range of 500-8214 for segment 0, and this is where the peak happens....

that 8214 is strange however, because the original DVD max-bitare is even just 7807.... but I don't trust bit-rate-viewer anymore...... I think have to ask my money back for this tool......
Any other way to get the right bitrates reported of a VOB ?????

Suppose the values presented by bitrate viewer are too high, I think if it gives 7807 as peak for the whole DVD, the MAX_SEGMENT_BITRATE should be set to this value at max., or I didn't understand the 'dynamic peak rate analyses' correctly. Or even MAX_BITRATE determination fails in BITRATE_VIEWER....

Even loaded the VOB1 demultiplexed in DVD-LAB-PRO and visualy the max was 7799 close to 7807 bitrate-viewer reported.

To mention that BITRATE VIEWER in DVD-LAb_PRO 1.53 is buggy too: it reported 670 Kbps average ??? ! and 8506 peak which wasn't found anywhere...

So finally output bitrate looks indeed higher than input bitrate.......

So it's just seeking for the truth:

1) what is useable for CORRECT bitrate analysis of a VOB ????
2) Does CCE tends to go over the specified VBR range ?
3) Anybody noticed higher bitrate peaks in processed backup then in the original ???

Thanx, I have a headache meanwhile and it's really time to catch some sleep...

Peter.

[MarkP]

According to the DVD specs HERE (http://www.videohelp.com/dvd) 10080 kbps is the maximum for compliant DVD for both PAL & NTSC.

Does it play and look ok? if does wouldn't worry too much about it..

Well i wouldn't anyway

[jdobbs]

You worry too much. You've mentioned before that you had problems with maximum bitrates with CCE, with HC (under previous user names ;)), and now with ProCoder. I have doubts that every encoder in the world is faulty in keeping its maximum bitrates.

Therefore I would think the conclusion made might be that the method(s) used for determining maximum bitrates in the input and output streams are incorrect.

There are way too many variables that we aren't sure are consistent between the programs reporting the bitrate. Does the software check just the MPEG stream, or does it check bitrate against the pack structure (which has a variable overhead)? Does it check maximum bitrate at a frame level, GOP level, or possibly some other level? Is it using the SCR values, PTS values, or maybe a frame_counter and the framerate as the basis for intervals in determining bitrate? Is it reporting the bitrate as it is stored on the disc, or the playback bitrate? They can be different.

Here's what I can tell you for sure:

1. DVD-RB calculates all original maximum bitrates conservatively and at the GOP level. This is to ensure any possible errors in maximum bitrate would fall on the low side. That way if there was a single frame peak error in the original stream that exceeded its maximum it wouldn't carry over.

2. No matter what value is determined in the original stream. The value set in max_bitrate (default is 9000Kbs) is never exceeded. Even if the determined peak bitrate would be higher.

3. A third check uses the calculated bitrates of all audio/subtitle tracks that are retained -- and prevents the combination of those tracks and the video from exceeding 9800Kbs, no matter what values are set (even though the multiplexed combination of all can go up to 10080Kbs).

Considering all of that, even in a worst case scenario an encoder would have to be so far off as to exceed the specified maximum bitrate by 280Kbs in order to break the DVD standard. Now -- to further make it unlikely consider this: When you are reducing a DVD to DVD-5 the average bitrate almost always results lower than the original (else there would be no reason to reencode). This would drive the tendency for the maximum encoded bitrate to be even lower...

Nothing is impossible. But the only way I could imagine creating an output stream that exceeds maximum bitrates would be if someone or some program in his/her/its ignorance decided to "out think" DVD-RB and went into and changed the values of the ECL file. In that case he/she/it would, of course, deserve the self-inflicted-sucking-chest-wound they created. On the other hand, as I said, it is a lot more probable that the calculations are off.

[GOPPER]

I consider youre right, so no tools to properly report bitrate although I'm using now the method of demultiplexing the VOB's to elementaire straeams in DVD-LAB-PRO and see the visual report, as this seems to be correct.

By the way, I tested the DVD in a Sony were all that's strange and is not strictly compliant will directly FAIL.

The DVD-RB backup didn't fail however, even not on DTS playback where it's for sure going to make problems if MAX_BITRATE > 10080 Kbps.......

So I assume and have to believe you...... (you in to this bussiness and I'm not, so forgive me my ignorance :stupid: )

By the way, you leave out youre explanation of the 'Dynamic Peak Analysis' which, so far I understand , was mentioned to even prevent the encoder from going above original bitrates, as this happens for sure, especially with sources that already have blocking in the original. The encoder at average compression levels event tends to get over this bitrate to reproduce all atrifacts generated by the decoder, now seen as 'input to be reproduced'.
(that was youre statement in the release notes of 1.02 if I remember correctly).

But indeed it plays well on that Sony, and it will be ok.

Just going to use demultiplexing and using DvD-LAb-PRO for a while, until there is a better tool if I need it.

The problem is I myself have a player that even tends to eat 12000 kbps rates, without making any comments about this.....

But Philips and Sony player are strict.

(nope the previous 'username' was indeed well known to me, but not me, and he should be back also soon , but I've suggested him to take a valium before posting.. :) )

Would be nice to hear about the 'Dynamic peak rate analysis' or has it been removed or 'was it in youre explanation, for which many thanks.

Peter

[jdobbs]

Here's some text where I talked about it in another post:

The point at which the dynamic allocation becomes most important is with higher bitrates and lower reductions... for example, when doing movie-only encodes. When you are doing greater reduction it is less likely that the peak rates will ever reach the original peaks.

That doesn't mean it won't happen, though. The reason I added dynamic allocation was because of an interesting phenomenon you find in reencoding a quantized source through a frame server. Any errors from the original quantization also become the baseline for the new encode. So in a very high action scene the original may have "blocks", for example. When you reencode, the blocks are viewed as sharp contrasting edges and may actually increase the bitrate requirement in order to reproduce what is essentially an error in the original. In the tests I conducted I saw numerous examples of frames that took more space in the reencode than they did in the original -- even though the average bitrate was smaller. That scenario steals space from the rest of the stream... So dynamic allocation limits the peak allocation to no more than the original.

[SAPSTAR]

Talking about bitrates, I would like to mention that the min bitrate value is not always very good...especially for Half-d1/Half space segs....500 was hardcoded in DVDRB, if the average bitrate is 1000 for example....it doesn't give much flexibility to the rate control....Quality could be greatly improved for low bitrates encodes, if the min value was adjusted.....

[jdobbs]

You can set the minimum bitrate with the MIN_BITRATE setting in the "[Options]" area of REBUILDER.INI. 500 was set as the default because of problems experienced with the introduction of CCE v2.70. I'm not sure if it still needed.

[SAPSTAR]

You can set the minimum bitrate with the MIN_BITRATE setting in the "[Options]" area of REBUILDER.INI. 500 was set as the default because of problems experienced with the introduction of CCE v2.70. I'm not sure if it still needed.
OK good to know...maybe you could implement a function like the Dynamic peak, but for mins ? (Dynamic pits ?! ;) )
Keep up the great works jdobbs !!

[GOPPER]

@JDOBBS,
thanks for explaining, it looks a lot like the release notes of 1.02, but i have get the picture I think. By the way even not having blocking (but that's a great example anyway!), can rise the CCE's bitrate to sky-high, because I always use VBR_BIAS=0 which makes CCE trigger happy in rising the bitrate (that's why I use it, because only fast moving scenes have troubles, not the slow ones using CCE SP 2.67.0.10). Think about using different matrices as used in the original-encoding, and the fact artifacts always will be introduced by the decoder, even if it's just 'musquito noise').

(by the way, you should have recieved a sample clip, we use as a test clip for driving the encoder crazy.., the one from HOSTAGE)

so :thanks: JDOBBS


@ SAPSTAR And another reason is that there really exists something like MINIMUM REQUIRED BITRATE in being DVD compliant (yep, looks strange but it's a fact).

So there is not only a MAX_BITRATE but also a MIN_BITRATE.


So the total of everything (audio, subtitling, video that is in the VOB-stream, aktive or not!) should follow the rule:

500 <= actual bitrate <= 10080.


So if you want to use average of 1000 (by the way, what are you trying to backup at average of 1000 ????, just "THE END" maybe :) ) , you even MAY NOT go down 500 Kbps Video, if there is no sound and subtitling present.

So whatch out for this one, it's kind of tricky I don't now if multiplexing overhead is included too, but as video_bitrate cab go down to 100 or so, youre warned !

For examples DVD's have short clips with a logo only. No audio and subtitling in that clip. If the clip blanks to black and you go under 500 Kbps, youre out of specs..........

Result: UNDEFINED !

So JDOBBS probably has had this in mind too, preventing a <500 kbps bitrate by always having the bitrate between 500-9000 in the first place (other mechanisms not dicussing now that also play a role).

But I even see the ENCODER going below 500 KBPS even if 500 kbps is clearly specified, but I even don't try to make this statement a 100% one, as determining the 'REAL' bitrate seems somehow too complex for any tool to perform. Even the well known bitrate-viewer not to trust 100% in all cases or maybe not at all with VOB-files as input.

@JDOBBS, as if you are THE expert on youre BQLC-tool (Best Quality Low Cost :) ) you should be able to please everybody with a REAL bitrate viewer..... Just a simple programm reporting bitrates in THE correct way would do great... such as in DVD-RB but based on original values "MAX/MIN/AVERAGE" taking VOB's as input. Even would be awesome as it would be reporting video, audio, subtitling bitrates on a stream by stream base. I think 99% is in DVD-RB..... but time is valuable for you, I can imagine (maybe someone wants a 'joint project' on this one).

[SAPSTAR]

@GOPPER :

Sorry but I disagree with your limitations on the minimum bitrate....please read the OFFICIAL specs : http://www.mpeg.org/MPEG/DVD/Book_B/Video.html

There is NOTHING concerning the minimum bitrate.......Moreover, the MPEG2 chips are also able to decode SVCD, if the minimum bitrate was 500 for SVCD, the quality would be really poor.....

[GOPPER]

@SAPSTAR:

Do you wanna make a bet ?????

I advise you do not do so....

I asked a Philips engineer on DVD-players........

So I wanna bet, you too ?????

Even if there is no mention of a fact, it won't mean it isn't existing....

Not complete is something else then being correct.....

Correction: I've reread the mail and probably I'm messing up here. There is a minimum requirement officially however, but it's concerning the existence of an audio track. So indeed there is a minimal bitrate concerning the MUXED stream, but it's audio based not video.....

The 500 i'll have to look up, maybe I'm indeed i'm wrong here. By the way I never said Video has to be 500 at least, but the muxed stream should.

And for sure I know, SVCD has nothing to do with DVD compliancy.

Nice having a 'multiple purpose player' DVD-compliancy doens't mean youre player MUST be able to play SVCD, it has nothing to do with that.

[SAPSTAR]

@SAPSTAR:

Do you wanna make a bet ?????

I advise you do not do so....

I asked a Philips engineer on DVD-players........

So I wanna bet, you too ?????

Even if there is no mention of a fact, it won't mean it isn't existing....

Not complete is something else then being correct.....
A bet on what...anyway, It seems I hurted a sensitive cord :) The only reason why jdobbs chose 500 was because of CCE having some errors with the original setting which was 100. (see the history of DVDRB).
In many guides, it's recommended not to go under 300 in the settings, not 500, because many MPEG2 chips don't handle well a too low bitrate. But my initial point was, not to put a zero as a minimum but maybe lower than 500.......and even higher sometimes....it should be something dynamically adjusted as well as the maximum.....

And BTW, I'm myself an engineer...not in DVDs...but as one I perfectly know that we dont know everything.....

[jdobbs]

As my dear old dad used to say: There are only two kinds of people in this world -- those that are engineers, and those wish they were engineers. :)

[Rockas]

... well... in Portugal engineers aren't much appreciated :D
for two reasons... prime-ministers

and

prime-ministers :D

when someone makes something really bad we usually say:
"you must be an engineer" :D

@SAPSTAR & jdobbs
You already know that this isn't about you guys... just a warning... if you ever come to Portugal... just say you are Doctors :D lol :D

[GOPPER]

No not an engineer, long time ago I was.......

Stopped because of more rewarding job offers... (it wasn't me, it was society)

Not knowing some things sometimes even has benefits :)

(although regarding my 'deep' feelings I would it be the other way around...)

I corrected my email by the way, as I couldn't make it up when rereading the engineers message. But a minimum MUX_RATE exists, not to be 500 definitely (i'm chasing this one) but being for the fact officially an audiostream is excpected of a certain type at least implying a minimum MUX_RATE..

If I find the source of my so 'beloved 500 minimum' I'll post it for reference purposes.... and maybe even what's referenced is false.....

But s..t happens, as everywere on the internet bitrate-viewer is used to determine bitrate and JDOBBS explained nicely to me that it can report wrong values and is not to trust 100% on the REAL bitrate..... So what's true and what's not ???

Sure I'm not gonna read 'the colored books'.....

And concerning variable settings: I favor them, always, definitely as long as an idiot like me can not mess things up using the wrong settings :D


Peter.

[SAPSTAR]

You already know that this isn't about you guys... just a warning... if you ever come to Portugal... just say you are Doctors :D lol :D
Good to know...thank you for the info !!! :)