Roger Wilco...

[magic144]

@jdobbs

just fyi (don't lose sleep over this)

I did a FULL-DISC (movie, menus & extras) backup of disc1 from the 2-disc "I Am Trying To Break Your Heart" DVD (documentary about musical group Wilco) - using the latest DVD RB Pro 1.02
http://www.imdb.com/title/tt0327920/

whilst the disc plays fine, I just wanted to report SCR discrepencies on this rebuild - there are a few I noted in VTS_1_5.VOB in the output (somewhere in the closing credits)

the following should be noted:
1 - the SCR is always < PTS (as it should be) - it is ONLY the DTS that seems to have drifted the wrong side of the SCR
2 - the playback seems perfectly fine on my Sony (hence, I am only really putting this out there for information purposes - it is not causing any grief)

I tried this today because I know 1.02 has some new functionality to further handle SCR discrepencies, and I had seen similar discrepencies on this disc before with a previous version of RB - in that instance too, playback was also seen to have been OK in the affected area

just wanted to let you know

it may be the case, therefore, that it is only when PTS drifts the wrong side of SCR that Sony playback will stutter

m

PS - this v1.02 full-disc backup also leaves the infamous "Sony Layer Break pause" in the output (I have removed the line from the RB ini file) - but you probably don't want to hear that!!!

[magic144]

hmm,

doing a subsequent movie-only rebuild of the Wilco disc1 (with DVD RB Pro 1.02),
it seems that in VTS_01_1.VOB, there is in fact a (at least one) case where BOTH
dts and pts are LESS THAN scr - from the output of the ChkTim program from this VOB:-

Video Pack Error: lba=9248 scr=2247484 dts=2239632 pts=2242635

in which case perhaps this disc does warrant some further investigation under jdobbs' meticulous gaze...

[magic144]

next tidbit

it seems I am also unable to remux (with muxman) this rebuild image
I can successfully run it through pgcdemux to separate the video, 2 audio and subtitle streams, but on the remux attempt,
muxman reports a failure "probably caused by excessive bitrate"

[jdobbs]

This was encoded with CCE? What version of CCE are you using? This problem of CCE's bitrate exceeding the maximum setting given is very concerning for me. Do you have one of the earlier versions (before v2.70) of CCE that you could try. I wonder if they might have introduced a bug when they did their v2.70 engine upgrade?

The fact that most encodes (except Movie-Only as is the case in each of your reports) lower the bitrate much more could be masking it. If I can find that CCE's maximum bitrate is consistently over by a set amount, I could do an adjustment before the encode.

At least that explains the SCR problem you report... the video won't fit (as reported by Muxman).

[magic144]

I'll try reverting to an older CCE and see what happens!

are you still using 2.67 yourself?

[jdobbs]

No, I'm using 2.70 -- but I generally do full DVD backups rather than movie-only, so this situation wouldn't have an effect. If I do movie-only, it's typically just for testing.

On the ones your are doing the original seems to be on the edge of what's possible in terms of bitrate -- and they have a very high reduction percentage (because of movie-only) -- so the reencode has a much higher probablility of creating an M2V where the high-end is near/equal to the original. That's when it is extremely important that CCE holds its maximums.

[magic144]

gonna run this thru with 2.67 today and see what happens
(start with the full backup, see if all the discrepencies are gone on that one first, since that's what I want to do with this disc)

mind you, the Wilco percentage was 88% which isn't all that close to the original... I've seen a bad looking transcode at that ratio

[jdobbs]

Not that close to the original? I would consider that exceptionally close.

The quality of the output can't be really directly related to the reduction percentage. For example, if you had an original that was quantized heavily (like you often see with home movies with unsteady camera movement) at its existing bitrate -- a small reduction in size could visibly reduce quality... but if you have one that was only slightly quantized at its original bitrate, you might be able to reduce it 50% or more with good quality.

Also, remember, the closer you get to the original the more likely the peak bitrate will reach the original peak as well. That's okay... except when you exceed the original and possibly put the stream out of spec. The maximum bitrate setting is supposed to prevent that.

[magic144]

see, now I'm going to have to go and find out what quantization is (I keep seeing that word)... if anything else, this is a fun learning experience :-)

so if 2.67 gives a more favourable output, is there anything you'll be able to do to make 2.70 any more target size-tolerant?

will let you know the results tonight - have to go and do that day job thingy now...

[magic144]

@jdobbs
just found this in the other thread

"Just as a note, I found a fault in my algorithm in which the dynamic value will not be considered for non-telecined NTSC sources. So those sources will continue to work the way they always have. I've modified it for the next release."

fyi, the Wilco disc appears to be interlaced (playback on Power DVD shows the tell-tale horizontal lines unless de-interlacing is turned on)
does this mean that using RB 1.02 on this source will not engage your new dynamic algorithm - in which case, I will retest it with 1.03!

please let me know

[jdobbs]

see, now I'm going to have to go and find out what quantization is (I keep seeing that word)... if anything else, this is a fun learning experience :-)

so if 2.67 gives a more favourable output, is there anything you'll be able to do to make 2.70 any more target size-tolerant?

will let you know the results tonight - have to go and do that day job thingy now... If it is consistent I can adjust. For example if I find that it peaks consistently at 5% above the specified rate, I can always reduce it by 5%.

[magic144]

@jdobbs

and do you think the fact the Wilco disc is NTSC interlaced means your dynamic algorithm isn't kicking in?

[jdobbs]

If it is NTSC interlaced... it definitely isn't kicking in. But that doesn't mean it will result in a bad encode... the maximum bitrate will be set to either the maximum or a value equal to 9800 minus the sum of the audio bitrates.

[magic144]

oh, agreed,

what I meant was, IF this was the type of material that CCE didn't handle very well because it was a high ratio reencode, your new algorithm wouldn't kick in in this case to try and prevent that... but you've said it won't kick in, so I will still try it with your next release to see what, if any, difference it makes

like I said, I am also running the reencode today with the older CCE, so I'll publish the results tonight - if the conclusion is that it is more accurate regarding its directed target bitrate criteria, I will probably stick with the older CCE until you can characterize the boundaries of the error margins with the newer version(s)

btw, one final Q about your adaptive maximum - does your new algorithm still have a built-in-default ceiling value of 9000? (i.e. if you don't specify a max value in the .ini file) - that is to say, if you calculated a value of 9100 taking the audio into account, would you still truncate it to a default value of 9000?

[jdobbs]

Yes. But you have to be careful when looking at the ECL file. On NTSC interlaced sources you might see a 7200 as the maximum. That's because the video is being encoded at a common denominator of 23.976 -- when you bring that back to 29.97 it will be 125% of the encoded rate... so DVD-RB sets the intermediate maximum proportionately smaller.

[magic144]

well, full-disc rebuild with RB1.02 using CCE 2.67.00.27,
there are still numerous SCR/DTS/PTS discrepencies

e.g. in VTS_01_1.VOB:-

Video Pack Error: lba=2426 scr=1236935 dts=1236630 pts=1245639
Video Pack Error: lba=2732 scr=1281894 dts=1281675 pts=1290684
Video Pack Error: lba=3042 scr=1327353 dts=1326720 pts=1335729
Video Pack Error: lba=3352 scr=1372702 dts=1371765 pts=1380774
Video Pack Error: lba=3662 scr=1418100 dts=1416810 pts=1425819
Video Pack Error: lba=3976 scr=1464034 dts=1461855 pts=1470864
Video Pack Error: lba=4292 scr=1510260 dts=1506900 pts=1515909
Video Pack Error: lba=4602 scr=1555609 dts=1551945 pts=1560954
Video Pack Error: lba=4915 scr=1601593 dts=1596990 pts=1605999
Video Pack Error: lba=5227 scr=1647234 dts=1642035 pts=1651044
Video Pack Error: lba=5538 scr=1692729 dts=1687080 pts=1696089
Video Pack Error: lba=5857 scr=1739394 dts=1732125 pts=1741134
Video Pack Error: lba=6166 scr=1784999 dts=1777170 pts=1786179
Video Pack Error: lba=6474 scr=1830105 dts=1822215 pts=1831224
Video Pack Error: lba=6780 scr=1875357 dts=1867260 pts=1876269
Video Pack Error: lba=7094 scr=1921291 dts=1912305 pts=1921314
Video Pack Error: lba=7403 scr=1966493 dts=1957350 pts=1966359
Video Pack Error: lba=7712 scr=2011745 dts=2002395 pts=2011404
Video Pack Error: lba=8025 scr=2057533 dts=2047440 pts=2056449
Video Pack Error: lba=8337 scr=2103174 dts=2092485 pts=2101494
Video Pack Error: lba=8651 scr=2149108 dts=2137530 pts=2146539
Video Pack Error: lba=8964 scr=2194895 dts=2182575 pts=2191584
Video Pack Error: lba=9149 scr=2221958 dts=2209602 pts=2218611
Video Pack Error: lba=9365 scr=2253556 dts=2239632 pts=2242635
Video Pack Error: lba=9397 scr=2292888 dts=2284677 pts=2293686
Video Pack Error: lba=9430 scr=2332073 dts=2329722 pts=2338731
Video Pack Error: lba=46381 scr=11767796 dts=11765148 pts=11774157
Video Pack Error: lba=46684 scr=11813523 dts=11810193 pts=11819202
Video Pack Error: lba=46990 scr=11858957 dts=11855238 pts=11864247
Video Pack Error: lba=47239 scr=11896053 dts=11891274 pts=11900283
Video Pack Error: lba=47501 scr=11935128 dts=11930313 pts=11939322
Video Pack Error: lba=47802 scr=11980587 dts=11975358 pts=11984367
Video Pack Error: lba=48110 scr=12026838 dts=12020403 pts=12029412
Video Pack Error: lba=48293 scr=12053608 dts=12047430 pts=12050433
Video Pack Error: lba=48494 scr=12097067 dts=12092475 pts=12101484
Video Pack Error: lba=48680 scr=12140425 dts=12137520 pts=12146529
Video Pack Error: lba=48867 scr=12183637 dts=12182565 pts=12185568
Total number of errors =37

wot to try next???
I've sent you the log/ecl files via email for your inspection

[jdobbs]

And what happens when you try to run that through Muxman? Is it still out of spec?

[magic144]

muxman reports the same error:-

"Multiplex operation failed.
Probably caused by excessive bitrate."

[magic144]

it does produce output up until near the end of VTS_01_5.VOB
then dies in that last .VOB file

but then there are a few more SCR discrepencies in the last .VOB file (VTS_01_5.VOB) of the source (i.e. the DVD-RB/CCE-2.67 rebuild) too:-

G:\_3_wilco_full_2.67cce\DVDFILES\VIDEO_TS>chktim VTS_01_5.VOB
Video Pack Error: lba=45111 scr=151903020 dts=151901831 pts=151910840
Video Pack Error: lba=45427 scr=151949247 dts=151946876 pts=151955885
Video Pack Error: lba=45740 scr=151995034 dts=151991921 pts=152000930
Video Pack Error: lba=46055 scr=152041114 dts=152036966 pts=152045975
Video Pack Error: lba=46364 scr=152086317 dts=152082011 pts=152091020
Video Pack Error: lba=46678 scr=152132250 dts=152127056 pts=152136065
Video Pack Error: lba=46992 scr=152178184 dts=152172101 pts=152181110
Video Pack Error: lba=47305 scr=152223972 dts=152217146 pts=152226155
Video Pack Error: lba=47625 scr=152270783 dts=152262191 pts=152271200
Video Pack Error: lba=47939 scr=152316717 dts=152307236 pts=152316245
Video Pack Error: lba=48250 scr=152362761 dts=152352281 pts=152361290
Video Pack Error: lba=48564 scr=152408695 dts=152397326 pts=152406335
Video Pack Error: lba=48876 scr=152454336 dts=152442371 pts=152451380
Video Pack Error: lba=49190 scr=152500270 dts=152487416 pts=152496425
Video Pack Error: lba=49512 scr=152547374 dts=152532461 pts=152541470
Video Pack Error: lba=49823 scr=152593065 dts=152577506 pts=152586515
Video Pack Error: lba=50137 scr=152638999 dts=152622551 pts=152631560
Video Pack Error: lba=50450 scr=152684786 dts=152667596 pts=152676605
Video Pack Error: lba=50763 scr=152730574 dts=152712641 pts=152721650
Video Pack Error: lba=51081 scr=152777093 dts=152757686 pts=152766695
Video Pack Error: lba=51390 scr=152822406 dts=152802731 pts=152811740
Video Pack Error: lba=51704 scr=152868339 dts=152847776 pts=152856785
Video Pack Error: lba=52023 scr=152915005 dts=152892821 pts=152901830
Video Pack Error: lba=52338 scr=152961346 dts=152937866 pts=152946875
Video Pack Error: lba=52653 scr=153007426 dts=152982911 pts=152991920
Video Pack Error: lba=52964 scr=153052921 dts=153027956 pts=153036965
Video Pack Error: lba=53279 scr=153099001 dts=153073001 pts=153082010
Video Pack Error: lba=53593 scr=153144935 dts=153118046 pts=153127055
Video Pack Error: lba=53903 scr=153190417 dts=153163091 pts=153172100
Video Pack Error: lba=54215 scr=153236058 dts=153208136 pts=153217145
Video Pack Error: lba=54525 scr=153281407 dts=153253181 pts=153262190
Video Pack Error: lba=54845 scr=153328219 dts=153298226 pts=153307235
Video Pack Error: lba=55154 scr=153373678 dts=153343271 pts=153352280
Video Pack Error: lba=55463 scr=153418880 dts=153388316 pts=153397325
Video Pack Error: lba=55775 scr=153464521 dts=153433361 pts=153442370
Video Pack Error: lba=56086 scr=153510016 dts=153478406 pts=153487415
Video Pack Error: lba=56406 scr=153556828 dts=153523451 pts=153532460
Video Pack Error: lba=56715 scr=153602372 dts=153568496 pts=153577505
Video Pack Error: lba=57023 scr=153647917 dts=153613541 pts=153622550
Video Pack Error: lba=57338 scr=153693997 dts=153658586 pts=153667595
Video Pack Error: lba=57652 scr=153739931 dts=153703631 pts=153712640
Video Pack Error: lba=57960 scr=153785622 dts=153748676 pts=153757685
Video Pack Error: lba=58276 scr=153831848 dts=153793721 pts=153802730
Video Pack Error: lba=58588 scr=153877489 dts=153838766 pts=153847775
Video Pack Error: lba=58902 scr=153923423 dts=153883811 pts=153892820
Video Pack Error: lba=59209 scr=153968968 dts=153928856 pts=153937865
Video Pack Error: lba=59525 scr=154015194 dts=153973901 pts=153982910
Video Pack Error: lba=59836 scr=154060689 dts=154018946 pts=154027955
Video Pack Error: lba=60148 scr=154106330 dts=154063991 pts=154073000
Video Pack Error: lba=60465 scr=154152703 dts=154109036 pts=154118045
Video Pack Error: lba=60773 scr=154198540 dts=154154081 pts=154163090
Video Pack Error: lba=61082 scr=154244085 dts=154199126 pts=154208135
Video Pack Error: lba=61397 scr=154290165 dts=154244171 pts=154253180
Video Pack Error: lba=61713 scr=154336391 dts=154289216 pts=154298225
Video Pack Error: lba=62026 scr=154382728 dts=154334261 pts=154343270
Video Pack Error: lba=62330 scr=154428126 dts=154379306 pts=154388315
Video Pack Error: lba=62642 scr=154473767 dts=154424351 pts=154433360
Video Pack Error: lba=62954 scr=154519409 dts=154469396 pts=154478405
Video Pack Error: lba=63263 scr=154564807 dts=154514441 pts=154523450
Video Pack Error: lba=63574 scr=154610302 dts=154559486 pts=154568495
Video Pack Error: lba=63885 scr=154655797 dts=154604531 pts=154613540
Video Pack Error: lba=64195 scr=154701145 dts=154649576 pts=154658585
Video Pack Error: lba=64508 scr=154747043 dts=154694621 pts=154703630
Video Pack Error: lba=64816 scr=154792295 dts=154739666 pts=154748675
Video Pack Error: lba=65129 scr=154838083 dts=154784711 pts=154793720
Video Pack Error: lba=65441 scr=154883724 dts=154829756 pts=154838765
Video Pack Error: lba=65756 scr=154929804 dts=154874801 pts=154883810
Video Pack Error: lba=66064 scr=154975788 dts=154919846 pts=154928855
Video Pack Error: lba=66371 scr=155021040 dts=154964891 pts=154973900
Video Pack Error: lba=66683 scr=155066681 dts=155009936 pts=155018945
Video Pack Error: lba=67003 scr=155113492 dts=155054981 pts=155063990
Video Pack Error: lba=67308 scr=155158744 dts=155100026 pts=155109035
Video Pack Error: lba=67613 scr=155203850 dts=155145071 pts=155148074
Video Pack Error: lba=67890 scr=155244371 dts=155187113 pts=155193119
Video Pack Error: lba=68021 scr=155287437 dts=155232158 pts=155238164
Video Pack Error: lba=68126 scr=155329918 dts=155277203 pts=155283209
Video Pack Error: lba=68220 scr=155366832 dts=155316242 pts=155322248
Video Pack Error: lba=68325 scr=155409605 dts=155361287 pts=155367293
Video Pack Error: lba=68425 scr=155452379 dts=155406332 pts=155412338
Video Pack Error: lba=68527 scr=155494860 dts=155451377 pts=155457383
Video Pack Error: lba=68619 scr=155537341 dts=155496422 pts=155502428
Video Pack Error: lba=68728 scr=155580114 dts=155541467 pts=155547473
Video Pack Error: lba=68791 scr=155616296 dts=155583509 pts=155592518
Video Pack Error: lba=68816 scr=155649476 dts=155619545 pts=155628554
Video Pack Error: lba=68862 scr=155688953 dts=155661587 pts=155667593
Video Pack Error: lba=68917 scr=155731288 dts=155706632 pts=155715641
Video Pack Error: lba=68966 scr=155767470 dts=155745671 pts=155754680
Video Pack Error: lba=69023 scr=155806802 dts=155787713 pts=155793719
Video Pack Error: lba=69082 scr=155849136 dts=155832758 pts=155838764
Video Pack Error: lba=69140 scr=155891325 dts=155877803 pts=155886812
Video Pack Error: lba=69191 scr=155924504 dts=155913839 pts=155922848
Video Pack Error: lba=69241 scr=155963689 dts=155955881 pts=155961887
Video Pack Error: lba=69296 scr=156005878 dts=156000926 pts=156009935
Video Pack Error: lba=69347 scr=156045063 dts=156042968 pts=156048974
Total number of errors =94

[jdobbs]

Please zip the REBUILDER.INI, REBUILDER.INF, REBUILDER.LOG, and REBUILDER.ECL and send them to me. If it isn't too large, I'd also like to get one of the M2V files that is causing both DVD-RB and Muxman to have a problem. It certainly looks like CCE 2.67 can't hold the maximum bitrate on this either...

[jdobbs]

Whoah... wait a minute... I just got your ECL file...

As humbly as I can, I'd like to say I have been blaming CCE inappropriately on this one. :eek:

This appears to be an NTSC interlaced source and the maximum bitrate shown in the ECL file is 7800. Since DVD-RB uses a common denominator framerate of 23.976 during encode -- that rate would become 9750 when converted back to 29.97fps. While that rate is legal (barely) -- it would probably be too big if you are keeping anything more than a single 2 channel audio track.

I seem to remember using that value for something when I was trying "work around" the CCE problem with the HBO "white noise" screen. I'll go back into the code and find out what's what.

Thanks, magic144, for keeping me focused until I recognized the error of my ways.

[Harrysmiith]

O/T Is this problem - bitrate exceeding a maximum - restricted to CCE or does it also apply to other encoders ( like HC) ?

[jdobbs]

The setting would apply to all... but that doesn't mean the problem would. In fact it would be very rare even with CCE. This only happens when the resulting average bitrate is sufficiently high enough that you would actually peak at the maximum provided. Also, all encoders don't decide equally how to distribute the available bits.

What I'm saying is "Don't panic" -- this is a rare occurance -- but one that needs to be fixed nonetheless.

[jdobbs]

@magic144

One other thing (just to be sure)... if you look at your log file, there aren't any "Encoding resulted in no M2V, retrying." messages, are there? One other possibiility is that when DVD-RB experiences it's second failure in creating the M2V file -- it raises the maximum bitrate in order to see if that will fix it (that was something put in for the CCE v2.70 "white noise" failure problem).

[magic144]

no, nothing like that

I have sent the files via email to dvd-rb@dvd-rb.com address - I did this before I started to read page 2 of this thread!

I think the m2v file you want (from the D2VAVS folder) is about 137Mb - how can I get that to you? do you still need it??
how can I be sure it is the one causing the problems... I only think it is the right one because it seems to correspond with the last bit of video MuxMan successfully included in the output before it died - is there a more scientific way to tell?

I noticed the 7800 yesterday and I remember you saying something about 7200 so I couldn't figure out the discrepency...

[jdobbs]

No, you don't need to send the M2V... this is very likely the problem. It's an after-effect of the code I'd put in to stop CCE SP v2.70 from bombing on the HBO intro screen... The whole ECL file creation routine has become convoluted because I have as many exceptions as I have rules related to all the differences in CCE versions.

[Harrysmiith]

What I'm saying is "Don't panic" -- this is a rare occurance -- but one that needs to be fixed nonetheless.

Thanks - I don't know enough yet to panic - that comes later. I'm still at the consciously incompetent stage.

[magic144]

jdobbs - I've just sent you an email reply about your proposed fix - all looking and sounding good