Modification and optimization of robshot method and its possible use with DVD2SVCD

[tylo]

The original Robshot pages are nowhere to be found. Fortunately, the doom9 DVD encoding guides includes the essense of these long gone pages.

However, the basis for the D2SRoBa plugin is here:
http://forum.doom9.org/showthread.php?s=&threadid=41033

The above thread refers to two methods:
- 2passRoBa: This is the original RoBa method. The D2SRoBa plugin uses these ideas only for estimating automatically the number of CDs needed, given a Worst Q.

- 1passBach: This is the OPV Bach algorithm for estimation of optimal Q (with binary search) for filling the last CD. D2SRoBa uses this algorithm.

(EDIT) I have uploaded D2SRoBa v2.4.1 (http://home.no.net/tylo/)
News:
- Added num GOPs in the '1-pass VBR RoBa' group: possible to make longer 'scenes' (more realistic) in sample mpv's created.
- Speedy startup
- Fixes small quirks
- More debug info
- (EDIT) Added batch support for the GOPs value (v2.4.1)

@Byteshark: This is clearly a configuration specific error. Please post or PM me logs with the new (upcoming) version.

[r6d2]

Originally posted by tylo
News:
- Added num GOPs in the '1-pass VBR RoBa' group: possible to make longer 'scenes' (more realistic) in sample mpv's created.

Tylo,

The error in prediction I was getting (measured as unused space on the last CD) without this feature was 2%-4%. (With default tweak 0.8, no gambling :))

I just used GOPs=6 (about 3 secs.) and precision improved 1% without additional test time.

Keep it up!

[Byteshark]

Useing D2SRoBa v2.4.1 - is this enough debug info? As you can see it chose a Q of 1.

========================================================
D2Sroba - DVD2SVCD with Robshot/Bach's method, by tylo
--------------------------------------------------------
- DVD2SVCD Shutdown
- 2003-07-03 07:32:45
--------------------------------------------------------
- Movie length : 01:21:29 (122247 frames)
- Audio size (mp2) : 78224000. bytes
--------------------------------------------------------
Calculations:
- Est. CD user data : (800 - 5) * 1024 * 1024 - 380000
- Est. target mpv sz : (833237920. * 2) / 1.0235 - (78224000. + 0)
--------------------------------------------------------
Results:
- Est. CD user data : 833237920.
- Use number of CDs : 2
- Est. target mpv sz : 1549988839. (2535 kbps)
--------------------------------------------------------
Find Q for mpv file:
- Estimated mpv size : 904919993. (Q=32 / 1480 kbps)
- Estimated mpv size : 1401775600. (Q=16 / 2293 kbps)
- Estimated mpv size : 1512040094. (Q=8 / 2473 kbps)
- Estimated mpv size : 1516169993. (Q=4 / 2480 kbps)
- Estimated mpv size : 1516684780. (Q=2 / 2481 kbps)
- Estimated mpv size : 1517918150. (Q=1 / 2483 kbps)
--------------------------------------------------------
- Q = (1 + 1.0 - #Infinity somewhere! - 1.3) = #Syntax error!
- Selected Q factor : Q=0
--------------------------------------------------------
- DVD2SVCD Recover
- 2003-07-03 07:43:12
--------------------------------------------------------
Replacing values in ecl file:
- opv_q_factor : 32 ==> 0
- opv_brate_max : 2230 ==> 2530
- vbr_brate_avg : 6000 ==> 2535
--------------------------------------------------------
- Actual mpv file size:
03/07/2003 07:44 AM 31457280 Encoded_Video_CCE_PAL.mpv
--------------------------------------------------------
========================================================
D2SRoBa v2.4.1 - DVD2SVCD plugin, by tylo
OS Version: WIN_XP
--------------------------------------------------------
- DVD2SVCD Shutdown
- 2003-07-04 06:13:33
--------------------------------------------------------
Settings:
- CD size : 800
- Number of CD's : auto
- Worst Q. factor : 40
+ One pass VBR Robshot/Bach : 1
- Sample percentage : 2.0
- Sample GOPs : 1
- Adjust Q. : 0.8
+ Do credits bitrate tweak : 0
- Start time : 0:00:00
- Bitrate : 1000
--------------------------------------------------------
- Movie length : 01:21:29 (122247 frames, 25 fps)
- Audio size est : 78238080. bytes (128 + 0 kbps)
- CD user data size : 833237920. = (800 - 5)*1024*1024 - 380000
- SelectRangeEvery : every 750, select 15 frames
- Num. sample frames : 2445
--------------------------------------------------------
Calculations:
- Estimated mpv size : 785131733. (Q=40, 1284 kbps, 15703020 sample sz)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Number of CDs : 2
- Target mpv size : 1549974759. (2535 kbps)
--------------------------------------------------------
- Binary search for Q:
- Estimated mpv size : 904919993. (Q=32, 1480 kbps, 18098844 sample sz)
- Estimated mpv size : 1401775600. (Q=16, 2293 kbps, 28036200 sample sz)
- Estimated mpv size : 1512040094. (Q=8, 2473 kbps, 30241544 sample sz)
- Estimated mpv size : 1516169993. (Q=4, 2480 kbps, 30324144 sample sz)
- Estimated mpv size : 1516684780. (Q=2, 2481 kbps, 30334440 sample sz)
- Estimated mpv size : 1517918150. (Q=1, 2483 kbps, 30359108 sample sz)
--------------------------------------------------------
- Determined Q. : 1
--------------------------------------------------------
Replace values in ecl file:
- opv_q_factor : 1
- opv_brate_max : 2230 ==> 2530
- vbr_brate_avg : 6000 ==> 2230
--------------------------------------------------------
- Encoding Movie
- 2003-07-04 06:15:27

[Holomatrix]

Seems right. Your movie is only 81min. You only need to use one CD not two.

[tylo]

The bitrate is just above max (2530). In this case, D2SRoBa, really should have swithced to CBR MAX automatically, instead of OPV Q=1. That is a planned feature .. sometime...

You could try selecting 1 CD: if you think the quality is OK, you can set Worst Q to the Q it uses, but Q's much above 40 is not recommended, IMO. (The bitrate for Q=40 is already only 1284 in this case).

Your previous error: "# AUTOIT - Unknown function/variable!" seems to be gone! The "#Infinity somewhere!" is also gone in 2.4.1 :)

[r6d2]

Hi Tylo,

Some more room for improvement. Range mode (i.e. 1-2, 2-3, etc.) is supposed to give you information for you to make a choice, right? I just encoded Changing Lanes with 1-2. Excerpts from the log file:


For CDs=1:
- Estimated mpv size : 719669686. (Q=22, 1012 kbps, 14489356 sample sz)
- Estimated mpv size : 736739281. (Q=21, 1036 kbps, 14833024 sample sz)
--------------------------------------------------------
- Q Weight : (720865193. / 49.6688524590164 - 14489356) / (14833024 - 14489356)
- Adjust Q : (22 + 1.0 - 7.00372561122296e-002 - (1.6)) = 21.3299627438878
- Determined Q. : 21
For CDs=2:
- Estimated mpv size : 1282627188. (Q=1, 1804 kbps, 25823572 sample sz)
--------------------------------------------------------
- Determined Q. : 1

However, the select window shows:

1: Q=21, BR=1014
2: Q=1, BR=2188

You can see that for Q=21 it shows a BR closer to that of Q=22, and for Q=1 it shows a BR equal to the target BR, which will not be reached in this case and that will give you a 68% filled second CD.

I think it would be more informational to show the propper obtainable BRs for the CD number in question, and to show also how much BR will be left unused in this case. The user might make then a decision to use that available BR to encode a second audio track or better audio quality, since the video quality could not be much improved.

Or, decide maybe to encode CBR to fill the CDs (even though it will not give you more quality, may speed up the encode a bit).

Regards,

[tylo]

Thanks r6d2,
I'm working on this right now. Also fixed a few bugs reported via PMs. I'll put up an intermediate version with bugfixes and correct bitrates reports. Then a version that also switches to CBR when appropriate + percentage fill of last CD. BTW: Good idea!
Cheers.

(EDIT): Uploaded v2.4.2: Bugfix release. Hope to release the new features before vacation is coming up next weekend.

[r6d2]

Hi Tylo,

I've been using D2Sroba OPV lately and I still have a difficult time to get a full last CD. I've played with the tweak factor (gambling), with the number of GOPs, sample size, and every time I seem to find the right settings for a fully automated encode (that's what this is all about, isn't it? :)), I get screwed by the next movie, which has some sort of different behaviour which has lead me to conclude that "standard" or "optimal" rules for prediction just suck.

I mostly used TMPGEnc before finding your plugin. Used ToK to do prediction. That program has 2 different algorithms to predict. They usually get good results, but you have to use big samples and at the end it's not worth it. You better go with 2Pass VBR at once to fill the CDs.

The same happens to me with D2Sroba. In some movies, D2Sroba overestimates the final BR by much (5%-10%), and on some movies, it underestimates too (very bad).

On the former case you just end up with a not filled last CD, which is not so bad after all given the time you saved by one pass + 2%*6 prediction phase (1.12 pass). But on the latter case, you end up with a too large last CD, which will not fit, or a small "next to last" which really sux: you lost all your encode time.

I tried to figure out why this happens, and I think I found it:

If you encode with an underestimated Q and set max=2530 (max by spec) you are telling CCE: "hey, fella, please feel free to use all the available BR range to keep the Q". And as prediction underestimated, the movie required more bits, so the final avg BR you expected is overrun.

I think that's why @DVD2SVCD passed avg.BR as max to CCE when doing OPV. That way he was pretty sure that CCE would not go beyond that, and thus the warning he gives that you will not get a last full CD!

And read my lips: You will not have BR overflow!

So, I think DVD2SVCD passing avg.BR as max to CCE OPV was not a flaw, it was a feature, and done on purpose!

On 3 out of 10 movies I do, prediction underestimates. Don't know why. I've not been able to find a pattern.

But don't panic. There may be a fix for this!

What about, once finding the Q by binary search (range 300-2530), you do another search for the max BR which will be needed for this movie in order not to overflow?

I don't know which method to use, linear interpolation, binary search, some testing must be done here.

Any opinions from the experts out there (@Bach, @Gerti, @DDogg, @RB, @DVD2SVCD, and others I may forget)?

[tylo]

If you encode with an underestimated Q and set max=2530 (max by spec) you are telling CCE: "hey, fella, please feel free to use all the available BR range to keep the Q". And as prediction underestimated, the movie required more bits, so the final avg BR you expected is overrun.
I don't follow here, but I have a feeling this is wrong: Remember how the samples are generated: You pick 0.6 secs every 30 secs out of the movie (2%, 1 GOP). It all depends how representable the sum of these small cuts are for the whole movie. If it is representable, the final size will be close to the estimated size. The parameters settings (MAX bitrate etc.) are irrelevant - they are set equal during estimation and real encoding. Naturally, if the target bitrate is higher than MAX, you won't get a full last CD, but estimation/real size ratio is unaffected by that. Here we should warn that the last CD won't be filled (%), and switch to CBR MAX. (coming up).
I think that's why @DVD2SVCD passed avg.BR as max to CCE when doing OPV. That way he was pretty sure that CCE would not go beyond that, and thus the warning he gives that you will not get a last full CD!
Nope. All of the above is wrong. OPV Max should be set to Max BR specified in d2s. However, if the target bitrate is higher than Max Avg., we should automatically switch to CBR. That way we also avoid the potensial problem with: Max - Max Avg. < 130. D2S' warning is general, and just state that OPV gives unpredictable sizes.

I think the best way to deal with underestimated Q's (resulting in too big last image), is to automatically reencode the movie with corrected Q (OPV), or use VAF file, and do second pass VBR with correct Avg. bitrate.

[Holomatrix]

There have been alot of good ways to try and guesstimate the final size of doing this one pass stuff but in my opinion there is no way to get exact estimates to fill the CD's all the time without doing sample after sample and ending up waisting more time doing that, then to just do a multi pass encode. The sample(s) choosen could be selected on the slow parts of the movie and miss the action so then would over-quesstimate the bitrate or visa-versa. With the speed PC's are able to encode now a days, I would just do the extra pass and be done with quesstimating headaches. My 2 cents :)

[r6d2]

Originally posted by tylo
I don't follow here, but I have a feeling this is wrong: Remember how the samples are generated: You pick 0.6 secs every 30 secs out of the movie (2%, 1 GOP). It all depends how representable the sum of these small cuts are for the whole movie. If it is representable, the final size will be close to the estimated size.

I've found that precision increases exponentially with sample size, converging to the actual movie BR at about 10%.

But small samples like 1% or 2% have a huge error, which probably is good enough to find the Nº of CDs to use, but not to guess the Q.

The parameters settings (MAX bitrate etc.) are irrelevant - they are set equal during estimation and real encoding.

Tylo, please take the time to do a test yourself. You will find that max *is not* irrelevant in OPV.
BTW, you will find that min *is not* irrelevant either, and in fact it weights more than the Q in OPV mode!


Naturally, if the target bitrate is higher than MAX, you won't get a full last CD, but estimation/real size ratio is unaffected by that. Here we should warn that the last CD won't be filled (%), and switch to CBR MAX. (coming up).

Good feature! But remember that CBR max won't give you a full last CD either. In this case it might be better to decrease the CD size in order to get equally sized CDs, much more comfortable to watch. Again, don't you just hate it when you have to change the last CD for just 12 minutes?

Also if you let the threshold to swith to CBR parametric, the user may set it to, for instance, 2200, and go CBR instead than MP VBR or OPV.


Nope. All of the above is wrong.

Tylo, just try and see for yourself. You will be surprised. Also check the last chapter on the CCE 2.66 manual.

I think the best way to deal with underestimated Q's (resulting in too big last image), is to automatically reencode the movie with corrected Q (OPV), or use VAF file, and do second pass VBR with correct Avg. bitrate. [/B]
Yes, but because of that, you rather end up going Multipass 1 + VAF at once, almost the same quality than OPV and predictable size and time. I.e., we miss the whole point on using D2Sroba OPV, as @Holomatrix and most users of prediction finally say!

Regards,

[Holomatrix]

Originally posted by r6d2
I.e., we miss the whole point on using D2Sroba OPV, as @Holomatrix and most users of prediction finally say!

Regards,

True, so since quick sampling OPV will never be, in my eys, very acurate, then yes, I would be one of thoes people that don't use that feature of D2Sroba but more so the bitrate tweaking for ending credit text, etc.. There's 3 cents :( @r62d, your costing me more money here :) just kiding. Hope you guys can get something 'pretty close' to acurate. Good luck :)

[telemike]

Suggestion for Tylo-

Can you make the status window so that isn't always on top? When I run other programs I have the status window stuck on top of my app and I can't move it.

ALso, what does changing the Ajsut Q setting do exactly?

[Holomatrix]

Originally posted by telemike
Suggestion for Tylo-

Can you make the status window so that isn't always on top? When I run other programs I have the status window stuck on top of my app and I can't move it.

ALso, what does changing the Ajsut Q setting do exactly?

Sorry to intervene Tylo, I was already browsing the forum and thought I would just answer this, if that's alright telemike :) so for the first question, you can modify the status_x and status_y values in the D2Sroba.ini file to put the window where ever you want and to answer the second question, adjust the adjust Q value to increase the video file size. Alot of times the quesstimated Q is to low so you can tell D2Sroba to decrease the Q quesstimated by the adjust Q amount, so if you have 3 in the adjust Q value then when D2Sroba is finished calcutaing the Q value it will be decreased by 3.

[telemike]

Thanks for the Q adjust info. Now it makes sense. I also think I understand the position of the status window, but can it made to be no always on top? I jsut want it disapear when I launch another app while ds3roba is running.

[r6d2]

Originally posted by telemike
Thanks for the Q adjust info. Now it makes sense. I also think I understand the position of the status window, but can it made to be no always on top? I jsut want it disapear when I launch another app while ds3roba is running.
I guess you cannot. If it really bothers you and you are using XP, you can set up a "Encoding" user to do your encodes, then switch back to your normal user to work. You can always go back to see how it's doing. Clean and simple.

[telemike]

you can set up a "Encoding" user

Ahh! I will have to try that! That way, no one can turn off my enocode accidently either! :)

[Holomatrix]

Originally posted by r6d2
I guess you cannot. If it really bothers you and you are using XP, you can set up a "Encoding" user to do your encodes, then switch back to your normal user to work. You can always go back to see how it's doing. Clean and simple.

Does that only work with XP? because in 2000 when you switch users, all open apps by that current user being logged off gets closed.
Thanks

[adge]

Can you make the status window so that isn't always on top? When I run other programs..

I was under the impression that it was dangerous to run other programs when AutoIt scripts (like D2SRoBa) were running. They expect windows to stay where they left them, and focussed windows to stay focussed. That's the reason for the warning about minimizing DVD2SVCD.

I think you run a risk of D2SRoBa suddenly writing "D2SROBA.ECL" or some such in your app :mad:

Adge

[tylo]

We're getting there, but we must put more intelligence into the plugin, before it is the preferred way for everyone to make SVCDs. (even for the multipass fanatics).

I changed GOPs to SF Range (sample frame range). I am experimenting with using fewer frames than one GOP in the SelectRangeEvery, in order to get a lower "every" number with the same Sample %. That gives a better distribution of the samples (every 17 sec, with 2%), but makes awfull short cuts.

I've tried 4, 5, and 8 frames, and it seems 8 works quite well (instead of 15). You'll need to boost the Adjust Q a bit, but it should give more stable estimations. I have also figured out another way to safely cut down number of tests by 1-2 (from 6).

I'll throw out the version for you to play with when I got time.

[r6d2]

@Holomatrix: I have not tried on 2000, but on XP you have to switch users, not log off and logon again (both users keep running their own aps).

@adge: I have only experienced this in very unusual tight CPU conditions, otherwise it gets data input right. However, Tylo suggests that you "don't touch the keyboard while D2Sroba is doing its magic", but that is a only during short periods of time.

@Bach:
Originally posted by Bach
- the last pages of CCE 2.66 manual are a good support of my method implemented by tylo. In other words: it is the prove that my method works as it is supposed to do.

Yes. And they show why.


- D2S passing AVG as MAX is deadly wrong, but have a reason to be there.

Didn't know that story, but makes a lot of sense and that was just my point. I think that once you find Q for 300-2530 then you can constrain MAX a bit to a value between AVG and 2530 that keeps the Q but also allows to get a VBR stream instead of CBR, and with maximum precision.


- For this kind of statistics an error prediction of 5% is not bad in any way.**** However, it is a pain in the a** if you are a "full fill disk freak". It could be easely fixed though if it was possible to switch the order of encoding used by D2S. Lets do the algorithm:

I guess I'm kind of a full disk freak then :). The point is that getting a 5% low is acceptable tradeoff for the time saved, but when you get 5% high, you've lost your time.
Besides, what your algorithm does is something like: "Hey, fella, I'm sorry I did not get it right this time. But you know what? I can increase the BR on the audio just for you to not waste some space".
And the point is that audio is probably OK already, and the unused bitrate was meant to get the video the best you can get.
It's like putting on a second T-shirt instead of a coat before going out on a rainy day.

- to use sample frames with lenghts not equal to (multiple) of GOP lenghts is a big NO NO.

I understand the theory behind what you explain (I guess Tylo does too, since we talked about it), but GOPs are not equally sized, anyway, so how can you write a SelectRangeEvery call with guarantied full GOPs?

****edit: it is not possible, in any way, to do a highly precise size prediction using pure statistics. If it was, I am 100% sure the CCE programmers would implement this method internally and call it "fast 2 pass VBR" or something like that.

You know what they say, "Statistics is the science that proves that if my neighbor has two cars and I have none, then we both have one car, on average".

However, I think you can get any precision you want using an heuristic. To prove the point: Iterate with sample=100% until error is 0.5% :)

More down to earth, Q. factor is a function of many variables: the sample size, the size of each clip, the MAX and MIN values, etc. If we narrow that to a set of variables, with their range of validity, that sistematically affect all movies the same way, we will get the precision we want and only will have to take care of how much time do we wan to invest on the prediction phase.

This is very easy to be done internally.

Maybe they have not thought of it. Remember they wrote a general purpose encoder. The only ones who are, in practice, limited by space constraints, are we who use discrete size CDs. If size was not a constraint, you would probably just go with Q=1 and get the best picture you can get with 1 single pass.

[tylo]

And the point is that audio is probably OK already, and the unused bitrate was meant to get the video the best you can get.
I have to agree with r6d2 here.
to use sample frames with lenghts not equal to (multiple) of GOP lenghts is a big NO NO. If you sample at 6 frames, for example, you will have a scene change every 6 frames. CCE will see this scene change and automatically code an I picture, starting a new GOP, and doesn't matter the GOP size that you have set in CCE. An I picture have a size MUCH bigger than a P picture, and even bigger than a B picture. So, The size of the sample_encode will be OVERESTIMATED by an unknown factor.
True - I see your point. I only wanted to test it. My hypothesis was that if the "unknown factor" is ~ constant then we could still use the results: by compensating for it afterwards (factor could be found by trial). But, like I said, I haven't tested it much.

(EDIT2): @Bach: Let's say 8 frames in each sample. That means that 12.5% of the frames are I-pictures (?) (It's not likely that there are many additional in the short samples). We could then compensate for the extra size of the I-pictures in our estimation.... Remember our main problem: Nonconsistent estimates because of too few samples. The length of them doesn't help us that much. (Hey, it's really just a shot in the dark) :)

@r6d2: (EDIT1): Removed statement about GOP sizes.

[r6d2]

Originally posted by Bach
What my algo says is "hey fella, I feel you want some psychological compensation in order to not feel bad because statistics is not as precise as you want".

Statistics, my dear Bach, can be as precise as you want. And you can always run tests on the data obtained to check its validity over a given universe.

I respect your knowledge very much and I will not enter a flame war with you, nor a right/wrong discussion where the only purpose is to say the last word. (I've seen what you like to do to people that dares to defy your absolute statements :))

Anyway, if you feel bad because your encode have not EXACTLY the size you want, and you must do a 2nd attempt, simply switch to multipass VBR, set 1 pass, set the right AVR, reuse the vaf and do your encoding. Now you have a "tweeked" 2pass ROBA method, better than any 5 pass.

This is a very good idea exposed by Tylo before, and will fix the cases where an underestimation occurs. In my experience, 3 out of 10. D2Sroba can do that automatically.

Wrong. No one of those variables have anything to do with the way the Q.factor is used by CCE. min and max are simply thresholds. Q.factor is just a function of the compressibity of the movie.

I think I read a post of yours that states they even weight more that Q, but anyway, Q is indeed a function of the compressibility of the movie, given MIN and MAX. As DDogg found, if you change the MIN and MAX you get different Qs for the same sample or even the same movie.

And the explanation is simple. I think CCE expects you to make an educated guess on what MIN and MAX should be for the movie you are encoding and the Q you expect. Otherwise, what happens if you say Q=1, MIN=MAX=300? Can't be done! (unless you want to go for a huge MPV).

TMPGEnc, in CQ mode, does something similar. I've experienced similar behaviours moving MAX instead of the CQ factor. I've found that it gives better results to use, for instance, CQ=80 and the propper MAX than MAX=2530 and a lower CQ.

I think CCE and TMPGenc assume you will be "educated" on the range you give as input, in order to remain within specs.

Also, it is obvious that Q depends on sample size. Otherwise, we should just go for sample size="1 GOP", shouldn't we?

Why a "general purpose encoder" could not have a "sampling prediction pass"? Wrong. CCE is used by DVD houses mainly. They are also constrained by "discrete size DVD's".

I sincerely doubt DVD houses use OPV :)

If you encode a private video using Q=1, min=0, max=15000 at DVD resolution you will get an AVR higher than the maximum allowed for a DVD.
Sorry, that would not be an educated MAX for a DVD :)

[r6d2]

Originally posted by Bach
Sorry. I cannot argue about Statistics with a kid who believe Statistics may become an exact science. Educate yourself on the subject.

Well, I must confess I expected more from you. But looking on the positive side, no newbie with ideas gets anywhere in this forum without being insulted by Bach, so I guess that finally, and only after a couple of months, I have become "someone": a kid.

Makes me very proud indeed.

The "tweaked" 2 pass was exposed by me before than tylo.

Nobody here is trying to take away the authoring of your ideas. In fact, your original RoBa algo was the first to use 100% sample size (axiomatically wrong :)).

Q do not depends on sample size. The statistics depends on sample size. And I will not argue about statistics with you anymore.

Please don't, provided that we are not able to find Q for MIN=0 and MAX=infinity, which is the absolute Q your non-automatable-super-fast algo finds, let's just assume that we are doomed to find Q(MIN, MAX, sample size), which is the objective of D2Sroba for OPV.

(I hope you will not dislike Tylo uses the name of your method on his plugin, being it so limited and unable to cope with real Qs. If you do, please tell him. I'm sure he can find a better name.)

(Just kidding :))

But as I said, I expected more from you. Maybe answering how to get a GOP-filled sample, or giving us hints on how to build a good and representative sample. But I guess you prefer to stay on your track and disregard anybody doing samples < 100% and trying to fine tune a utility that might save hundreds of hours to all DVD2SVCD users.

As you said in some earlier post, what a shame.

(No kidding :))

PD: And about the exact nature of statistics (maybe you are confusing terms like precision and exactitude), which you just discovered is not such, we might as well ask the Nobel Commitee to revoke all Physics prizes related to Quantum Mechanics awarded already. They may even donate something to this community. :D

[tylo]

Can someone explain what happened to Bach's posts? If he removed them himself, I will seriously consider to change the name of the plugin, and give him no credits at all. :mad: Moderators, are you reading this?

Anyway, my experimenting with smaller ranges than GOP sizes are over, and the conclusion is that it doesn't work well (no suprise). However, I think that 1 GOP is best to use, because it gives the best sample distribution, so I may remove the GOPs setting as well. (it was good for experimenting).

I work on many enhancements on D2SRoBa now, so please be patient with new updates.

[pacohaas]

looks like he took off, and took his "sacred writings" with him....seems like that kid back on the playground who said: "I'm taking my ball and going home" when things weren't going just his way.

[telemike]

Keep up the imporvements Tylo! I am starting to like your plugin. Has worked good for teh three last encodes I did. Filled up the Cd's to the brim! Cool. :D

[davidh44]

I'm a big fan of Tylo's D2SRoBa plugin, and actually didn't start using DVD2SVCD until I discovered his plugin. It's newbie-friendly for doing a quick and efficient OPV encode. :D

With the default OPV 'sample size'/'adjust Q' settings and 'num of Cds' set to 2, I typically end up with a second disc that's ~730MB out of 800MB. Every now and then, I get an encode which fills the second disc almost completely. So I've been hesitant to tweak 'adjust Q' on new encodes, as I might end up with a resulting file size too big to fit on 2 CDs (happened once). So for me, there's been little in-between: the second disc has been either right around 730MB or close to 800MB. Good enough for me. :)

[telemike]

Is the defaul adjust Q = .8?

I think I bumped it up to 1.0 and nearly filled the second disc.

[r6d2]

However, I think that 1 GOP is best to use, because it gives the best sample distribution, so I may remove the GOPs setting as well. (it was good for experimenting).

Please, leave it there... My tests have been inconclusive so far, but I'm getting closer.

Also, I found a workaround for D2Sroba Multipass not using the maximum BR that DVD2SVCD calculates if it were to do the actual encode (with scan offsets and align headers off):

I set the CD size 6MB bigger in D2Sroba than in DVD2SVCD. So, believeing it has more space, D2Sroba calculates a bigger BR.

This works but requires manual tweaking. If you just forget to uncheck scan offsets and header aligning in DVD2SVCD you are screwed and end up with too big images.

I hope you can fix this in other way, like asking DVD2SVCD the actual calculated BR, and still doing the encode yourself.

Regards,

[DDogg]

Gee, he was Beethoven, Mozart, Bach and I seem to remember one more nic. I guess we wait for the next composer to be reincarnated unless this stunt gets him banned for life. He is a brilliant guy, but I fear he is an egg short of a full omelet.

[tylo]

Yeah, someone should write his memoars. But, rightfully, inbetween his ill-tempered insults and childish outbursts, one could find brilliant ideas and knowledge. Too bad for him.

Talking about brilliant. I've modified the binary search algorithm, so that it reduces the tests by normally 1, often 2, and sometimes 3. I strongly believe this algorithm is robust, but if it should fail (it won't), increase the safety_zone: 100 would give the old behaviour (always 6 tests). However, do not lower the safety_zone.
I'll upload it today.

@r6d2: I now repeatedly read the sample frame num from the tmp ecl file until success (EclCCE done writing it), so hopefully this works for you now.

I'm off on three weeks holiday this weekend. :cool: but I'll check in now and then.

[tylo]

This brings up the general question: Should it be allowed to edit old posts?

This must surely have been widely discussed before, but I still see no good reasons for it. Except for fixing typos, it only allows tampering with the past. Allowing editing some hours after should be OK, but even no editing would be better than the current arrangement. We have the preview function, which should be enough.

Technically this should be a simple restriction based on time.

A post that contains the status of a software project, within a sticky thread, should be made editable. That wouldn't impose much extra overhead, because making the thread sticky requires an intervention from a moderator anyway.

(EDIT): Well, here I used it. Obviously it should be allowed to ADD information, but that would maybe have a few tecnhical issues. (To be honest, I changed a few words in the original text too). ;) Also, it is more OK to edit when you have the last post, IMO. I don't expect many to go berserk like Bach, but I believe that most people would prefere that at least the past is predictable.

As a funny sidenote, Bach had already removed the mentioned "first post" the 6th of July, replaced with the message "gone with the wind". Two days after he still helped us out here, from fumbling in the dark. He later realized that to erase oneself, also the sweet farewell letter had to go.

If you switch users somehow AutoIt/D2Sroba thinks the app is no longer there and shuts down itself.
I will fix this simple issue after vacation.

[pacohaas]

Ok, the forum backup was from mid-May, so I was able to "restore" Bach's important posts from before then. However, the only way to keep them "inline" was to edit existing posts, so I've indicated where I've done that with red headings. I trust that no other members will be as selfish as Bach and delete that info.

If anyone has any important bach posts beyond the last one I inserted, PM them to me with info on where it should be placed.

[homerjay]

has anybody toyed with 1cd creation at all - im using d2sroba 1.44 selecting 1cd
sample 3%
gop 1
adjust q 0.8
and i am getting a second mpg between 150 and 250 mb depending on filters used :confused:

[Holomatrix]

I have a question on how the '?' feature works. I thought it was supposed to give you an option of picking what Q you want to use if the quesstimated Q was going to be not what you set as worst Q, but when I select '?', my encode just goes all the way through with a Q of 1. Is there away for D2Sroba to automatically select a 2 CD encode if the quesstimated Q is going to higher than what I pick as my worst Q?
Plus it would be good for D2Sroba to ask the user what bitrate to use on a given Q because I did two different movies of equil length and one was encode at Q=25 Bitrate=940 and the other movie encoded at Q=63 Bitrate=1623, weird, I would of liked it if D2Sroba gave me an option to decrease the picked Q and decrease the bitrate on the second movie or poped up a window with various Q/Bitrate option I could choose from based on the movie examination that D2Sroba did.


Hope that makes sence :) Thanks

[telemike]

I am using a sample of 2%, Q adjust of .8

I have done a couple 1-cd encodes using CVD format (352x480, 128K audio) with d2sroba 2.4.4 and it works pretty good. You need to change the adjust Q number so it doesn't overflow 1-cd.

http://home.no.net/tylo/

[homerjay]

thanks telemike which way do you adjust the queue for longer films up or down ?

[telemike]

ALso, what does changing the Ajsut Q setting do exactly?
----------------------------------------------------------------------adjust the adjust Q value to increase the video file size. Alot of times the quesstimated Q is too low so you can tell D2Sroba to decrease the Q quesstimated by the adjust Q amount, so if you have 3 in the adjust Q value then when D2Sroba is finished calcutaing the Q value it will be decreased by 3.

[telemike]

Another option is to change the 800MB cd size down to 790MB or some lower number to give you some safety room.

[homerjay]

hi telelmike firstly i gave your advise a go and dropped output to a vcd - end result fitted onto 1 cd with 30mb to spare when selecting a 740mb cd size ( THANKS :D )

but i really want to produce svcd's so i did some more testing and it looks to me like d2sroba isnt calculating the bitrate correctly for 1cd svcd quality i will give egs of my last couple of tests
106 minutes onto 1 800mb with 128kbit audio cd was searching for a bitrate of 843 (bitrate calc says 892)
but when it started encoding it settled on a Q of 63 and a bitrate of 1014 (too high)

so i cancelled out and changed the cd size to 700 just to see what happened overnight
d2sroba log shows

Calculations:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Number of CDs : 1
- Target mpv size : 613427641. (791 kbps)

which is correct but then settles on

Determined Q. : 63
--------------------------------------------------------
Replace values in ecl file:
- opv_q_factor : 63
- opv_brate_max : 915 ==> 2530
- vbr_brate_avg : 6000 ==> 791
--------------------------------------------------------
- Encoding Movie
- 2003-07-24 01:03:41
--------------------------------------------------------
- Actual mpv file size:
24/07/2003 04:19 763151664 Encoded_Video_CCE_NTSC.mpv

hope this info helps

[jsquare]

Same problem here with Saving Private Ryan for 2x800MB SVCDs, I know it's a long movie (2:49) but I can't get D2SRoba v2.4.4 to calculate the correct bitrate, it says target bitrate=1,153 when testing, but when CCE starts the bitrate goes up to 1,650 and end up with a 2nd file that's 1.3GB in size, D2S stops with an error before creating the image for the 2nd CD. I also tried setting the CD size to 700MB with similar results.

Update1:
Tested with VCD resolution 320x240 2x740MB and the bitrate calculation worked fine, but I need SVCD and not VCD settings. It may be a bug, I will try CVD next.

Update2:
Same problem with CVD, CD1=750MB CD2=980MB. The video quality was terrible.

[telemike]

Have you tried the "AUTO" setting and set your worst "Q" level at around 30?

[homerjay]

sorry i think im being dumb :o
if when it is calculating the Q it settles on 63 for any of the longer 1cd conversions i have been doing
how is setting the Q to 30 going to produce a 1cd encode
i would give it a go but i am in work and currently and left my home pc doing a conversion based on number of cd's 1 sample 3% (have found this helps) and adjust Q of 0.7

*edit 13.10 gmt * well normally when doing a conversion i cant connect in but have managed to and aborted my current attempt at 11% the final size for the mpv before audio added would be 755 mb using a Q of 63 and a bitrate of 928

have set as you have suggested and started it up again ( its now i wish i could recycle the earlier stages up to encoded audio completion )and will report back

[ric1234]

Hi all, here is some feedback on my experience.

I do SVCDs only, PAL/permanent subs mostly, did 6 DVD conversions with D2sRoBa already and overall, quality is really good and the software works very well : Thank you Tylo for it :)

Small problem however : my last CDs are never filled up, generally I have 750 Mb out of 805 Mb.

I did some Q adjustments but I have to put as much as -7 to fill up the CD, which seems strange to me... Also I tried to increase the movie sample to 4 or 5 % => it does not help... So I was wondering if other people have the same results with D2SRoBa ? Anyone knows how to get closer to fill up last CD ?

Also here is an idea : since there is a risk of overflow, would it be possible, after the final .mpv is created, to check its size BEFORE the "muxing and cutting" part, and if it appears that it is going to overflow last CD, then automatically add 1 CD image for the "cutting" ? This would be great because most of the time, you would just loose some end credits and I'm ok to discard these.

So for instance, you would select 2 CD but end up with 3 images : 805 + 805 + 30 Mb and just discard the last one.

IMHO the most important is to create cd images ALWAYS smaller or equal to max CD size otherwise the whole job is lost...

I have no idea if it's easy or not to do and I know Tylo's on holidays but if anyone wishes to comment...

Ric

[r6d2]

Originally posted by Holomatrix
[B]I have a question on how the '?' feature works. I thought it was supposed to give you an option of picking what Q you want to use if the quesstimated Q was going to be not what you set as worst Q, but when I select '?', my encode just goes all the way through with a Q of 1. Is there away for D2Sroba to automatically select a 2 CD encode if the quesstimated Q is going to higher than what I pick as my worst Q?

@Holomatrix,

Tylo has made extensive explanation on how does "?" work in this thread, but if you're lost please read again the docs (http://home.no.net/tylo/).

Plus it would be good for D2Sroba to ask the user what bitrate to use on a given Q...

I think you are not getting the idea behind D2Sroba. The whole point is you select the quality you want, no the BR or he number of CDs. The program determines both based on you worst Q.

If you're getting weird results, is just because of the old encoding rule: garbage in, garbage out. You have to set up parameters in their range of validity.


I would of liked it if D2Sroba gave me an option to decrease the picked Q and decrease the bitrate on the second movie or poped up a window with various Q/Bitrate option I could choose from based on the movie examination that D2Sroba did.

In "?" mode, it will let you choose only if two Qs will satisfy your requirent. If only one does, it won't ask and just proceed.

@homerjay,

In particular, D2Sroba is not good to make 1 CD rips, since it cannot calculate Qs above 64, and Qs > 60 are not good anyway in terms of quality. Don't use 1 CD rips with D2Sroba, unless the movie is short and you can get a good Q. IMO, Q=30 is a good figure to start testing.

D2Sroba is quality driven.

If you are looking for size and not quality, there are other tools for that, probably using TMPGEnc (Tok, CQmatic, etc.)

Hope this helps.

[r6d2]

Originally posted by jsquare
[B]Same problem here with Saving Private Ryan for 2x800MB SVCDs, I know it's a long movie (2:49) but I can't get D2SRoba v2.4.4 to calculate the correct bitrate
Hi, @jsquare,

I'm under the impression your problem has nothing to do with resolution. I don't know which Q are you using, but if you get a BR of 1100 os so, quality won't be good at high res.

There is no black magic in D2Sroba. It just automates the decisions you would make manually in DVD2SVCD. You wouldn't go for an SVCD of 1100, would you? (BTW, CCE is not good at low BRs, TMPGEnc does a better job at low BRs and using MPEG-1).

Please make sure you're starting from scratch every time you change the res., and pick mode=auto and Q=30. The program will tell you how many CDs you really need. I guess it would be 3.

Hope this helps.

[homerjay]

thanks for your comments r6d2 i appreciate what you are saying but

the results i am getting out despite being slightly too large are very good quality using a combination of filters

i have previously done 4 pass 1cd encodes that compare very highly to multiple cd rips i have seen using this combination but takes over 18 hours so i want to persue this further

hopefully tylo might have some thoughts on his return

* edit * telemike the last settings you suggested just started producing a 2cd rip qith a Q of 16 :confused:

[Holomatrix]

@r6d2 - "garbage in, garbage out" Yes, that could be what happened. Even though the DVD looks good, I've seen some 3.5 GIG VOB DVD's look better than some 6.5 GIG VOB DVD's.

[r6d2]

Originally posted by homerjay
[B]thanks for your comments r6d2 i appreciate what you are saying but

the results i am getting out despite being slightly too large are very good quality using a combination of filters

Heavy use of filters should increase compressibility a lot indeed. And quality is in the eye of the beholder :)

But also heavy filtering makes prediction more difficult sometimes. Remember the prediction is based on a sample, and the more filtering you do, the more "unrepresentative" becomes your sample.

OPV tries to keep the quality constant. (Quality here means "most alike with the source", not "absolute" quality.) That may be the reason OPV ends up with a higher file size.

With the use of heavy filtering you are probably reducing the Q a lot, Q again understood as "alike the source" but careful: the source is no longer your original source at this stage, it's your source heavily filtered, a different source. So lower Q will mean "alike my filtered source". That by no means guaranties your source will be "respected".

So, you'll end up with a 1 CD rip of a totally different movie... :D

i have previously done 4 pass 1cd encodes that compare very highly to multiple cd rips i have seen using this combination but takes over 18 hours so i want to persue this further

Remember that how many hours to put into it will not change the fact that D2Sroba is quality driven. You are aiming for good quality too, but constrained to disk space, so that makes your encodes size driven.

I think you may be using a fork instead of a spoon to eat your soup. You will get it done, but with a lot of effort.

Best regards,