D2SRoBa Plugin

[tylo]

D2SRoBa is a DVD2SVCD plugin that mainly automates the Robshot/Bach one-pass VBR encoding method.

Download : http://home.no.net/tylo/
v2.5.0 beta2 (30. Aug 2003)

Previous messages on this plugin are in the following thread (from page 4) :
http://forum.doom9.org/showthread.php?s=&threadid=57518

Also, the following thread contains some background info :
http://forum.doom9.org/showthread.php?s=&threadid=41033
Unfortunately, the most important posts (from Bach) are gone,
which makes it harder to understand the underlying ideas.

Features : Predictable one-pass VBR encoding with CCE - fills last CD to the edge
Automatic estimation of optimal number of CD's, based on Worst Q. factor
Setting of exact number of CD's (overrides DVD2SVCD)
Setting of a range of CD's (see below)
Automatic credits bitrate tweaking - available through RB's EclCCE.
Full batch/cmd line support with "one-click" to prepare it!
Status window - tells what's going on
Fixes the OPV max bitrate flaw in DVD2SVCD
Works with CCE 2.50 and CCE 2.66+.

Please, continue the discussion on this plugin here.

I will post a status and planned new features shortly.
Cheers

[telemike]

Tylo-

Any chance of adding AVI2SVCD capabilities?


You're doing a great job! :)

[homerjay]

welcome back :) came into this quite late but love what you are doing so far as i understand it & avi2svcd support would be a great addition please

[jorel]

i'm late too.
welcome back tylo!

great work(as always)....thanks.

:)

[tylo]

Thank you all,

I have only briefly read trough all the posts, so I may have missed a few points, but the state of D2SRoBa doesn't seem too bad. Still, there are a few of things that I would like to do: [list=1] Many want higher Q's than 63. I can extend the search range to 1-127 by adding one sample test. However, this need only be done when the target bitrates is lower that a certain threshold, e.g. 1500 (I'll put that in the ini file), because you won't get that high Q's for the more normal bitrates.
Change in the '?' functionality. Go back and use limits Min Avg, and Max avg. as my original idea. If Min Avg. is not ticked on in D2S, still use the greyed out value there, but set it to a lower one (1200) during D2SRoBa setup. This will avoid the problems DDogg had using my first implementation of this. ;)
Switch to CBR when the target bitrate is higher than Max Avg. + an offset. Two cases:
A) Target bitrate in Max Avg - Max: When target bitrate is at Max Avg., it is still better to use VBR than CBR, but at some point between Max Avg and Max bitrate, we should switch to CBR. This will also fill the last CD completely.
B) Target bitate above Max: This means that Max bitrate must be used. Also switch to CBR and reduce the CD size, so that each CD will be about equal length. (thx, r6d2).
Support for AVI2SVCD. Haven't looked into this for a while, but it's hopefully not too hard.
Use D2S's bitrate calculations for multipass (bitrate credits tweaking). This requires a second termination of CCE, but replaces my own (inaccurate) bitrate calculations. (thx r6d2). [This may get lower priority. I guess cutting credits using frame selection is a more viable solution for SVCD, but OK for DVD encoding. Some folks are actually using D2SRoBa for that!]
[/list=1] Please, post additional tasks that you think should be higher prioritized, and problems that you still have. E.g. How is the batch-feature working for you?

PS : I cannot promise anything here - it's just a nice TODO list to look at when I have some spare time. Also, you folks can see what may come.
Cheers.

[r6d2]

Originally posted by tylo
Many want higher Q's than 63. I can extend the search range to 1-127 by adding one sample test.

Why don't just put a "Really Worst Q" :) parameter on the INI file which is the maximum of the binary search? That way anyone can suit it to its needs and requires minimum coding from your part.

Besides, a parameter like this would help people to find its worst Q faster. For instance, setting "Really Worst Q"="Worst Q", discarding by Q instead of Min/Max BRs manually defined in D2S or in the INI file.

because you won't get that high Q's for the more normal bitrates.

I've found that Qs above 60 really suck, as Bach "predicted". For instance, Q=63 gives a quantization above 10 for almost every GOP, really bad even on a TV.

Change in the '?' functionality. Go back and use limits Min Avg, and Max avg. as my original idea.

Great!

Use D2S's bitrate calculations for multipass (bitrate credits tweaking). This requires a second termination of CCE, but replaces my own (inaccurate) bitrate calculations. (thx r6d2). [This may get lower priority.]

Taerg!

(The opposite of Great :))

Also, you may take a look at a way of ending the binary search when you got the result! (I think I posted an example of going all the way to the 6 tries even when found at first try!) Perhaps some kind of threshold parameter may be used, like 1% precision.

How is the batch-feature working for you?

I PMed you on this. Not working for me.

Also, I don't know if anybody is experiencing this, but sometimes launching a different app makes believe D2Sroba that CCE has ended and DVD2SVCD resumes from Pulldown. I've got this with Ant Movie Manager (great tool) and with Plaxo (a plugin for Outlook). :confused:

I hope you had a good time. Glad to have you back.

[Holomatrix]

Keep up the great work :) For the past few movies I've had use a sample rate of 5% and Q adjust of 0.8 to get pretty acurate results.

[DDogg]

I am at at a loss why you would want to use a "grayed out" number that may not even be there in future versions. So what is broken that needs fixing? I kinda thought it was working pretty well now. I must have missed something.

As for cool and new, do you think it feasible to do a optional second pass depending on some discussed X criteria? Enabling this would have to enable VAF creation of course. (No, I have not thought this out in any detail. As you have noticed, I don't do details very well lately :)) /Add: I was actually drawn to this plugin by the name which implied to me that it did a one pass and then analyzed the vaf and redistributed the bits in some way.

[r6d2]

Originally posted by DDogg
As for cool and new, do you think it feasible to do a optional second pass depending on some discussed X criteria? Enabling this would have to enable VAF creation of course.

One possibility for the "X" criteria which would absolutely work is Bach's original RoBa method: 1 pass (OPV+VAF) and then Multipass VBR (1-pass). This is somehow included (unsupported) with the "Roba Visibel" INI flag (I guess the typo was done on purpose by DVD2SVCD to make it even harder to use it :))

Anyway, I did some terrific encodes with this unsupported method. It takes twice the time as an OPV, but fills the disks to the edge. I've read some reports of problems with this, but it has worked for me every time except for two ridiculously low BR encodes (1 CD) that in spite of all did not look too bad.

It does not require any manual bitrate allocation, and you "always" get the quality you want.

If DVD2SVCD is working on other stuff and won't ever support this method formally, maybe Tylo might do it. It would not be hard.

You might even improve the method by using the current binary search for a first OPV pass, and then correct with the real Q for the whole movie and obtained BR, using a simple proportion.

Then you do a second OPV with much more precision, and filling the CDs to the edge or getting really, really close (it would be a 100% sample, so to speak :)).

Any opinions?

[Crackhead]

first, really great tool
i use it mainly for 1cd-encodes, and it works great, only the end-file-size sometimes doesn't fill the cd to the edge, so a little manual tweaking with the settings is necessary, but that's ok!

little change i would like to see in the next release (if any further releases come;) ) is that the D2SRoba window could be minimized or at least could be moved to any position on the screen!
and if it's possible, let D2SRoba be permanently opened/working, even if D2S is minimized!

Sometimes I work with the PC while D2SRoba does his job and it's unnerving when some buttons or anything is hide behind the D2SRoba window, and I instinctively minimize all working apps which I don't need at that moment, so it did happen to me that i minimized D2S during encode...that's annyoing!:mad: :o :angry:

I hope you could implement this little changes;)

Greetz, Crackhead

[Holomatrix]

But you CAN move it to any part of the screen in a round about way :) In the D2Sroba.ini change the values of status_x=320 status_y=0 to whatever position you want on the screen. Or even put in minus values or a too high of a value then you could only see a little smiggin of the window on the screen if you wanted to :)

[tylo]

@r6d2: The "Really Worst Q" must be a power of two, because we search in integer space. Only 64 and 128 are then realistic numbers (32 if you are a real quality freak). I'll add an entry: power_of_2_max_q=64 (default) in the ini file, but as many noted, normally you'll get terrible results with higher Q's.

Regarding "early stopping of the binary search", I think its a good idea to use a threshold parameter, like you suggested. I'll add it to my list.

@DDogg: We had a fight over the "?" before :p, so I won't touch it now. There are some room for improvements here so I'll come back to it, but there are more important things...

On your second point, if I get you right, RB may be the guy you should ask. Is what you want something similar to the functionality that he did with the credit bitrate tweaking in EclCCE? There he lowers the bitrate from a certain time, and raises the bitrate before that, so it equals up. What he needs to do is to modify that method, to do more advanced bitrate changes... I could always make the interface to kick it off during DVD2SVCD (as with the credits tweaking). Actually, I may be off topic here.

@Crackhead: Thanks for reminding me on the minimize problem. I had planned to fix it, but it went away during vacation.

[r6d2]

Originally posted by tylo
[B]@r6d2: The "Really Worst Q" must be a power of two, because we search in integer space.
Not necessarily, you can do a binary search on any ordered set you want, integer or real, positive or negative, with or without power of two limits.

On an integer space, you just have to make a choice whether you'll round or truncate (I'd rather round), but the binary search always will find its target, no matter what.

[homerjay]

:confused: i love it when it gets tech in here lol :p

[tylo]

On an integer space, you just have to make a choice whether you'll round or truncate ... but the binary search always will find its target
You're absolutely right, silly me (actually, Bach gave me that idea - yeah, blame it on him :p).

[DDogg]

One possibility for the "X" criteria which would absolutely work is Bach's original RoBa method: 1 pass (OPV+VAF) and then Multipass VBR (1-pass). This is somehow included (unsupported) with the "Roba Visibel" INI flag (I guess the typo was done on purpose by DVD2SVCD to make it even harder to use it )

Exactly, however I was inactive when that whole thing came up and I never came up to speed. I have a couple of questions.

1> Should not the first vbr pass use the full min max bitrate as tylo does instead of min and projected as D2S does now?
2> The next and susequent multipasses should use the min and projected bitrate. Correct?

I think dvd2svcd would take a second look at this section of his code if he could get good feedback and an exact set of instructions on what that special mode should do step by step. He has way too much on his head at the minute to do all the research and go back to that old thread. I elect r6d2, anybody second that? :-)

\Edit
From the long thread (I am having trouble understanding this. Hey, remember I am just a tool user):
- run CCE 1passVBR, Q.factor = 30 (or 40), min = 0, max = 2500, bias = 10, and the other settings as usual;
- Calculate the average bitrate (avb_1pass) of the mpv file created by CCE, in accordance with the equation of item 4;
- Calculate the average bitrate that should be used to fill 1CD (avb1), 2CDs (avb2), 3CDs (avb3), or 4CDs (avb4):
-If ((avb_1pass - avb1)/avb1) < 0,1, then avb_1pass = avb1;
-Else if ((avb_1pass - avb2)/avb2) < 0,1, then avb_1pass = avb2;
-Else if ((avb_1pass. avb3)/avb3) < 0,1, then avb_1pass = avb3;
-Else avb_1pass = avb4.
- Execute multipass, 1, 2 or 3 passes, using min = 0, max = 2500, avr = avb_1pass, bias=10, and the remain settings as the previous one. Note that this multipass will re-use the vaf file created in the 1passVBR. So, the first pass of this multipass is NOT the vaf creation. It is the mpv creation.

[r6d2]

Originally posted by DDogg
1> Should not the first vbr pass use the full min max bitrate as tylo does instead of min and projected as D2S does now?

By first VBR pass you mean OPV, right? That's what the method is about. And actually, in RoBa D2S uses Min and Max as specified on the Bitrate tab (as it should).

2> The next and susequent multipasses should use the min and projected bitrate. Correct?

Wrong again. It must keep on using Min and Max, and it actually does, as expected.

Come on, DDogg, you should really trust more on DVD2SVCD's abilities as a programmer :)


I think dvd2svcd would take a second look at this section of his code if he could get good feedback and an exact set of instructions on what that special mode should do step by step.

That would be great, but I think it's quite OK as it is. You have to do some template tweaking if you're using RB's templates (enable the use of the VAF), but that's all I think.

It would be great if DVD2SVCD decides to support this, since it is definitely the best of both worlds: Quality you want and filled CDs, with the minimum time required to achieve those two objectives.

[Holomatrix]

I posted a question/mis-understanding in the other thread but maybe it could go here in regards for how the '?' amount of CD's should work or if what I'm suggesting is even feasible.

I had a discussion a little while back which I can't remember right now (memory is going weary) about what I thought the '?' in D2Sroba meant, I was under the impression that the '?' in amount of CD's would allow me, or D2Sroba tell me that "I have determined that your movie will fit on 1 CD at a Q of 63 or 2 CD's at a Q of 50, What do you want to choose?" then encode at that, or automatically pick 2 CD's if the determined Q is higher that your picked worst Q. If determined Q is worse than my worst Q then I wouldn't mind D2Sroba automatically doing more passes to determine the Q for 2 CD's. Does that make sense?

[DDogg]

By first VBR pass you mean OPV, right? That's what the method is about. And actually, in RoBa D2S uses Min and Max as specified on the Bitrate tab (as it should). Oh, I see what you mean. In regular 1pass vbr d2s uses min and predicted, which IMO, is incorrect. I think you are saying in that special mode, which I just figured out how to turn on, it does not. OK, I'll check that. Remember, I was away during all of this. Plus, with MAF, it may well bake up an entirely new cake.

This may be entirely wrong, but what I am going to try is letting dsroba does its thing with vaf turned on. Then try a multipass on top on that. May that will not work, but I think a lot of the problems mentioned in the old thread were from 2.50 crc errors and such. Maybe with the newer version it may be worth a shot. Sometimes, NOT reading old information may keep one from getting preconceived notions.

[r6d2]

Originally posted by DDogg
Oh, I see what you mean. In regular 1pass vbr d2s uses min and predicted, which IMO, is incorrect.

It is incorrect with what we know now, but according to Bach, it was your very own discovery of low Q findings using Max=Predicted what lead DVD2SVCD implement it that way, which turned actually being sort of equivalent to CBR :)

I think you are saying in that special mode, which I just figured out how to turn on, it does not. OK, I'll check that. [...] This may be entirely wrong, but what I am going to try is letting dsroba does its thing with vaf turned on. Then try a multipass on top on that.

@DDogg, This is exactly what the internal RoBa does, no need to do it manually!

May that will not work.

Chances are it will, give it a shot. :)

If you're using RB's templates maybe VAF activation in DVD2SVCD won't be enough. I remember having to enable "Use existing" on a special Progressive-RoBa template.

[DDogg]

It is incorrect with what we know now, but according to Bach, it was your very own discovery of low Q findings using Max=Predicted what lead DVD2SVCD implement it that way, which turned actually being sort of equivalent to CBR Yeah, but the problem is I don't remember that being so, but d2s and I used to be in constant touch via IM back then, so maybe that came out of some of those chats.

Yep, I am trying the internal robshot-bach special mode with the semi-auto MAF as we speak. Frankly, if it works, this is what I was looking for. A easy "tool-users" approach to robshot-Bach, and yes it is embarrassing that I am just getting around to it now. More so because I didn't even know it was in there. :o :eek: :o I thought it had been a temporary measure that had been abandoned and stripped out of the code as a failed test.

I can see a problem already in that the first pass after the vbr pass is creating a VAF. So, do you have a link to whatever special template you were speaking of?

[r6d2]

Originally posted by DDogg
I thought it had been a temporary measure that had been abandoned and stripped out of the code as a failed test.

I guess you're somewhat right, that's the feeling Bach itself left on that thread. But gee, the code is still there!

I can see a problem already in that the first pass after the vbr pass is creating a VAF. So, do you have a link to whatever special template you were speaking of?
This is why I had to set "Use Existing" in the template. Come on, DDogg, read before posting! :D:D:D

Anyway, don't abort your encode. It will work anyway, just with one more pass than strictly needed. This virtual second VAF pass is the same as if you would have guessed at first the number of CDs, and the next VBR pass will do the actual encode.

[tylo]

r2d6, I too think all this stuff is pretty hard, so be fair. E.g. the template for the second pass must have create_new_vaf=1, which means that CCE should use the existing one, and does not mean it should create a new vaf, as you would think. Makes my head spin.

[DDogg]

Ok, see if I got it. I had thought that smack talkin' R6D2 :) was speaking of the EclCCE.ini. I see now that he actually meant the particular CCE template I was using. Ok, now, Tylo, you are saying the EclCCE.ini should also have create_new_vaf=1. Alrighty then? Is there any other freaking secret settings or handshakes to get this secret society operating?

/add: Looks like that is working. You know if I get this working I am going to do a write up so the average Joe Dokes tool-user does not have to go through this crap :devil:

[r6d2]

Originally posted by tylo
r2d6, I too think all this stuff is pretty hard, so be fair.

Sorry, DDogg, just in case it bothered you, I did not mean to be rude, hence the grins. And BTW, I don't think this stuff is a summer camp either.

E.g. the template for the second pass must have create_new_vaf=1, which means that CCE should use the existing one, and does not mean it should create a new vaf, as you would think (Grr, RB). Makes my head spin.

I think this is a CCE "feature", ill named parameter. IMHO nothing to do with RB's templates.

Maybe the programmer is using negative logic :)

[tylo]

Sorry, I guess I just added more confusion. When you set the 'Use existing file' (vaf), and you save it as a CCE template or an ecl file, that will have an entry "create_new_vaf=1", (if you should look at it - you dont need to). So, don't touch EclCCE.ini. Otherwise, I think you are all set.

/Edit: It's going too fast. This thread will be 15 pages in no time.:eek: .:D

[r6d2]

Originally posted by tylo
Sorry, I guess I just added more confusion.
Just open CCE, select RB's Progressive template, save it under a new name (i.e. Progressive RoBa) and select "Use Existing" instead of "Create new file". Replace your new template with this one and select it in DVD2SVCD.

[DDogg]

"just in case it bothered you"
@r6d2, now I thought you already "knew" me better than that. I'm pretty dumb so my skin is thick as a bullsdogs. :) Btw, the last instruction is precise and cystal clear. I'll steal it if I do a "how to join the secret freaking society of automated robshot-bach using dvd2svcd for dysfunctional dummies and ddoggs" :)

As for your "Wrong again. It must keep on using Min and Max, and it actually does, as expected." That may confuse people. Of course it uses the same min and max, but it uses the predicted as the average because it is in multipass mode. That is the whole point of the conversation. Comeon, you know that :) Had to throw that in.

@Tylo, I just realized I pulled this thread off way topic. Sorry man. I know you were trying to get a clean thread. Maybe I can get a mod to move some of this stuff back to the robshot-bach thread.

[Holomatrix]

Originally posted by r6d2
Just open CCE, select RB's Progressive template, save it under a new name (i.e. Progressive RoBa) and select "Use Existing" instead of "Create new file". Replace your new template with this one and select it in DVD2SVCD.
Ok, just so that more people (newbies) don't get confused, you ONLY make that ONE change to the CCE template ("Use Existing" ) ONLY if you want to try the RoBa MultiPass VBR option in DVD2SVCD and not be using this plugin, right?

[r6d2]

Originally posted by DDogg
[B]"just in case it bothered you"
@r6d2, now I thought you already "knew" me better than that. I'm pretty dumb so my skin is thick as a bullsdogs. :)

You never know when an addict is high on crack so better safe that sorry :)

because it is in multipass mode. That is the whole point of the conversation. Comeon, you know that :)
Actually, I had not the slightest idea.

:D:D:D
to move some of this stuff back to the robshot-bach thread.
Maybe a better idea would be to split this thread on a user's one and a developing one. So, we may deliver cooked guides to the uninitiated and still fell free to ramble about new ideas. Sort of the AviSynth Usage/Development forums, but on a thread level.

Besides that, I still think the "pure" 2 pass RoBa method belongs in here and may be optimized by Tylo (remember it's the same idea with 100% sample size :)) DVD2SVCD's silence on this matter and Clixo's posting in other thread makes me think he is quite busy indeed.

What do you guys think?

[DDogg]

"(remember it's the same idea with 100% sample size )"

1> Expand and expound please: By that, do you mean the vbr run, in this case done by Tylo's plugin, has the Q calculated to approximate an equal bitrate to the bitrate that will be used in the multipass runs? Why is that better than an empirical Q of X (like say 50) for the first run?

2> [clarified] And for the purposes of further conversation, Why would a 100% one pass vbr pass with VAF followed by one multipass run using the proper calciulated bitrate be any better than a 1 or 2 pass standard multipass which would be a VAF run and 1 or 2 standard multipass passes? (That was hard to say)

Oh, d2s is way stressed right now with beta testing, work and probably me bugging him about last minute additions. I am sure he will get around to this when he gets some air.

[r6d2]

Originally posted by DDogg
"(remember it's the same idea with 100% sample size )"

Expand and expound please: By that, do you mean the vbr run, in this case done by Tylo's plugin, has the Q calculated to approximate an equal bitrate to the bitrate that will be used in the multipass runs? Why is that better than an empirical Q of X (like say 50) for the first run?
I don't quite understand you question but let me elaborate more on the matter to see if I make myself clear.

RoBa_method_explained_by_a_lesser_creature_mode ON

RoBa consists originally on 2 full passes (at least). The first one is an OPV with the Q you want, based on that all movies with the same Q have the exact same quality (I cannot elaborate on the basis on this assumption, they are above my knowledge, but let's grant Bach this one.)

On this OPV pass you get the VAF file, as well as the exact BR the movie needs to get the selected Q.

Based on this BR, you set aside 10% for audio and estimate how many CDs are needed to make it fit with at least that Q.

Here is the second important Bach's assumption: the VAF can be reused since you did not change any of the things the CCE manual says you shouldn't.

Then you go for the second pass, using Multipass VBR (Bach did not define N), and using the VAF you got before.

RoBa_hoping_to_be_understood_by_other_lesser_creatures_mode OFF

So, the first pass is like if you did a 100% sample with Tylo's plugin in Multipass mode, knowing previously the number of CDs you needed.

Just a glitch the method has: It's not convenient to reuse the VAF if the BR has changed "heavily". According to the CCE manual (I happen to love the RTFM acronym), heavily is when outside a safe zone defined by: [50% BR, 200% BR], in other words, when dividing by more than 2 or multiplying by more than 2.

But moving around the borders of this vicinity is dangerous. The manual says, OK, but what happens on the borders? Border conditions usually break up programs. And the manual quotes this talking about Multipass: "if the setting of the BR is a major change, however, it is better to recreate the VAF because better encoding result can be obtained with less number of passes". I think they assume you are doing at least 2.

But IMHO, 1 more VBR pass is long enough!, because since you already have all the needed BR for the quality you expect, more that 1 pass is superfluous. N-pass cannot build more BR from nowhere, that's what there is and no more, and by definition, since it is the Q you used which defined it, there is no reason for it not to be enough!

And the glith DVD2SVCD's current RoBa has is a border case: it may indeed happen.

Suppose you have a 16:9 movie around 90 minutes, every guide will say: use 2 CDs! don't go below 1600 kpbs on video! However, thanks to this mode, you dare to try a mid-range Q, and you find out that it would fit on 1 CD, with a ridiculously low BR, or in 2, with double the BR. i.e., you are at a border condition.

By just a bunch of bytes DVD2SVCD may decide to use 2 CDs, and then the VAF may not be too useful for just 1 more pass. In this case it is recommended to recreate it, or keep the 1 CD encode.

Tylo implemented a soft way to deal with this cases: the Min and Max BRs on the INI file. They tell D2Sroba don't use stupid BRs, which we have found not to be so stupid after all if you know what you're doing and have the appropriate filters set up.

So, where does the "Really Worst Q" come in? The really worst Q is the absolute Q you don't want to tresspass (or even try!). It's the high limit for your encodes. The "simple" Worst Q is likely to be something like 30, but your Really Worst Q eliminates the need for the soft limits, again, if you know what you're doing.

The decision is always made on a quality basis and not on magic numbers.

There's another option I insinuated previously: go for a second corrected OPV instead of VBR 1 pass! But my fingers are in pain and your eyes too. Some other time.

Sorry about the rant guys, I only hope this is clearer now. And @DDogg, please don't ever ask me to expand and expound again :)

[DDogg]

Hmm, I am still going for simple and stupid.

Let's assume one uses Tylo's plug-in to get the predicted Q. Lets say it is 47 for conversations sake. Now we plug 47 into d2s for 1-pass vbr and set N passes for dvd2svcds RoBa Multipass VBR Mode. That should be all it takes to get the best encode you can under the circumstance of low BR. Would you agree it is just this simple?

As an example, I just finished a dvd2svcd RoBa Multipass VBR Mode encode. I plugged in a q of 50 because I knew that was reasonably close to Tylo's predicted and used 2616 as max and 200 as min as I was using 128 audio. I set dvd2svcds RoBa Multipass VBR Mode for only 1 pass. So it only did 2 passes.
1> A 1pass vbr @Q50 which also created the VAF
2> Standard vbr pass which used 2616 950 200

The size was a perfect and exact 1 80 min CD as I had set all my bitrate slots to 1. Quality was very good given the ridiculous BR of 950 (approx 144 min progressive source). I used the semi-auto/MAF. Only a trained eye would have noticed the artifacts on textured walls and such although the viewing was just on a 32in standard 12 year old JVC TV. I think this was a tough encode because it was 4:3 full screen and do remember I only used 1 pass in dvd2svcds RoBa Multipass VBR Mode. I am encouraged to pursue this.

Note: I used CCE 2.67.00.11 BTW, I had previously done the same encode with ProCoder 2 pass vbr mastering mode. It is awesome at low bitrates. The CCE encode described above equaled or exceeded it in quality.

Update: I have tried subsequent attempts using the dvd2svcd RoBa Multipass VBR Mode and have not been able to get it to work. The first one pass vbr run goes fine, but at the start of the second pass, dvd2svcd seems to get into some type of endless loop. It brings up CCE, but the log file just continuously scrolls and CCE is never loaded. I'll try to figure out what is happening.

[Holomatrix]

I had the same thing happen "It brings up CCE, but the log file just continuously scrolls and CCE is never loaded. I'll try to figure out what is happening." I'll see if I can find out what I did again. I think I just changed Tabs before the encode started and didn't leave it on the log Tab.

Edit: yep, the only changes I made was not leaving DVD2SVCD on the log tab (changed it to the conversion tab after start) and changed the Q to 20 (not 5)) try a chapter and let me know.

[homerjay]

@DDogg please start a new thread when you have worked all this out a dummies guide to d2sroba if you will including links or paste ins of any .ini file scripts used etc.

i have been trying to keep up with this but it has gotten out of hand very quickly and without wanting to offend any of the great minds at work in here your thread above is the first i have understood since tylos initial explanation of the tool :p

[r6d2]

Originally posted by DDogg
Let's assume one uses Tylo's plug-in to get the predicted Q. Lets say it is 47 for conversations sake. Now we plug 47 into d2s for 1-pass vbr and set N passes for dvd2svcds RoBa Multipass VBR Mode. That should be all it takes to get the best encode you can under the circumstance of low BR. Would you agree it is just this simple?

Yes indeed. And not only in low BR cases. Anytime you find a Q and use it to to build a VAF to use with Multipass VBR. This sort of works like this today with Tylo's plugin setting Multipass Mode.

However, in this approach we are replacing the first VAF creation pass of a normal Multipass encode by a manual OPV, which will generate 2 files (the VAF and the MPV), but we only use one. This is probably slower than letting CCE just build the VAF as it normally does.

What you're doing is nothing more and nothing less than a 100% sample size Q search :), so we are perfectly tuned, simple and stupid:
Originally posted by r6d2
So, the first pass [of RoBa original method] is like if you did a 100% sample with Tylo's plugin in Multipass mode, knowing previously the number of CDs you needed.

See?
Originally posted by DDogg
I only used 1 pass in dvd2svcds RoBa Multipass VBR Mode. I am encouraged to pursue this.

That's my whole point. No need for more. You got the BR you need for your Q and 1 pass is enough.

The concept behind Bach's original version of RoBa is that you don't tune the D2S Q.factor for every movie, you set it to your personal taste (remember assumption 1) and then keep on using it.

Using Tylo's plugin to find the Q previously helps to avoid the border case mentioned on my previous post, which is very good. So this is just the perfection the current D2S implementation needs!

[r6d2]

Originally posted by Holomatrix
Ok, just so that more people (newbies) don't get confused, you ONLY make that ONE change to the CCE template ("Use Existing" ) ONLY if you want to try the RoBa MultiPass VBR option in DVD2SVCD and not be using this plugin, right?
Yup. No need to do template tweaking if not using internal RoBa.

[DDogg]

Ok then, now it sounds like my job is to try to get D2s (him) to look at the error that is happening in the auto RoBa method. Frankly I kinda of think it might be a simple thing. Maybe Tylo will add this functionality. This whole topic is not exactly at the forefront of Dvd2svcd's efforts at the moment as he is concentrating on the DVD2DVD stuff. However, the RoBa method, copled with DVD creation would really get peoples attention. It is an obvious natural.

[off topic rant on]
Who knows, maybe somevody might actually donate a few bucks to the guy. It really pisses me off, he works so damn hard on this stuff and then gets sooooo screwed by people. Hey, isn't this stuff worth at least 10, 15, or 25 bucks for goodness sake![/rant off - sorry]

[r6d2]

Originally posted by DDogg
However, the RoBa method, copled with DVD creation would really get peoples attention. It is an obvious natural.

Yes indeed.


Hey, isn't this stuff worth at least 10, 15, or 25 bucks for goodness sake!
Certainly it is! I hope he maintains his Amazon wish list current for when the new version is released. I think most people find that a more trusted way to transfer money.

And if I were him, would make it shareware at a reasonable amount like those you suggested. All new features might be enabled only after doing your contribution.

BTW, the link to the Tylo/Roba method in www.dvd2svcd.org is not working for me, requires admin access. Maybe it's like that since the Bach post deletion.

I don't know if he is reading frequently, but you have direct line with him. Please give him my suggestions, obviously only if you support them.

[r6d2]

Originally posted by tylo
How is the batch-feature working for you?
Hi Tylo,

This is the batch generated for 1 movie:

@ECHO OFF
@REM DVD2SVCD Batch Control

Start "2003-08-09 16:19" /WAIT "C:\Archivos de programa\DVD2SVCD\Tylo\D2Sroba.exe" -batch -opv_roba -num_cds 1 -cd_size 800 -sample 2.0 -gops 1 -adjust_q 0.8 Start /WAIT C:\ARCHIV~1\DVD2SVCD\DVD2SVCD.exe -d2s:"H:\Volver al Futuro\Source\dvd2svcd project file.d2s" -run -exit

When run, D2Sroba starts, D2S starts, and then D2Sroba shuts itself down.

[DDogg]

Tylo, how would one script a specific number of randomized frames from a source? Let's say I always wanted to sample 100,000 randomized frames from a source no matter if it was 90 minutes long or 2 hours long. Do you know an easy way to do that?

[tylo]

In between all discussion, I've added a new version: D2SRoBa v2.4.6
A number of small, but very nice fixes and enhancements there:

- In v2.4.4, the Adjust Q had a bug that could rarely yield a wrong determined Q, by one - fixed now.
- You may minimize D2S now. If D2S is closed before convertion, D2SRoBa must be closed from icon in tray (right click).
- Two different ways to expand Q search range:
1. If target Q > 48, it automatically expand the search up to Q=112.
It uses up to eight sample test, otherwise as normal (six).
2. You can specify: really_worst_q. Setting that to e.g. 128,
it will then always use (up to) seven sample tests. (2^7=128)
- The smart q search algo (from 2.4.4) skips intermediate sample tests, based on how close test sizes are from their target size.

http://home.no.net/tylo/

@DDogg: A somewhat strange request(?) 100.000 frames is about 65-70 minutes. I guess you can compute the Sample %, in order to get that. Have to think about that formula, though. Is it likely to be a common need?

@r6d2: I think the batch feature should work for you now. Also those syntax messages should be gone.

[telemike]

Stupid question:

I still don't understand how the "Adjust Q" factor works....

Do you adjust the number up or down to make the file size smaller?

Thanks :scared:

[r6d2]

Originally posted by tylo
- The smart q search algo (from 2.4.4) skips intermediate sample tests, based on how close test sizes are from their target size.

Thanks, Tylo! I'll try it ASAP. Is there a threshold parameter on the INI?

@DDogg: A somewhat strange request(?) 100.000 frames is about 65-70 minutes. I guess you can compute the Sample %, in order to get that. Have to think about that formula, though.
Why not just SelectRangeEvery(<TotalFrames>/<SampleSize>,1)?

PD: Time to update the docs then :)

[r6d2]

Originally posted by telemike
I still don't understand how the "Adjust Q" factor works....

Do you adjust the number up or down to make the file size smaller?
Look at the log and you'll get it. Here is an example:

- Adjust Q : (53 + 1.0 - 0.838344334415145 - (0.8)) = 52.3616556655849

The value is substracted from the obtained Q, so it increases file size.

[tylo]

Adjust Q: Adjust the Q.factor down by the (positive) amount given. When the adjusted down, the encoded file increases.

Here is a deeper explaination:
Example: Adjust Q=0.8. D2SRoBa finds initially Q=35.
That means that the target Q (if it was possible to use real numbers) should ideally be between 34 and 35 somewhere. Now, D2SRoBa weights how close the ideal Q is to 35. If the ideal Q is close to 35, the weight is near 1.0, and if it is close to 34, the weight near to 0.0.

It then computes the NewQ = Q + weight - adjustQ.
(Note the weight in 2.4.6 is the (1.0 - 0.xxx) expression in 2.4.4).

If the weight = 0.9 (i.e ideal Q near 35):
NewQ = 35 + 0.9 - 0.8 = 35.1
Finally truncate this number: trunc(35.1) = 35 (unchanged)

If by chance the ideal Q where closer to 34, i.e. weight 0.6:
NewQ = 35 + 0.6 - 0.8 = 34.8 Trunc(34.8) = 34 (one down)

------------------
@r6d2: There is a theshold parameter for the smart binary search (set to 34) but don't change it. I will explain how it works first(sometime). ;)

@DDogg: A movie that has 400000 frames:
Sample % = 100000/400000 * 100 = 25. You'll have to explain why I should bother to add GUI for that. :p

[DDogg]

Tylo and everybody, my face is red as I am starting to realize how much I have contributed to much confusion about the Roba method. As a penance I am planning to do a dummy's guide for the average non technical tool-user, but first I need to continue to concentrate on getting a full and complete understanding.

In order to do that I need to ask Tylo some more questions and make some statements that may seem strong and aggressive. My aim is not to offend, but rather to find out if what I believe is true. That is the only way I can get to where I want to go. Hopefully my questions will cause a more directed dialog. I don't have any problem if I am wrong, as the resulting dialog will be educational for all.

Statements and questions:

1> Tylo, as I think I understand it now, your statement about D2SRoba, "Adds Robshot/Bach one pass VBR encoding to DVD2SVCD" could not be correct as there is no such thing as "RoBa one pass VBR". RoBa by definition must consist of a combination of One fixed Q pass and at least one additional average bitrate VBR run to distribute the bits to achieve practical size targets.

2a> RoBa, by definition, says size is secondary to quality and the first VBR run must use a Q of the approximate quality you desire without any consideration to final size. Again, at this stage of the theory, predicted size is of NO issue. That first Q run is normally in the 32- 46 range.

2b> After this run, from a practical standpoint, one or more standard VBR passes that reuse the VAF file created in the first quality run use a special RoBa criteria to calculate an average bitrate to redistribute the bits in such a way as to refine the end size of the encode so it does not partially fill the number of target media calculated by the RoBa process. The user NEVER actually selects the amount of target media as that decision is inherent in the RoBa process itself.

3> To use a predictive method that alters the Q in such a way as to change the size of the finished encode is directly opposite to the intention of the RoBa method.

4> The D2SRoba plug-in is in fact a totally different size predictive method very similar to that used in the D2S TMPG CQ mode predictive method where quality is altered as a way of achieving a predefined size target.

Tylo, could you please reply and tell me if I have it completely wrong yet again. I really want to get this right this time and I apologize to the group for my thrashing about while trying to understand this complicated subject.

/add: BTW, Thanks for the replies about the framecount question I asked. I'll follow up on it later after I get a better understanding. It is about adding a pre-indicator to the RoBa process somewhat like that used in GKnot. I don't want to get in to that now as it might distract from the questions above.

[telemike]

@DDogg- I think you are on the right path..

It seems that Tylo's plugin is more a TMPG CQ predictor than the Roba thingee doodle.

I originally used Kwag's and other's TMPGenc methods until I discovered that CCE is way faster for SVCD's. Then my search for a predictor that was equivalent to CQ led me to Tylo.

I think the Tylo plugin should be called:

CCE OPV 1-pass file prediction. ;)

Now, if only Tylo's plugin could be fully integrated with DVD2SVCD internally........

[r6d2]

Originally posted by DDogg
Tylo and everybody, my face is red as I am starting to realize how much I have contributed to much confusion about the Roba method.
@DDogg, hope you don't mind me attempting to answer this post addressed more to Tylo, but you said "Tylo and everybody" I consider myself in the everybody category. :)
but rather to find out if what I believe is true.
The Search for the Holy Truth, starred by DDogg as Indiana Jones and Tylo as Dr.Jones, Sr. :)

(Remember Indiana chose his name after his DDogg, since he did not like his father, who named his son Henry after him, to call him "Junior".)
1> Tylo, as I think I understand it now, your statement about D2SRoba, "Adds Robshot/Bach one pass VBR encoding to DVD2SVCD" could not be correct as there is no such thing as "RoBa one pass VBR". RoBa by definition must consist of a combination of One fixed Q pass and at least one additional average bitrate VBR run to distribute the bits to achieve practical size targets.
Yes, by definition it is. However, Tylos's statement is still correct, since what he does is precisely add RoBa support for OPV encoding method in DVD2SVCD.

The primary intention of Tylo was to avoid the second pass, using Bach's idea of a 100% sample to a shorter x% sample, and still do (1 + 6 * x%) pass.

What came after that by popular demand was to support range mode, multipass and other stuff which added the tool a great generality which, IMHO, deserves a name like RoBaTy method.

2a> [...] Again, at this stage of the theory, predicted size is of NO issue. That first Q run is normally in the 32- 46 range.
The Q you set is up to you. Bach recommended something like 30 and 40 for Roba, both magic numbers.

2b> [...]The user NEVER actually selects the amount of target media as that decision is inherent in the RoBa process itself.
You're right. In pure RoBa, size does not matter :)

3> To use a predictive method that alters the Q in such a way as to change the size of the finished encode is directly opposite to the intention of the RoBa method.
The nature of OPV requires you to find a Q in the accepted range defined by [0, Worst Q], in order to fill the CDs to the edge (or at least try).

This approach has the favorable side effect of avoiding border conditions where RoBa may give a sub-optimal encode.

4> The D2SRoba plug-in is in fact a totally different size predictive method very similar to that used in the D2S TMPG CQ mode predictive method where quality is altered as a way of achieving a predefined size target.
With a huge difference to DVD2SVCD's CQ mode: you define the least quality you're satisfied with, and the plugin determines the size. You cannot do that in standard CQ mode.

Hope I'm not putting more hairs into the soup.

[tylo]

To be honest, I didn't have much of a clue how the Robshot method worked when I started to write this plugin. Actually, I only read Bach's posts. I have a fair understanding today, and it seem to match your understanding pretty well. Actually I've learned a few things in this thread too.

I believe most of the statements you made are true, so I will rather give you a general explaination that will hopefully make you all understand why the name D2SRoBa both makes sense and confusion.

-------

RoBa is a specific method, which is a combinations of ideas from Robshot and Bach. This plugin does not try to exactly implement that, but it uses some ideas from both of them:

Robshot: selects a (Worst) Q for the initial pass, which creates the vaf file, and establish the compressablilty of the movie. This is then used for computing the number of CD's required (as you say - inherent in the method).

D2SRoBa: does a similar, but 'compressed' first pass, using only a fraction of the movie as sample. We are only interested in the compressability of the movie, which let us compute the number of CD's required (hence, we dont need the vaf file). This is the only part that is taken from the Robshot method. /Add: This is the NumCDs=auto, Worst Q functionality.

Then, ideas from Bach are used, specifically, the selection of only a fraction of the movie as a 'representative' sample, and the binary search for a Q that creates a file target size that we are looking for.

The essence:

The goal of the thread: 'Modification and optimization of robshot method and its possible use with DVD2SVCD' - was exactly that: a way to cheat, in order to make the RoBa method useful for DVD2SVCD. The original RoBa method takes the long road with: one pass creation of the vaf, then redistribute the bits, then a second pass using the modified vaf information in order to 'achieve practical size targets', as you say.

Instead, we cut the path by just finding a more suitable Q (instead of doing bitallocation redistribution), and then doing only one full VBR pass -the OPV method!

/Add: The first post in the mentioned thread states that multipass is still required. In the CCE thread: 'Question about the "Getting the best out of CCE" guide...' http://forum.doom9.org/showthread.php?s=&threadid=41033 , the one-pass method was suggested by Bach, this was refered to as '1passBach'.


Hope this helps. If you need more detailed answers/comments on your statements, I'll be happy to answer.