D2SRoBa Plugin

[DDogg]

Tylo, I have always wondered why you do not offer an option to do a AvBR VBR pass at the end of your process as a conditional based upon user selected criteria. I know my thoughts could not be original so I assume you have it in your mind.

When CCE ends you know, or could know easily, how big the final video encode is as well as the size of the audio and a close estimate of the muxing overhead, so it follows that you know whether the final product is going to be under/oversized and by how much.

It seems then that offering an option based upon a preset error threshold set by the user would be useful. One additional pass would then make the size exact, and in fact would then be the most efficient hybrid RoBa process as it would only be invoked when needed.

Repeating myself, if the prediction was accurate, the sizing pass would not be invoked. If the size is off by XX%, then the time of the one additional run is much less than a complete redo with all the predictive passes and would be part of the same automated process.

Pardon me if this has been brought up directly before. My eyes are about shot, I am afraid.

[tylo]

Tylo, I have always wondered why you do not offer an option to do a AvBR VBR pass at the end of your process as a conditional based upon user selected criteria. I know my thoughts could not be original so I assume you have it in your mind.
Indeed I had, but it slipped my mind again. I've only thought about this beeing useful when the video gets oversized, but surely it's useful when it undersizes too.

Actually, this idea fits nicely in with r6d2's request for making support for the internal DVD2SVCD RoBa feature. For that, I thought of changing the 'Normal Multipass' mode to 'RoBa Mode', or something.

BTW: Would a name change from '1-pass VBR RoBa' To '1-pass Bach' be more appropriate? If you wish, please elaborate more on how you imagine the GUI for all this (naming and boxes).

Now I just need to find the time to do some programming too! ;)

[DDogg]

Well, I have not thought it out very well at all. Just off the top of my head you could change the normal to "Final Sizing Pass" then add a drop down with "On Oversize Only", "Always","Never", and 1,2,3,4,5,6,7,8,9,10 as a percent. I would probably type in 2.5 if that was allowed.

Nothing wrong with 1-Pass RoBa in my mind. Also, I think it would be cleaner for you to do the extra final pass as you do the others instead of invoking any D2S pass.

[kru]

Thank you for a great program Tylo :D

Here is some suggestions:

-What about encode the audio after the video?
(If there are space left on the CD, the audiobitrate will be greater)

-And a "auto choose bitrate" for the audiobitrate??? (That is about 10 % of the audiobitrate)

-And what about if the videosize is n % less or greater of the calculated filesize, D2SRoBa will encode again, and try too make a better prediction???

[DDogg]

Tylo, some of my index thoughts have started to crystallize while thinking about the feedback I received and while making my reply to Barker here: http://forum.doom9.org/showthread.php?s=&postid=359256#post359256 which shows the bitrate has little to do with the quality of the final encode whereas the compressibility has everything to do with it.

I now think the AI thing I was looking for in your file-size methods could be easily achieved if you would put some special test settings in the INI which basically allowed:

AISKIP_TESTMODE_ON=TRUE
AISKIP 1CD Q-32PassKB > xxxx
AISKIP 2CD Q-32PassKB > xxxxx

Just to be clear, If the Q32 pass was bigger than xxxx then you would instantly negate any more predictive Q passes and increment target media by 1 (meaning 2 becomes the target), if bigger than xxxxx then you would increment target media by 2 (meaning 3 becomes the target). Hmmm, am I getting fuzzy again?

This would save a ton of predictive passes for the file size based methods for the advanced user. Actually, eventually if this pans out, I guess I am asking for an "AI" mode to go along with the AUTO and the "?". For testing purposes, we could just do this in the INI and set the GUI to "1" so as not to cause you a lot of work. What are your thoughts?

Hmmm, the sample size is a slight bit of sand in the oil. I guess people using this method would understand they would always have to use the same sample size for this to work, or unless you think it viable to extrapolate from a 1% in the INI to whatever sample size they used. I would think that would have a bigger error margin though.

[tylo]

I now think the AI thing I was looking for in your file-size methods could be easily achieved if you would put some special test settings in the INI which basically allowed:

AISKIP_TESTMODE_ON=TRUE
AISKIP 1CD Q-32PassKB > xxxx
AISKIP 2CD Q-32PassKB > xxxxx

Just to be clear, If the Q32 pass was bigger than xxxx then you would instantly negate any more predictive Q passes and increment target media by 1 (meaning 2 becomes the target), if bigger than xxxxx then you would increment target media by 2 (meaning 3 becomes the target). Hmmm, am I getting fuzzy again?
No, this is pretty clear to me, and very good idea! :D In fact it's so easy to put in that i'll do that right away!

This would save a ton of predictive passes for the file size based methods for the advanced user. Actually, eventually if this pans out, I guess I am asking for an "AI" mode to go along with the AUTO and the "?". For testing purposes, we could just do this in the INI and set the GUI to "1" so as not to cause you a lot of work. What are your thoughts?I don't see how it can be used for the AUTO mode - only the "?". The AUTO mode computes the num CDs directly from the WorstQ-PassKB size - it doesn't test each num CDs...

Hmmm, the sample size is a slight bit of sand in the oil. I guess people using this method would understand they would always have to use the same sample size for this to work, or unless you think it viable to extrapolate from a 1% in the INI to whatever sample size they used. I would think that would have a bigger error margin though. You are talking about the sample percentage I assume? Yes I think it is viable to just divide the size of the actual percentage used (like normalizing) before testing. It will actually have a lesser error margin (e.g a 2% sample test size / 2 is a more accurate 1% sample pass size, that a true 1% sample pass produces...)

BTW: I am just about to release a new intermediate version now. It has a much improved "?" function already. Adds an extra CD if oversizing before mux and cut (much requested). Also reduces and equals out image sizes if it cannot fill CDs completely.

[r6d2]

Originally posted by tylo
No, this is pretty clear to me, and very good idea! :D In fact it's so easy to put in that i'll do that right away!

@Tylo
It is easier than you think, I guess. It is already implemented (sorta). I PMed you on this on Aug.13. I post it here now for if there is anyone who whises to try.

My other idea is this. You set your desired Q as worst_q. In my case, 30. You set your really_worst_q on 44, in my case.

When conducting a binary search in auto mode, you start by searching with target=30 (30 is the maximum of your binary search).

When in range mode, or in "?" mode, you only search on the [1, 44] range.

With really_worst_q enforced like this, you just have to put max and min to the spec values [300, 2530], and you'll find the Q faster. More important: the Q will always be adequate to your taste and the outcome BR is just a result you'll have to trust even if it is ridiculous! The max and min parameters can still be used, but this approach completely confines them to spec values.

[DDogg]

It is easier than you think, I guess. It is already implemented (sorta). Yes, I see what you mean. Also, if one wanted to accept the programs choice of the number of CDs they would be using the Automode anyway. While I still think the technique I mentioned above has merit, I don't think it belongs in Tylo's plug-in, certainly not in sizing mode!

r6d2, changing the subject slightly, do you agree that the second conditional sizing pass is highly desirable? If so, I hope it is implemented as soon as it can be.

[r6d2]

Originally posted by DDogg
r6d2, changing the subject slightly, do you agree that the second conditional sizing pass is highly desirable?
Absolutely. I had suggested some weeks ago to go for a second OPV pass, conditionally:
Quoting myself :)
You might even improve the method by using the current binary search for a first OPV pass, and then correct with the real Q for the whole movie and obtained BR, using a simple proportion.

Then you do a second OPV with much more precision, and filling the CDs to the edge or getting really, really close (it would be a 100% sample, so to speak ).
But probably it would be better to stick Multipass, as you suggest. It is 0% risk of odd-sizing and with comparable quality if you reuse the VAF.

[DDogg]

Yeah, I remember trying to figure out what you meant when you posted that, but stopped when my eyes started crossing :p

Like you said, I think the standard BR based pass for the second pass is really the easiest way to get where we want to go.

[tylo]

Ok, I actually started to implement the AISKIP thing, but I suddenly realized that I was wasting my time. (I should have learned by now not to jump on things too fast). Setting such limits translates to the same thing as setting a really_worst_q ultimately - just as r6d2 saw.

I must confess I didn't take the time I should have to understand r6d2 suggestions. The Aug.13. post is the sensible way to implement the "?" functionality. As you (r6d2) suggested in another post (I think), we can start a test with the really_worst_q, and use the size found as sort of AISKIP limits. This is basically what is done with worst_q in Auto mode, but here we go for the optimal (realistic) number of CDs directly, giving no user choices.

- On the other subject, I was thinking of supporting an optional second-pass (multipass), reusing the vaf file, as you mentioned.
- Then support for the internal RoBa method (but finding a good initial Q by binary search).
- Third. If I'm not fed up with the plugin, I may put audio encoding after the video encoding. We can adjust the audio bitrate in various ways, depending on the size of the video created.

New subject: I am thinking about removing the 'Bitrate Tweaking' fuctionality. It doesn't really belong here, and I don't think many is using it. As I have said before, cutting credits with frame selection (or chapters) is often a better alternative. Couldn't the tweaking also be done by an Avisynth script? E.g: Add a filter that make the bitrate drop after a certain number of frames?
I need feedback on this one.

[DDogg]

- On the other subject, I was thinking of supporting an optional second-pass (multipass), reusing the vaf file, as you mentioned. That adds a lot of bang for the buck as, I think, it would only take a small amount of effort? With the conditional, it turns the plug-in into a true RoBa process, but one that is hybridized as the pass is only done if needed.- Then support for the internal RoBa method (but finding a good initial Q by binary search). I think this calls for some thought and discussion. First, I would suggest we forget the plug-in for a moment and concentrate discussion on any weaknesses of the Internal RoBa process of D2S. In other words, is it broken and do you need to fix it? If it is broken in any way, who can best fix it, D2S or an external Plug-in?

Isn't it true that the only weakness of the D2S internal process is if the equivalent bitrate of the initial Q for the OPV pass is over 50% off from the bitrate actually needed? Yes, I am not very clear on that. If that is so, I would think if D2S did just one short measurement pass that would no longer be an issue. [edit] Here is the quote from Bach on that subject-- Decision: Calculate video average bitrate (CAVG) necessary in order to fit 2 CDs.
c) do multipass reusing vaf from first pass and using CAVG.
WEAKNESS OF ROBA: if CAVG > 2*(1st_pass_mpv_size/time_length) or CAVG < 0.5*(1st_pass_mpv_size/time_length), you MUST recreate the vaf file in order to do the multipass encoding. [Add:] This brings up a question. For discussions sake, let's say an encode is correctly sized with a OPV Q of 48. Let's also say we have set the first Q pass in D2S internal RoBA to 32. The second bitrate based pass will size it correctly for us and the bitrate will be equivalent to the Q 48 pass. The question then is: would we gain anything if we had used the Q 48 as the Q of the first pass in the internal D2S RoBA process. My understanding is we would not, and in fact may lose quality although it may be the same. Anybody got any comments on this?

- Third. If I'm not fed up with the plug-in, I may put audio encoding after the video encoding. We can adjust the audio bitrate in various ways, depending on the size of the video created. No comment except to suggest you may be opening a can of worms for yourself and you might end up losing all your hair (assuming you have any :)).
I am thinking about removing the 'Bitrate Tweaking' functionality. Maybe some folks use it. With the D2S frame selection so easy to use, I certainly don't and can't imagine doing so. I do understand your inferred suggestion that some users might not like to cut any of the credits and might prefer to just use a lower bitrate on them. Sounds good, but would it really be worth the effort and would anybody actually use it? Dunno.

What the bitrate tweak section does do is use up a lot of valuable "real estate" on your interface. That space might be better used to more fully differentiate between the two radically different methods available in your plug-in (Quality based and Sizing based). I know that caused me a lot of confusion and I am sure many others.

[r6d2]

Originally posted by DDogg
This brings up a question. For discussions sake, let's say an encode is correctly sized with a OPV Q of 48. Let's also say we have set the first Q pass in D2S internal RoBA to 32. The second bitrate based pass will size it correctly for us and the bitrate will be equivalent to the Q 48 pass. The question then is: would we gain anything if we had used the Q 48 as the Q of the first pass in the internal D2S RoBA process. My understanding is we would not, and in fact may lose quality although it may be the same. Anybody got any comments on this?
What OPV does, attempting to quote a Bach post which unfortunately was not recovered, is this:

1. Encode a frame with desired Q.
2. If resulting BR > MAX, forget Q and BR and adjust to MAX.
3. If resulting BR < MIN, forget Q and BR and adjust to MIN.
4. Repeat from 1.

I tend to think from the above that using the correct Q, or a nearly correct Q from the start will give you a better encoding, since there would be less "forget Q" frames.

Again, I may be terribly wrong. But I invite anyone to rip apart this logic to continue for the search of the Holy Truth :)

What the bitrate tweak section does do is use up a lot of valuable "real estate" on your interface. That space might be better used [...]
@Tylo, I think (IMHO) the credits stuff adds generality to the tool. If you are concerned about keeping the GUI clean, you might want to implement an "Advanced" button, or even better: follow D2S's equivalent parameter.

Don't worry about using a bigger window either to accomodate for more optios and modes. DDogg and I will give the users, sooner than later, a good guide for no brainers, whose only requisite is being able to understand what you read. ;)

Besides, even though frame selection does the job, it requires you to wait for DVD2AVI to finish, and that is not ideal for multiple batch encoding.

[DDogg]

I tend to think from the above that using the correct Q, or a nearly correct Q from the start will give you a better encoding, since there would be less "forget Q" frames. This then becomes the pregnant question that we have to get the right answer to. I sent off a PM. Maybe I can get an answer and/or opinion.Don't worry about using a bigger window either to accomodate for more optios and modes. DDogg and I will give the users, sooner than later, a good guide for no brainers, whose only requisite is being able to understand what you read. Disagree :). Docs have to be read. An interface should be intuitive and not require reading to understand basic functionality, in this case that is defined as two completely different methods combined in such a way as to not be clearly differentiated.

[r6d2]

Originally posted by DDogg
[B]This then becomes the pregnant question that we have to get the right answer to. I sent off a PM. Maybe I can get an answer and/or opinion.
Send my warmest regards to God when you see him ;)

Docs have to be read. An interface should be intuitive and not require reading to understand basic functionality, in this case that is defined as two completely different methods combined in such a way as to not be clearly differentiated.
Disagree :). For instance, D2S interface is not intuitive. You must read a guide to learn to tweak the tool. On the other side, D2S (him) managed to set it up in such a way that if you know nothing, dislike to read the docs and even have a hard time with readmes, the default values will do a great job anyway.

Software Engineering must balance usability with powerness. The KISS concept is crucial for starters, but you must also take into account power users and the evolution of your tool. An approach with Advanced mode seems to me the best deal, just like D2S (him) chose to do.

[DDogg]

Software Engineering must balance usability with powerness. The KISS concept is crucial for starters, but you must also take into account power users and the evolution of your tool. An approach with Advanced mode seems to me the best deal, just like D2S (him) chose to do. I don't disagree with that at all and the advanced tab is a good option if Tylo feels like messing with it. However, you said it yourself. "...Kiss is crucial for starters". When you have two completely opposite methods (Quality based and Sizing based) mixed together "without notice", so to speak, you do not even remotely adhere to the KISS GUI system that basically says 60-75% of the functionality should be readily apparent, usable and understandable within 5-8 minutes of first use. Anyway, Tylo can take the suggestions or leave them, so let's not beat it to death and get too off topic :)

[r6d2]

Originally posted by DDogg
When you have two completely opposite methods (Quality based and Sizing based) mixed together "without notice"
I agree with you here on the different methods stuff, they are for different audiences. I tend to think that starters are quality oriented (you are just too affraid to screw up your encoding). Then, when you get more confident, you get also mode daredevil and begin to be cheap with CDs.

If Tylo would like to take this approach, I'd default to quality based mode in the eventual new GUI.

[tylo]

I'll respond later. Here's a new beta release.
http://home.no.net/tylo/setup_D2SRoBa248beta.exe

/Edited:
Nothing new in the GUI, or second-pass stuff yet, but:

- creates extra image for the oversized part (often only credits)
- if bitrate > max: equals out images sizes, shows fillrate.
- "?" mode: uses 'really_worst_q' to estimate lowest numCDs to test.
- auto mode: searches only within 0 .. Worst Q
- removed N-M ranges (replaced by "?")
- removed special high/low bitrates limits for "?" => uses D2S min/max
- default really_worst_q=128 now (maybe too high)

Not well tested, so beta - please test.

[DDogg]

r6d2, here is the answer from god (slightly edited) :)

The quote [from r6d2] is correct. I have tested your hypothesis at the time I formulated the nowadays so called 1pass-Bach method. My conclusion was that the quality of the encoding increases when I do the first pass of (internal) RoBa method using the Q value closer to the correct one. The difference is not high, though. It is most noticeable when you have a lengthy still scene.
The reason? Well, I think only the CCE coders themselves could explain with 100% of sureness (is there such word in english?), but r6d2's guess (the "less forget Q" thing) is a good guess.

Anyway, CCE's manual gives us a hint: "if you change the average bitrate of a given encode by twice or half of its previous value, it is mandatory to create a new .vaf file".

In your example, lets suppose the calculated bitrate for Q=32 is 1.5 times bigger than the target bitrate (the one which you achieve with Q=48). You are not in the region which CCE's claims you should discard the vaf file. However, since there is a previous_bitrate/target_bitrate ratio (<0.5 or >2.0) for which the already created .vaf is simply useless, this means that the farther you get from the previous_bitrate/target_bitrate ratio=1 , the worse the re-usage of your .vaf is. [edited sentence structure - hope I got it right]

This most probably is not a linear correlation, so that if ratio is something between, say, 0.9(?) and 1.1(?) it is not possible to realize any loss.

[r6d2]

@Tylo,

Your safe zone stuff made me start thinking about the application of Newton's method to find roots for an equation and its possible use to D2Sroba finding of Q. Here you are.

We have bitrate as a function of Q, BR(Q), a function which is unknown to us, but we suspect it to be linear in some range and curve in others. What is clear is that it is a continuos function and with a negative slope.

Let's define

f(Q) = BR(Q) - Target_BR

Our goal is to find Q for which f(Q) = 0. Let's define x = Q.

Newton's method states that you can find a root by iterations with the following formula:

x(n+1) = x(n) - f(x(n)) / f'(x(n))

where f'() is the derivative of f(). Since we don't know f(), we cannot calculate f'(). However, f'() can be aproximated by:

f(x(n)) - f(x(n-1))
f'(x(n)) = -------------------
x(n) - x(n-1)

If we replace the above formula in Newton's, we have that:

x(n) - x(n-1)
x(n+1) = x(n) - ------------------- * f(x(n))
f(x(n)) - f(x(n-1))

This method requires you to do 2 blind searches before iterating (for instance, n=1 with Worst_Q and n=2 with Really_Worst_Q). Then you stop when the error, calculated as |f(x(n))| < E, is adequate. E can be set to 1% or any value you choose.

It converges very fast. I did some tests using as seeds x(1)=Worst_Q and x(2)=Really_Worst_Q, and the results are incredible. I found the target Q in 4 tries, no matter what seeds I fed in.

I hope this is not too technical, you might want to try it yourself and hopefully implement it instead of binary search. If you have questions please don't hesitate to ask.

[tylo]

Thanks, r6d2.
Yes, the Newton-Raphson method would probably work better than the current (optimized) binary search, but not much - maybe one test less on average.

For the fun of it, and that we probably find Q reliable after (hopefully) 4 iterations, I'll implement it when there is time.

Ps: I knew the Newton-Raphson method, but somehow forgot how great that method is. Here's a fine link to it:
http://www.sosmath.com/calculus/diff/der07/der07.html

/Add: The error from the root (zero) to be found in the N-R method, is messured based on the f'(x). But when f'(x) is only approximated, the error may also be wrong. When you hit Q with very few iterations, it may actually be the wrong Q, if your unlucky...

[r6d2]

Originally posted by tylo
Yes, the Newton-Raphson method would probably work better than the current (optimized) binary search, but not much - maybe one test less on average.
Actually, is converges in 3 to 4 tries, and it does so regardless of Really_Worst_Q. Those guys on the 2 digit range will really appreciate it, but it would be better to everyone.
/Add: The error from the root (zero) to be found in the N-R method, is messured based on the f'(x).
The error E I'm talking about is the absolute value (modulus) of f(x(n)) (remember f() is the delta from the resulting BR and the target BR)), divided by target BR, hence the percentage.

There is no error this way or possibility of being unlucky if the slope of the function does not change sign, which in our case it does not.

Also, for Internal RoBa, 3 tries will always produce an adequate Q, if you plan to support it.

[telemike]

Tylo-

can you make a version of your plugin that does this:

1. Calculate the number of CD's based on a worst user specified "Q" like you do now by stopping DVD2SVCD.

2. Push the "Q" value calcualted and OPV bitrates back into DVD2SVCD.ini and DVD2SVCD.d2s files.

3. Restart DVD2SVCD from recover, DVD2SVCD then uses the new values that were written into the files.

Why?

Well, it might be an easier solution for some of us.

[tylo]

@Telemike!
AFAIK, this is what the D2SRoBa plugin does now. Please, be more accurate.

By 'OPV bitrate', I guess you mean the target bitrate for that Q. Do you want to do a final OPV or a multipass VBR encoding?

I'm only guessing here: Do you want that the plugin should restart D2S from the 'Video Encoding' phase in order to do the multipass VBR, rather than do the video encoding itself (and start D2S from the 'Mux/Cut' phase)? The only problem with the current way is that my own size estimation isnt as accurate as D2S's. However, I may change to the described approach. (way earlier versions worked that way).

[telemike]

Here's what I mean-

1. DVD2SVCD does it's thing until video encoding.
2. d2sroba calculates the Q
3. Instead of calling up CCE from d2sroba, push the values back into dvd2svcd's dvd2svcd.ini an dvd2svcd.d2s files, re-start dvd2svcd from the encoding phase with the new calculated values.


I do all mine as 1-pass VBR OPV. I think it might be "cleaner" to have dvd2svcd handle CCE.

What would be REALLY NICE is to have the programmer DVD2SVCD integrate TYLO's plugin into DVD2SVCD as an internal routine.

[kru]

Sometimes D2SRoBa seems to work to fast when addeng the first AVS file too CCE and save the .ecl file.

Encoding takes several hours, why not add a couple of seconds when D2SRoBa loads / save files to make the program safer?

What about:
; Add modified avs file...
RightClick, 100, 130
sleep, 2000
Send, a%g_robapath%.avs{ENTER}
Sleep, 2000

or:
; Add modified avs file...
RightClick, 100, 130
WinWaitNotActive, %w_title%
sleep, 500
Send, a%g_robapath%.avs{ENTER}
WinWaitActive, %w_title%
Sleep, 2000

:sly:

[r6d2]

Originally posted by telemike
I think it might be "cleaner" to have dvd2svcd handle CCE.
@Telemike,

Your suggestion works only in D2S's Multipass, since D2S's OPV sets Max=Predicted. Tylo would have to fix this too, maybe it is possible (he has already in his own OPV passes).

However, I also sustain the theory of the plugin to be as much compliant as possible with D2S to be compatible with future enhacements. In fact, Tylo and I have exchanged some PMs and posts on the matter but I think it has become a religious topic. ;)

Anyway, a possible solution to D2Sroba's imprecise calculation of target bitrate (error which systematically affects all calculations thereafter), might be the following:
Let D2S start, as currently does. It will calculate BR and number of CDs taking everything into account: multiple audio tracks, subs, muxing parameters (scan offsets, etc.), frame selection, frame rate... Everything. This data is not as precise as FitCD, but much more precise than the plugin's bbmpeg_overhead=1.0235 and bbmpeg_cutoff=380000 parameters.
Let D2S start CCE.
Abort CCE.
Get the target BR and number of CDs D2S calculates. Even though this data is based only in movie duration (disregarding compressibility), you can use it to extrapolate the target BR for any number of CDs like this:

Available_BR_per_CD = D2S_target_BR / D2S_number_of_CDs
Then, in D2Sroba, we have that:
Target_BR_for_CDs(n) = Available_BR_per_CD * n
Have D2Sroba calculate all its stuff with these values.

It is just me or this stuff makes sense and would save a lot of hassle?

[telemike]

Get the target BR and number of CDs D2S calculates. Even though this data is based only in movie duration (disregarding compressibility), you can use it to extrapolate the target BR for any number of CDs like this:

Yes! Exactly!

Have the plugin grab the number of cd's from dvd2svcd!

I have had some problems when it comes to muxing and the images don't split correctly. For example, the first images cuts at 795mb but the second runs to 1120MB.......It should cut at 795 and make 3 images...
:D

[DDogg]

hmm, before poor old tylo gets all that thrust upon him I would suggest waiting for a few days (defined as "real soon now" :) ). A beta release candidate is forthcoming and will have the OPV BR changed back to Min/Max bitrate as it is in the internal RoBa process.

The next release after that should have a lot of this stuff incorporated internally. Certainly as it pertains to correctly identifying the close approximate Q for the RoBa OPV pass and perhaps including a sizing mode as is presently done for TMPG CQ mode. The Newton's third law of pecker measurement from r6d2 may come in handy for that phase.

Again, IMO, the best intermediate measure is adding the conditional sizing pass into D2SRoBA.

[/Add] Presently I use D2SRoBa 1% passes to give me the approximate correct Q. Then I restart D2S and use the Internal RoBa process with just 1 pass (set RoBa Visibel=1 in dvd2svcd.INI to turn it on, and yes that spelling is what he uses. The new choice will then become available under CCE in the encoder tab).

This gives me the best of both worlds because I don't mind the additional pass which Tylo has worked so hard to save us from.

[greenphantom]

Originally posted by tylo
I'll respond later. Here's a new beta release.
http://home.no.net/tylo/setup_D2SRoBa248beta.exe

- default really_worst_q=128 now (maybe too high)

Not well tested, so beta - please test.

Glad to see the new one allows a higher Q ;) I didn't realize the previous version had a max. of 112 and I ran into a problem during my first test (see below). I'm going to give the beta version a try now.

I just ran my first test with the D2SRoBa 2.4.6 plug-in yesterday and the results were a bit puzzling. It produced a very high Q (112) and a file that was about 15% too big to fit on one cd, yet the quality was absolutely fantastic despite the high Q.

I tried backing up CD 1 from a History Channel WWII documentary I have on DVD and it's 98 min. long. I've seen some surprisingly good 1-cd svcd encodes lately so I thought I'd give it a try myself. Here's what I used:

* D3SRoBa 2.4.6
* CCE 2.67.00.11 with EclCCE 1.6 & AviSynth 2.5.2
* DVD2SVCD ver. 1.1.3 build 2
* MY_SemiAutoCropAndResize_MA script (2.5.2)

With a 128kbps audio encode, I used FitCD and found that the video bitrate should be around 960kbps. I then ran the plug-in and ended up with a video file that was 845MB with a Q of 112. I was surprised that even though the Q was so high, the quality was FANTASTIC! I cannot stress this enough! I've watched hundreds of svcd's and the quality of this one was virtually indistingishable from the original dvd. However, I should mention that it didn't have a lot of motion; as it was a documentary, there were a lot of still images on the screen with background narration along with interviews etc. and the occasional clip of a battle scene.

Anyway, because the maximum Q was reached, the plug-in didn't seem able to find a bitrate low enough so the file would fit on one CD. Perhaps I didn't do something right (entirely possible, since it was my first try!) but I'm hoping it'll work this time with the beta version. I just thought I'd post the results of my first experiment since (a) it demonstrates the need for the higher maximum Q, and (b) it shows that, at least for certain types of footage, excellent quality results can be obtained despite a Q that's over 100.

P.S. Thanks to Fastchef & Homerjay for helping me get this far :)

[EDIT - Sorry for the newbie question, but do I need to set the avg. bitrate in DVD2SVCD when using the plug-in? I assumed it would find both a recommended Q and avg. bitrate to use, but I just noticed in the log below that it made a note of the bitrate that was set in DVD2SVCD ("Bitrate: 1000") - I would have filled in a lower value here (the one recommended by FitCD, around 960kbps or so) but didn't bother since I thought the plug-in would take care of it. That must be why I ended up with a file that was slightly too big... Doh! :rolleyes: ]

========================================================
D2SRoBa v2.4.6 - DVD2SVCD plugin, by tylo
OS Version: WIN_XP
--------------------------------------------------------
- DVD2SVCD Shutdown
- 2003-08-23 21:12:02
--------------------------------------------------------
Settings:
- CD size : 800
- Number of CD's : 1
- Worst Q. factor : ---
+ One pass VBR Robshot/Bach : 1
- Sample percentage : 2.5
- Sample GOPs : 1
- Adjust Q. : 0.8
- Safety Zone : 34
+ Do credits bitrate tweak : 0
- Start time : 0:00:00
- Bitrate : 1000
--------------------------------------------------------
- Movie length : 01:38:15 (176700 frames, 29.97 fps)
- Audio size est : 94334335. bytes (128 + 0 kbps)
- CD user data size : 834286496. = (800 - 4)*1024*1024 - 380000
- SelectRangeEvery : every 600, select 15 frames
- Num. sample frames : 4425
--------------------------------------------------------
Calculations:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Number of CDs : 1
- Target mpv size : 720796585. (978 kbps)
--------------------------------------------------------
- Binary search for Q:
- Estimated mpv size : 1575832244. (Q=32, 2138 kbps, 39462692 sample sz)
- Estimated mpv size : 1394226014. (Q=48, 1891 kbps, 34914828 sample sz)
- Estimated mpv size : 1112274309. (Q=80, 1509 kbps, 27854068 sample sz)
- Estimated mpv size : 1003548982. (Q=96, 1361 kbps, 25131320 sample sz)
- Estimated mpv size : 953061899. (Q=104, 1293 kbps, 23867000 sample sz)
- Estimated mpv size : 929311981. (Q=108, 1260 kbps, 23272244 sample sz)
- Estimated mpv size : 918136715. (Q=110, 1245 kbps, 22992388 sample sz)
- Estimated mpv size : 912743791. (Q=111, 1238 kbps, 22857336 sample sz)
--------------------------------------------------------
- Determined Q. : 112
--------------------------------------------------------
Replace values in ecl file:
- opv_q_factor : 112
- opv_brate_max : 967 ==> 2530
- vbr_brate_avg : 6000 ==> 978
--------------------------------------------------------
- Encoding Movie
- 2003-08-23 22:27:52
--------------------------------------------------------:rolleyes:

[r6d2]

Originally posted by greenphantom
I just ran my first test with the D2SRoBa 2.4.6 plug-in yesterday and the results were a bit puzzling. It produced a very high Q (112) and a file that was about 15% too big to fit on one cd, yet the quality was absolutely fantastic despite the high Q.
Hi, @greenphantom,

My bet is this happened because the Q for that encode is indeed higher than 112. The plugin, in its current form, will let you encode up to 300 if you wish. Try setting the Really_Worst_Q parameter on the ini file to say, 150, and it will find a Q that will fit better.

Though I cannot see how a Q higher than 60 may end up being terrific, the plugin will do its job.

Regards,

[r6d2]

Originally posted by DDogg
Certainly as it pertains to correctly identifying the close approximate Q for the RoBa OPV pass and perhaps including a sizing mode as is presently done for TMPG CQ mode. The Newton's third law of pecker measurement from r6d2 may come in handy for that phase.
[/Add] Presently I use D2SRoBa 1% passes to give me the approximate correct Q. Then I restart D2S and use the Internal RoBa process with just 1 pass (set RoBa Visibel=1 in dvd2svcd.INI to turn it on, and yes that spelling is what he uses. The new choice will then become available under CCE in the encoder tab).
For those of you willing to play with other convergence methods, I've posted a simulator in:

http://forum.doom9.org/showthread.php?s=&threadid=60191

[tylo]

New beta out: v2.5.0 beta1

http://home.no.net/tylo/
Go get it!

Aside from support for internal DVD2SVCD RoBa encoding, this version should have some significant advances in that it takes all the estimated available bitrate/space values from DVD2SVCD. E.g. the bitrate tweaking should use exactly the same bitrate as a regular multipass run would have given.
Many other fixes have been made too. :D

/Add: A small downside of this version: You may not open CCE manually (e.g. to configure it), when D2SRoBa is running (in "wait mode"). :p

/Add: When using the 'X-pass RoBa' (with CCE 2.66+), remember to modify RB's progressive template, such that it uses existing vaf. (edit: comment deleted).

[homerjay]

will do tylo sir - thanks

notice 2.4.8 didnt come up with an error when i had left misc on avi
thought my luck was in :rolleyes: warning is back now in 2.50 b1:(

[r6d2]

Originally posted by tylo
New beta out: v2.5.0 beta1
Thanks, Tylo! Are you supporting equally sized images yet? If so, we need to update the docs again ;)
/Add: When using the 'X-pass RoBa' (with CCE 2.66+), remember to modify RB's progressive template, such that it uses existing vaf. (it set it to 'Create new' by RB).
I think you must do the opposite, set it to "Use existing" :confused:

[DDogg]

Wonderful news, Tylo! I was worried we had driven you nuts and you had jumped off a cliff or something :) I see now you have had your head down coding.

A couple of things: I typlically have edit script on. I hope you might be able to flip the variable after the initial edit and then potentially change it back after D2S is restarted in RoBa? If not, it requires three edits and mistakes can be made easily.

Also, I noted one must have RoBa preselected in D2S before stating your process? That's fine so long as people know it. I got the impression you were flipping that yourself. Maybe because I am using the new RC and there is several different INI files instead of just the one SVCD.ini.

/ADD: maybe provide a reg file with the proper RoBa Template? As r6d2 said, the template must be set to use existing VAF, unless you are doing something I don't understand.

/ADD2: Creating templates. So simple it is confusing as hell. Select the standard template so that it is checkmarked, now just select it again and the template manager will come up. Change the name and the settings, then select "add". You will now have a new template available in CCE and D2S. If you want to modify the template, click twice on it in the dropdown, then make changes and select "replace"

[tylo]

Thanks, Tylo! Are you supporting equally sized images yet? If so, we need to update the docs again
Yes, equally sized images are supported - and yes it must be added to doc.
Wonderful news, Tylo! I was worried we had driven you nuts and you had jumped off a cliff or something I see now you have had your head down coding.Ah, well, I got the flu real bad, so I was pretty delirious. Perfect time to do a bit of coding. I feel better now. :) The plugin should set the RoBa mode in D2S - I guess the RC has changed the key or ini-file name.
I have also thought about just installing the template into CCE, and let D2S select it, but it's maybe not a task for this plugin. Good documentation would do.
I quote: remember to modify RB's progressive template, such that it uses existing vaf - (?) maybe bad english, but it means that the template should use the existing vaf. :sly:
As usual, please report long errors by PM, others here.
Cheers.

[r6d2]

Originally posted by tylo
Ah, well, I got the flu real bad, so I was pretty delirious. Perfect time to do a bit of coding. I feel better now. :)
If you had used Newton's method to beat the flu it would have taken you half the time! :D:D:D
The plugin should set the RoBa mode in D2S - I guess the RC has changed the key or ini-file name.
It works for me. As you say, the RC must be the cause.
remember to modify RB's progressive template, such that it uses existing vaf - (?) maybe bad english, but it means that the template should use the existing vaf. :sly:
You're right. The parenthesis stuff confused me.

I see you removed my GOPs variable :( Well, my research was not getting anywhere on sampling better anyway

On another topic, just a matter of taste, but I would not have left out pure multipass. Remember internal RoBa is "not for everyone" yet, as is unsupported by DVD2SVCD.

[tylo]

If you had used Newton's method to beat the flu it would have taken you half the time!Hehe, yeah it might have. :p

- You may still change the GOPs in the ini file.
- Yes, pure multipass should probably still be there. Then the plugin just add a better way to select number of CDs (also automatic). Next time.
DDogg:I typlically have edit script on. I hope you might be able to flip the variable after the initial edit and then potentially change it back after D2S is restarted in RoBa? If not, it requires three edits and mistakes can be made easily. Hmm. please elaborate.

- About the multipass D2S RoBa: Haven't tested much, but D2S seems to increase the bitrate in the second pass, so it oversizes. Anyone else seen this?

[DDogg]

Well the new release is finally out so you can check the new INI structure. Sticky in regular forum.

/Add:
Re the edit script, I normally keep "edit as part of video encoding" checked in the frameserver tab. I typically use this when doing a lot of restarts testing new setups. It was not a problem before, but now, there is an initial close and restart which brings back up the edit dialog, then after you find the Q, another restart of D2S brings it up again.

I was hoping you could flip that setting in the ini before you kill d2s the first time, then once the D2S internal RoBa has been started up for the full encode, flip it back in the ini so it will be in the same state as it was before using the plugin.

Hmmm. or will that just cause D2S to create a new script from the template? I guess I could just leave it as "edit after dvd2avi is finished". Then it would use it without the edit screen popping up. That makes it harder to do on restarts, though.

[r6d2]

Originally posted by tylo
- Yes, pure multipass should probably still be there. Then the plugin just add a better way to select number of CDs (also automatic). Next time.
Perhaps removing the ranges was not a good idea. I'm trying The Two Towers now and the plugin keeps on looking up to 4 CDs, which makes no sense. You know from experience that 1-2 and 2-3 are likely useful.

You know what? You might implement "?" mode to try only n and n+1 (where n is the suggested size based on Maximum Q). That would do, for those without "experience" ;) (That way you can leave the ranges out, which might probably confuse newcomers anyway if reinstated.)

[r6d2]

Originally posted by tylo
- About the multipass D2S RoBa: Haven't tested much, but D2S seems to increase the bitrate in the second pass, so it oversizes. Anyone else seen this?
The cause of this, believe it or not, is the Adjust Q and the imprecision of sampling.

You know that the sample may underestimate. When D2S encodes, as it is not aware of your plugin, it thinks the Q.factor it receives is "the exact one". Then, based on the resulting MPV size of first OPV pass, it recalculates the number of CDs. If you're not lucky, he will think more is needed (by a bunch of bytes, remember that?).

When (and if) D2S supports a Q search internally, it will know the number of CDs must not be reacommodated.

A workaround on the meantime is to increase Q a bit manually (it sucks), or by setting Adjust Q to a negative value (it also sucks, but should work), I don't know if your plugin currently accepts a negative value here.

The other option is D2Sroba to do the second RoBa pass.

[telemike]

Excuse my stupidty-

But how do I download the new beta?:confused: :confused:

/add

DOH! my stupid browser cache!!!!!!!!!!!!! :rolleyes:

[telemike]

For the new beta version, what settings do you reccomend for the bitrate tab?

[tylo]

For the new beta version, what settings do you reccomend for the bitrate tab?You mean for the minute - sizes stuff? For the 1-pass RoBa it is not used, so it doesn't matter. For X-pass D2S RoBa, currently I override the sizes to ensure that bitrates for each num always are below the SVCD standard. I don't want D2S to truncate the bitrates to MAX (2530). Not sure if I need to do that, though? - So, the settings don't matter.
You know what? You might implement "?" mode to try only n and n+1 (where n is the suggested size based on Maximum Q). That would do, for those without "experience"Seems smart. If you choose a high Maximum Q, it is for a reason: You want the number of CDs for that Q, or maybe one more - but not yet another. Easy fix.
- A workaround on the meantime is to increase Q a bit manually (it sucks), or by setting Adjust Q to a negative value (it also sucks, but should work), I don't know if your plugin currently accepts a negative value here.
- The other option is D2Sroba to do the second RoBa passSo what you're saying is that the D2S's algorithm must use a Q that gives a bitrate just below the target bitrate (or a few higher - less optimal), but if you give a lower initial Q, it will fail? - In that case, both of your options are viable. Hm.
BTW: the plugin should accept negative adjust Q.

[DDogg]

So what you're saying is that the D2S's algorithm must use a Q that gives a bitrate just below the target bitrate (or a few higher - less optimal), but if you give a lower initial Q, it will fail? - In that case, both of your options are viable. Hm.r6d2, are you sure you are not full of beans? (I'm grinning when I say that). I have not had a problem. Also, would we not need to know what D2S is doing (algo) to know? Maybe he will post here and tell us.

[r6d2]

Originally posted by tylo
So what you're saying is that the D2S's algorithm must use a Q that gives a bitrate just below the target bitrate (or a few higher - less optimal), but if you give a lower initial Q, it will fail? - In that case, both of your options are viable. Hm.
Actually, any Q giving within +/-10% of the target BR is good enough for RoBa (higher imprecision would require more than one additional pass for an optimal encode).

The problem is not D2S "fails" since it does exactly what it is supposed to. The problem is that it doesn't know the number of CDs is already correct.

The negative Adjust Q will fix this "unawareness" for the case of underestimation.

[DDogg]

OK, I see what you are saying. Maybe tylo could just adjust internally slightly if RoBa is chosen? It would be a drag if the user has to do it as that would change the settings for the other methods and they would have to remember to set them back.

[r6d2]

Originally posted by DDogg
[B]r6d2, are you sure you are not full of beans? (I'm grinning when I say that).
Just checked. Only one bean and I did not eat it ;)
I have not had a problem.
Me neither, not speciffically this one. Underestimation occurs in about 1 out of 4 cases. I guess Tylo got the one.
Also, would we not need to know what D2S is doing (algo) to know? Maybe he will post here and tell us.
Of course, that would be great. My theory (I should have stated it was that, until proved ;)) is just the result of fuzzy thinking mode. Trying to imagine how the clock works without opening it.

[/Add]I have had another problem though. I received the following message after first RoBa pass: "Sorry, only one instance of CCE allowed".
Maybe CCE is not closing rapidly enough when D2S starts the second pass.

[DDogg]

arghh, our posts crossed again.