Call me dense, but what is the point of "The RoBa method for CCE encoding"?

[Matthew]

I'm referring to the dvd encoding guide as posted on doom9.org ;)

From my reading of it, the process is exactly the same as performing a standard X pass VBR encode, except instead of letting CCE create the vaf as normal, it is made using the 1 pass VBR settings.

Is it just me or does this seem a little odd? It's as if the method is based on the idea that Custom Technology has botched the vaf creation routines for a standard X pass VBR.

I understand the idea of running a 1 pass VBR and using that to decide upon the bitrate that should be used (good for deciding no. of CDs to use when making SVCDs, for example), however the guide I am referring to is about encodes where the bitrate is fixed (based on DVD-R size).

So the question is, why is it better to run 1 pass VBR automatically followed by 4 passes, rather than simply run a standard 4 pass encode? :confused:

[RB]

You are right, it makes absolutely no sense to follow the "Getting the best out of CCE (aka Robshot)" guide if your average bitrate is fixed, that is, you are targetting a specific number of media like 1 DVD-R or 2 CD-R. Manually tweaking the local bitrates in the GOPs only makes sense if you can play with the global average bitrate as well, say if you wanted to fit the video on a minimum number of CDs.

Also see http://forum.doom9.org/showthread.php?s=&threadid=58825

[Matthew]

Thanks for the reply :)

I was actually referring to the second CCE guide http://www.doom9.org/mpg/cce-roba.htm

Which doesn't involve any tweaking but still involves running a 1 pass VBR to create the vaf.

But your advice just to run a standard X pass VBR still answers my question anyway :)

I'm just a little perplexed how this method ended up as a guide on doom9 and template in DoCCE4U, etc.

[DDogg]

So the question is, why is it better to run 1 pass VBR automatically followed by 4 passes I personally don't think you need anything more than the 1 OPV pass and 1 BR based Vbr pass. Remember, RoBa, superseded RobShot (some would argue that) and always was and is a constant quality based method, it is not a size based method and thinking size is where the confusion starts to creep in. Also, RoBa, unlike RobShot, requires no hand tweaking. All of that is handled in the OPV.

Originally, as I understand it, the purpose was to create a constant quality model (Using your preferred Q), then calculate the number of disks that would be needed to maintain that quality from the size of the first pass. The second pass was used only to make sure those disks were not over or under filled.

Technically (I think this is right), according to Bach, the BR of the second pass should be no more than 10-15% off what the BR generated by the Q based OPV pass would be. In other words, the size correction of the second pass should correct no more than 10-15%.

Like RB said, it is not much use if you are going for a specific size like a DVD and have a luxury of BR to work with.

However, say you had two encodes (or 3 or 4) you wanted to put on 1 DVD . Well that sounds like a simple BR based operation, BUT it may not be that simple at all if you care about your work. Say one was The Matrx which is highly compressible, and one is SPR which is not very compressible, and yet you want to make sure you get about the same quality for both and you also don't want to take forever to do it.

This is where RoBa comes in. By using a sampling statement like SelectRangeEvery(750, 15) to sample 2% of the source at a chosen Q you can extrapolate the approximate total about of bits that encode will take to "hold" that particular Q level. You will find The Matrx will take much less space at a fixed Q of X than will SPR at the same Q of X. All you need to do is run and extrapolate the sample on both sources until you find a common Q that will use up the space on the target media. It does not have to be exact, just within a 10% error margin (preferably). Now do the encodes at that Q.

With the OPV pass finished you can now calculate the exact bitrate you wish your encodes to have to fill the exact space you have. You then run a BR based VBR pass (1, or 2 if you insist) that reuses the VAF file created in the OPV pass (important to make sure your template has "use existing VAF" checked). This makes them the EXACT size you wanted them to be,but they are also the exact quality you specified and they only took two full passes each.

So, to finally try to answer your question, RoBa is one hell of a lot different than blindly specifying a bitrate in a multipass and expecting the encodes to be the same quality.

BTW, If your two sources returned about the same 2% sample size at the same Q you would then know you do not have to worry with all of this rigmarole. Just do a bitrate based encode on both.

2nd BTW, the upcoming new release of DVD2SVCD (DVD) will support the creation of images via Scenarist. D2S already has an automatic RoBa mode when it is enabled in the INI, but depends upon the user to input the approximate Q desired for the space available. This works well but takes a few manual sample runs. The plan, as I understand it, is for the next release to have an Auto RoBA mode that will do that for you.

[RB]

Thanks DDogg!

Hmm, but that guide at http://www.doom9.org/mpg/cce-roba.htm Matthew mentions really seems to be broken. It simply creates the VAF using One Pass VBR and then does additional multipass VBR passes. The guide does not explain why a Q factor of 40 is choosen for the VAF creation nor how this is better than a standard multipass VBR encode. This is definitely not RoBa.

[DDogg]

Sometimes sticking a name like "The RobShot Method" or "The RobShot-Bach Method" (RoBa) makes something sound so much more complicated than it really is. (Warning: I did this as a self learning exercise, but this got much longer than I anticipated. Sorry!)

RoBa is just CCE one pass VBR with a tweak!

Quantizer based encoding methods like CCE 1 pass VBR (OPV) maintain a certain quality without regard for final encoding size. Simple in theory, but the real world has Max bitrate specifications on DVDs and SVCDs so it will play on all equipment, as well as fixed sized delivery media which consumers don't want half filled, or a disk and a half, or whatever.

Since the size of quantizer based encodes vary with the source complexity they are very difficult to accurately "fit" on a certain size disk and that is why quality based methods have not been popular (for obvious reasons). BTW, I think TMPG's CQ mode is somewhat similar, but I think there are differences that I don't yet understand.

Here is some data that might help. All settings were the same Q setting of Q-32. A 1% sample was taken and CCE 2.67.00.11 was used to do the encode. The quality is the same, but the size is all over the place.

1> "Final Dstination" --- D2SRoBa_Q32.mpv = 8,196KB, Len -- 133341 frames Time 1:32:41 Target mpv size:-(1043 kbps)
2> "The Usual Sspxts" - D2SRoBa_Q32.mpv = 10,380KB, Len - 148070 frames Time 1:42:55 Target mpv size:-(926 kbps)
3> "The Mtrix" -------- D2SRoBa_Q32.mpv = 8,388KB, Len --- 186356 frames Time 2:09:32 Target mpv size:-(709 kbps)

So in a natural way, Length, Aspect Ratio, and Compression characteristics are all taken into account. Note how #3 compresses to the approximate size of #1 even though it is nearly 40 minutes longer. Here the the encodes of #1 and #3 would be of similar quality even though the bitrate to fill one SVCD on #3 is completely down in the weeds. So bitrate is not the primary judge of quality...Compressibility is, measured in this case by the level of Q that can be "held" within a certain level of bitrate.
-------------------------------------------------------------------

Doesn't sound real practical does it? But, if you start to look at it from a "What minimum quality can I get on this disk in my hand" it starts making much more sense. In fact, another way to look at it is, "What quality will I end up with BEFORE I do the encode". That is a very handy thing to know and RoBa allows you to do just that. Let me try to get there. I'm a long winded Texan ;)

I think a person could argue that CCE's OPV (1-pass VBR) creates a better encode than multipass, however you can't get decent size control. Several tools like Tylo's D2SRoBa plugin for D2S try to do multiple sampling passes to predict the Q you need to fill a certain size. It does a great job and is getting better everyday, but by the nature of predictive sampling there is normally always some error. That's ok for a lot of folks, but it makes me crazy. Remember, its about quality, not size. :rolleyes:

The value of these tools, to me, is more what they tell us. If a tool predicts it will take jacking the Q up to 56 to get the encode on the target I know right off it will look very bad (to me). So I can either blow it off, or perhaps use some advanced filtering and compression scripts to help get the compression level up and the Q down.

Again, the prediction tool will tell us if those special scripts are doing the job by just running 1% samples and seeing if the Q dropped from the previous test pass. You can also just look at the size of the sample.mpv resulting from your test runs. If it is getting smaller you are on the right track.

This doesn't take very long on modern machines and at this stage I don't give a hoot if the size projection is perfect and exactly accurate, rather I just need a comparison to the previous 1% test to tell me if I am getting where I want to go, Q wise. We will fix the size in the second pass (so long as it is not too far off, say 10%-15%).

Anyway, let's say we are darn good compression artists and we get it tweaked so our prediction tool now tells us we can get a Q of 32 on our target media. My personal choice is anything between 20-32 will be stellar and 32-40 ought to be very good (IMO-subjective comment!).

If I were a guy who trusted sampling methods I would just go ahead and do the CCE 1-pass vbr pass with the predicted Q. Then I would be finished and have a great encode. (BTW, this is the so called 1-pass Bach method.) Unfortunately, we do not yet have any tool that will give us that level of accuracy. The encode might be undersized or over-sized just enough to make you angry enough to kick your dog.

Enter RoBA as a the safe guard. Ok, so go ahead and do the predicted Q pass, you may get lucky and be done. If not you have the second phase of RoBa to get you out of trouble. [note concept of conditional here. If size is <> than xxx do second pass, else be very happy boy with a 1 pass encode]

Using the RoBa Process you will reuse the Quality based VAF that was generated in the first OPV pass. A very powerful feature of CCE is that this second BR based VBR-pass will read the VAF previously created and use the information created in the first Q based pass, but will slightly "tweak" it to hit the bitrate target you wanted.

That's why you don't want more than a 10% (15% if you have to) size correction. We want to keep close to what we would have had IF we had known the exact Q we needed to exactly hit the size target.

The bitrate we will need for the sizing pass is calculated using standard methods. Then, after that BR based VBR second pass is finished we have a known Q AND exact sizing. That is the RoBa Process in a very simplified nutshell. Not very complicated after all, was it?

More: So, in the worst case we would have only two passes. In the best we would have only 1 pass, although it would need several 1 or 2% prediction passes. Problem is we do not have a stand-alone tool that will do these Q prediction passes accurately enough. If we did we would not need the second pass at all. Hint to RB :)

r6D2 in his post of 20th August 2003 16:07 in this thread http://forum.doom9.org/showthread.php?s=&threadid=59054&perpage=30&pagenumber=4 suggested the use of the "Newton-Raphson method" to more accurately compute the correct Q in only 4 passes. This Stuff went way over my head and caused my eyes to cross permanently, but it may make sense to some of you brainiacs out there. Tylo found a good link: http://www.sosmath.com/calculus/diff/der07/der07.html (something about Babylonians having square roots :confused: :D )

[b0b0b0b]

I was totally following it until the page said: Easy calculations give http://www.sosmath.com/calculus/diff/der07/img11.gif

I guess after some thinking I can see how you would divide the y by its derivative, but I have tiger woods 2003 to play. Thanks for the link, it's a cool page. :)

[DDogg]

A stand-alone "no brainer" tool to do the prediction in a fast and efficient way would be a god send to all of us and I think it would be one of those tools that is everybody's toolbox.

It would help the DVD'ers and the SVCD'ers to save a lot of time. I don't think it is a big deal to do one, but I am not a coder so nothing I can do about it. A coder would not have to use the fancy Newton stuff above. It would just take an extra pass or maybe two (I have been told) to do it with regular math.

[DDogg]

@b0b0b0b

Great news! r6d2 did a lot of work and demystified his suggested process. http://forum.doom9.org/showthread.php?s=&threadid=60191

[Jan Marijniszoon]

I have a question (especially for DDogg)...

I ended up with a file that is just 30 MB too short. So it will fit and I will 'waste' just 30 MB.

I did the encode with a one pass at a constant quality of 11.

Do you advice me to use a second pass to fill up the extra 30 MB?

While I'm at it...
"Image quality priority" is always set at 16 by DoCCE4U.
What's the difference with 15?

Thanks in advance!

[DDogg]

I guessing you mean you used an OPV Q-Value of 11? Assuming you did, your AQuant is already going to be very low and the video quality should be outstanding. Given that, I don't think it is worth the time and aggravation to do the second pass. You surely could never see any difference. 30 megs isn't much on a DVD.

No difference between 15 and 16. Btw, is that for CCE 2.5? It uses a 1-100 scale. Later versions of CCE use a 1-64 scale so you have to adjust upward to like 24-27, IIRC.

[Jan Marijniszoon]

DDogg, tanks for responding so soon.

Yes I indeed did. I don't mind taking the time.
I re-try untill I have the Q-factor that fits best. I usually take a good guess and end up pretty good.

I didn't understand the RoBa for a 100%. But your elaborate explanation above made it more clear for me.

It's indeed about 2.5
I just don't understand. DoCCE4U says priorty 15, but in CCE it always becomes 16. Why is that?

[DDogg]

Well do understand that "RoBa" can mean different things to different people. I try to not use the term anymore because of that. The process I use is called "2 Pass Conditional Bach" by some of us. That very directly states what it is. One OPV pass that can be followed by a 2nd bitrate based VBR which reuses the VAF created by the first pass. This second pass can be used to achieve perfect sizing if the size correction is no more than 10% or the second pass can be used to potentially increase quality, especially if the OPV Q-Value used was above 32-40. Since the second pass is normally only used when certain preset conditions are present, it is conditional.

RoBa as used by some of the other tools may reflect a different understanding, or may be based upon older methods that some feel are still the correct methods. I would not want to get in to that :)