How many passes in CCE?

[Toranaga]

How many passes is advisable in CCE? What I mean is if it is worth the extra time to do 5 passes instead of 2 or 3? Maybe it depends on the length/size of the movie?

[lab-one]

Heh, I'll put my foot in my mouth....

I rarely do more than 2 (in DVD-RB this includes the .vaf pass). Sometimes I have done 3 but I PERSONALLY can not tell the difference on my 32" tv. (I have found that when I have tried 5+ that I PERSONALLY felt that the quality became worse. It looked "grainy".)

Searching the forums will reward you with a vast wealth of posts in regard to this. However, what you will learn in a nutshell is, it's all in the eye of the person viewing the final result as to what is best and/or what is "right".

[nwg]

It depends on how much reduction it is.

For DVDs down to 80% of the original I use 2 passses. Anything less than I do 3 passes.

[GooglyBear]

depends on the movie! if it's soft porn or some campy b grade type movie hell just do one pass!

but if it's like my incoming matrix revolutions from 'flix.. I'm definitely letting my pc go 17 hours and do 5 passes! :devil:

donnie darko i used 3.. confessions of a dangerous mind I did 5.. rundown I did 3 but shoulda done 5.. pitch black unrated I did 3 but shoulda gone 5 as well..

but these were during dvd-rb's early days and going 17-25 hours for a failure wasn't worth it so i took what i could :P

[kheops]

hi

i second lab-one
i've read here and there that more than 4 (or 5) passes could make the result worse than better

+

[echooff]

My home video looks best after 3. 2 is not enough and 4 or don't seem to make any difference.

[Lazza]

I'll third lab-one. :D

Been doing just 2 passes so far and everything has looked fine, did a longer film this afternoon on a 3 pass and perfect to my eyes.

[int 21h]

Originally posted by GooglyBear
...but if it's like my incoming matrix revolutions from 'flix.. I'm definitely letting my pc go 17 hours and do 5 passes! :devil:
...

Go buy the movie, and don't talk about piracy here.

[Toranaga]

LoL. It IS my own movie I am using.

[int 21h]

My original reply was directed towards 'GooglyBear'

In regards to CCE passes, if its above 3700 in bitrate, I usually do 3 passes, anything lower I do 4 or 5 passes (+1 for the VAF generating pass)

[jdobbs]

My opinion is that virtually every advantage you get from multi-pass VBR comes after only two passes. The encoder knows where the demand is and applies more bits appropriately. Each additional pass may show improvement but the amount of that improvement is exponentially smaller than the previous. So it's a matter of whether you think the improvement is worth the time. I've seen the reports -- but I have a hard time imagining how additional passes can actually damage picture quality.

So the bottom line is that with electricity at 7 cents a Kwh and the true draw on my computer at about 150 watts... I'm able to run continuous passes, 24 hours a day, for a month -- and it would still only cost me 8 bucks... so what the hell... pass away!

[DDogg]

int 21h, since I know you are extremely quality oriented, knowledgeable, and fact based, I'll hazard a slightly OT question since the thread originator has got his answers.

Have you spent any appreciable time analyzing the quality of your multi-pass encodes to the equivalent sized/bitrate OPV? If so, and only if you feel comfortable venturing your opinion, I would appreciate knowing your thoughts on whether you feel there is any appreciable quality difference between the final product of the two different methods.

BTW, I'm not baiting, or attempting a religious conversation on the topic. Just looking for an opinion from a person I know has the background to offer a valid and fact based commentary.

[int 21h]

Originally posted by DDogg
...Have you spent any appreciable time analyzing the quality of your multi-pass encodes to the equivalent sized/bitrate OPV? If so, and only if you feel comfortable venturing your opinion, I would appreciate knowing your thoughts on whether you feel there is any appreciable quality difference between the final product of the two different methods...

Background
About a month ago, I did investigate this, albeit briefly, I saved my results for further comparison and reflection (I guess its time to reflect? :)). Using a bright and colorful children's movie, I selected a chapter that I felt showcased the unique colors present. Then using Bitrate Viewer, I recorded the relevant information. Then, I encoded that same chapter in CCE with 4 different Q Levels in OPV mode. (Q15,Q30,Q60,Q90) using a bitrate range of 1.0-6.5 Mbps. While they encoded, I noted the minimums and maximums in bitrate that CCE displayed it was using just to make sure that Bitrate Viewer wasn't feeding me entirely inaccurate numbers. Finally, using Bitrate Viewer's calculated average values, I again encoded the chapter using 3 pass mode, specifying 1.0-6.5 Mbps as my range. Here are my not too surprising results:


Method Bitrate(max) Avg. Quant File Size
-------------------------------------------------------------------------
Original 7296(9516) 6.62 216 MB
Q15 4944(6703) 3.39 163 MB
Q30 3488(6328) 4.91 118 MB
Q60 2352(4463) 7.79 77 MB
Q90 1845(3425) 9.70 61 MB

3pass @ 4944 4631 (6066) 3.65 151 MB
3pass @ 3488 3361 (5359) 5.20 110 MB
3pass @ 2352 2204 (3646) 8.13 72 MB


Misc Comments:
Q15 - nearly indiscernable from original DVD, some light noise around edges, existing noise slightly amplified
Q30 - discernable from original, softer edges, light to medium noise
Q60 - very discernable from original, gradients are less gradual, medium to heavy noise
Q90 - nothing like original, heavy noise, obvious gradients, generally unusable

3pass @ 4944 - Identical to Q15
3pass @ 3488 - Less noise than Q30, but otherwise identical
3pass @ 2352 - Identical to Q60, quantization artifacts exhibited in the same macroblocks.

Conclusion
Not surprisingly, multipass VBR improves filesize. However, in my impromptu tests, it only improved filesize by around 10%. (Factoring how much space the main feature of a DVD takes up in relation to its extra features, etc., this usually equates out to around 5-7% of an entire DVD) Furthermore, multipass VBR also results in a lower average quantization factor, however this is such a small difference that, in my opinion, it is imperceptible from an average television viewing difference (analog or digital). (i.e. The quantization difference between Q based encoding and Multipass encoding can be considered statistically insignificant) To my eyes, at higher bitrates (higher Q level), the output is identical, as bitrate decreases, there is a point at which multiple passes can help, but as bitrate decreases further, the outputs become more similar, even so much as exhibiting the same compression artifacts in the exact same locations in a frame. (this may vary if different encoders are used, this was only observed in CCE)

Bottom Line
In high bitrate situations, multiple passes make absolutely no difference. It simply supplies you with the assurance that your file will reach your target size. In very low bitrate situations, multiple passes still make no difference. The quantization factor is so high at these low bitrates that its simply impossible to accurately reproduce the original image. In medium bitrates, its up to you. Its entirely subjective.

The situation is only complicated by transcoding applications like DVDShrink and Rejig. I personally tend to use DVDShrink in most of my encoding, simply because of the overall timesavings it allows (No ripping needed, no preprocessing, no postprocessing, i.e. generates ISO for me) and in high bitrate situations (30% or less in overall reduction) its performance is very comparable to the traditional processing methods. But again, this is very subjective. Obviously CCE can always give the best possible image quality from a given source (because of its post processing, etc.), but the object of transcoders has never been the best possible image quality. The object of transcoding is acceptable image quality with least CPU-time input.

When I encode, and I encounter a very high bitrate situation, I usually just let DVDShrink handle it, when I encounter a medium range bitrate, or a relatively high factor of reduction (40-60%), I encode with CCE to have greater control over the quantization, and when I encounter a very low bitrate, I just look at the calender, and if date is even, I use DVDShrink, odd, CCE. :-p

Various Disclaimers
*Bitrate Viewer can be innacurate, but since I used it for all the results and source, the general comparison statistics should still hold true.
*Comments are based from still-frame analysis, its quite likely results would vary in full motion analysis watching in a typical viewing arrangement, but I didn't have time to reproduce this scenario.
*This test is somewhat qualitative and as such, is nearly 100% subjective, therefore, its quite likely you don't agree with some or even any of my findings, that's ok, this is 100% my opinion, not to be taken as the holy book of encoding video or any other scripture, religious artifact, or cult icon.
*As my original post to this thread shows, even though I have these facts at hand, I still sometimes do 5+ passes... Why? Because my PC gets bored at night without me, at least encoding can keep its mind off of my absence.

[DDogg]

Nice work. My feelings are in parallel with yours for the most part. Frankly I just could never personally tell the difference between an OPV in the sweet-spot of 18-36 and a 4 pass multi-pass. Especially since it took a fifth of the time of a 4 pass. Other people say they can and some say they can't. Typical bs and it sure ain't very scientific. It is very good to see these less subjective results from you. Thanks for taking the time to post them.

I had been thinking of doing something similar. Maybe I might do the source generation for the comparison slightly differently - If I ever get off my duff I thought I might do a series of 0-9000 OPV passes of Q12,15,18,22,25,32,38 and 40. Then extrapolate the KBPSs of those encodes and do the average bitrate equivalents in 3 pass multi-pass. Just so the comparative samples would be more nearly exact in size and headroom to each other. Then I thought I could probably use SSIM to compare each back to the original and chart the results. Seems like that ought to work in theory but also sounds like a grind, especially since I think the results would mirror yours.

I might as well ask if you see any immediate problems with that method returning valid results? I have not really thought it out in any detail, but would eventually like to do it and present some valid results so people could actually have another fact based comparison like yours.

[int 21h]

I would be very surprised if you turned up results that didn't agree with these.

And it makes perfect sense if you consider it.

OPV lets you manipulate the quantizer, the lower the number, the lower the 'target' quantizer, and the higher the bitrate. The goal being to keep the quantizer within a specified value range (dictated by Q).

Multipass lets you manipulate the bitrate, thus manipulating the quantizer indirectly. The goal being to keep the average bitrate at a specific value (to manipulate filesize) by scaling the quantizer.

Multiple passes can help in the low bitrate situations, but anything more than 3 won't make a difference, and once you get to very low bitrates, the number of passes won't help at all. (Because you'll hit a wall with your quantization and you can only manipulate bitrate through quantization, in fact, you may even find your target bitrate impossible to reach... if you hit maximum quantizer level and your size is too big for instance)

But feel free to test more, no harm in checking things out. You could probably check out some literature somewhere to get more details about this, you would want to look at information specifically dealing with Mpeg-2's quantization structure (step sizes, bins, etc.) I looked at it briefly a long time ago, but have since forgotten most of its details.

[maa]

Just for the hell of it I put two DVDs on one just recently with original menus, (not re-authored)two languages and re-encoded to amorphic 16:9
I tried Main Concept and CCE.
At this scale (Video output 1.8 Giga)I found CCE could cope with the faster scenes where MC packed up altogether - best example - someone walks by the camera. I went with MC anyway because it not only delivered a sharper picture but didn't reduce the brightness like CCE did.
Forgive me if this is not scientific, I'm very new at encoding.

BTW. two original menus on one DVD is not easy so don't rush to try it unless you have a great deal of experience with GPRMs and the command set in general.

cheers

maa

[DDogg]

Originally posted by int 21h
OPV lets you manipulate the quantizer, the lower the number, the lower the 'target' quantizer, and the higher the bitrate. The goal being to keep the quantizer within a specified value range (dictated by Q).

Multi-pass lets you manipulate the bitrate, thus manipulating the quantizer indirectly. The goal being to keep the average bitrate at a specific value (to manipulate file-size) by scaling the quantizer.
Yes, both processes end up yielding approximately the same final results, but only IF multi-pass has been allocated enough bitrate to carry the source complexity. In my mind, that really brings up the major differences between the two processes that I think have been obscured by the quality discussion.

Let me throw out a few of the parts and pieces of the working notes (very rough) of a FAQ that I've been doing as a self learning exercise. I've been wrestling with getting this on paper (which I doubt I'll ever finish) and am having hell trying to present this stuff clearly, especially when analogy is attempted. Having somebody to discuss this stuff with would be very helpful.

Notes:
[to do - insert basic overview of both processes. Identify the basic strength and weaknesses of Mpass and OPV as implemented in CCE - Introduce why adding predictive sampling to CCE's OPV creates a third distinctly separate hybrid process with additional benefits not available in the CCE internal processes]
======================
Predictive OPV -
Concepts - Why understanding source complexity (resolution, length, and compressibility) is important to understanding the strengths of predictive OPV
Goal - to most efficiently deliver a known quantizer (loosely quality) in the least bitrate (loosely file-size), OR to know the quality that will be delivered in a known file-size, all in advance of actually doing the full encode.

1> Each and every video source has unique compression characteristics. At the most basic, the reader has seen(I hope),that a slower moving dark source will compress much better than one of high action, grain and brightness. Likewise, a full screen resolution takes more bitrate than a widescreen resolution. In combination these factors can be called the source complexity.

2> It makes sense then to say the complexity of the source is what solely defines the bitrate required to hold a controlled level of quality. In other words, bitrate, by and of itself, is irrelevant to the measurement of quality. Like a truck, it is only the vehicle. How much material that can be packed in to the truck is based on how complex the shapes of the contents are, i.e., many more small compact square blocks will fit in a truck of a given size than oddly shaped irregular blocks.

3> It follows that if we desire to generate a known file-size at a known quantizer we must know the source complexity (shape of the blocks) before allocating file-size (truck space) via the average bitrate to achieve a controlled level of quality.

These points bear on some of the weaknesses of CCE multi-pass for our stated goal as it is unaware of the complexity of the source until after the VAF pass. Remember, to start creating the VAF requires us to declare the bitrate in advance of knowing the complexity of the source. This seems a primary contradiction to our stated goal mentioned above.

[to do - complete - describe how the awareness of source complexity (resolution, length, and compressibility) are automatically incorporated within predictive sampling as an inherent part of the process]


Let me know your comments. Please don't feel the need to be conservative with them if you feel I am way off the mark, or that I have explained poorly. As I said, I'm having great difficulty getting this stuff written. I could use some help.

[jdobbs]

Multi-pass lets you manipulate the bitrate, thus manipulating the quantizer indirectly. The goal being to keep the average bitrate at a specific value (to manipulate file-size) by scaling the quantizer. Maybe I misunderstand, but I think that multipass does more than just manipulate the quantizer indirectly, it allows you to allocate the Q at a fixed level across the entire encoding at the highest level possible within the bitrate limitation.

[bobwillis]

I couldn't pass up on this discussion; religious or otherwise. I hope no-one minds. This is a re-iteration of my previous ramblings on the subject, but hopefully better written!

For me it is this simple (or complex).

1. OPV gives constant quantisation encodes. If you examine I,P & B frames whilst encoding, their quantisation value is approximately constant across the frame range. This has been stated previously by other people as 'every frame has equal quality to the previous one or the next one'.

2. VBR depending upon the bias setting does not encode with constant I,P & B. If bias = 0, then quantisation is constant (like OPV), so you may as well not use it. As bias is increased, the bitrate swing is constrained. In other words, it would not go as low as OPV, or as high as OPV for a specific frame. This means that the quantisation is non-constant; yielding visual differences between the two techniques.
Setting Bias=100 yields a CBR encode, i.e. quantisation is massively variable.

This can be summed as:

1. OPV or VBR with bias=0; quantisation constant, bitrate most variable.
2. VBR bias non-zero; quantisation becomes more-variable as bias increases, bitrate becomes less variable as bias increases.
3. VBR bias=100; quantisation most variable; bitrate constant; effectively a CBR encode.

Because OPV gives constant quantisation, and VBR (with bias set to non-zero) does not give constant quantisation, a comparison between the two techniques can lead to visual differences because the same scenes will have different quantisation applied by each technique. Which is better depends upon your faith, but I'm perfectly prepared to accept that some scenes will look better with one technique and other scenes will look better with the other technique.

I think that with high Q (over 40) material, constant quantisation begins to looks suspect in simple scenes, and think that VBR is better employed here.

Regards,
Bob

[int 21h]

Originally posted by jdobbs
Maybe I misunderstand, but I think that multipass does more than just manipulate the quantizer indirectly, it allows you to allocate the Q at a fixed level across the entire encoding at the highest level possible within the bitrate limitation.

You can't directly influence the quantizer level in multipass. If you specify an average bitrate of 2700, you have absolutely no way of knowing what kind of quantizer level that will yield for your source, if you specify a OPV Q level of 10, you know your quantizer will never go above 4 or 5.

[int 21h]

Originally posted by bobwillis
I couldn't pass up on this discussion; religious or otherwise. I hope no-one minds. This is a re-iteration of my previous ramblings on the subject, but hopefully better written!

For me it is this simple (or complex).

1. OPV gives constant quantisation encodes. If you examine I,P & B frames whilst encoding, their quantisation value is approximately constant across the frame range. This has been stated previously by other people as 'every frame has equal quality to the previous one or the next one'.

2. VBR depending upon the bias setting does not encode with constant I,P & B. If bias = 0, then quantisation is constant (like OPV), so you may as well not use it. As bias is increased, the bitrate swing is constrained. In other words, it would not go as low as OPV, or as high as OPV for a specific frame. This means that the quantisation is non-constant; yielding visual differences between the two techniques.
Setting Bias=100 yields a CBR encode, i.e. quantisation is massively variable.

This can be summed as:

1. OPV or VBR with bias=0; quantisation constant, bitrate most variable.
2. VBR bias non-zero; quantisation becomes more-variable as bias increases, bitrate becomes less variable as bias increases.
3. VBR bias=100; quantisation most variable; bitrate constant; effectively a CBR encode.

Because OPV gives constant quantisation, and VBR (with bias set to non-zero) does not give constant quantisation, a comparison between the two techniques can lead to visual differences because the same scenes will have different quantisation applied by each technique. Which is better depends upon your faith, but I'm perfectly prepared to accept that some scenes will look better with one technique and other scenes will look better with the other technique.

I think that with high Q (over 40) material, constant quantisation begins to looks suspect in simple scenes, and think that VBR is better employed here.

Regards,
Bob

The quantization isn't constant. If it were, looking at a graph of the quantization would reveal a nearly straight line, but it doesn't. A constant quantization would be a different mode (i.e. a CQ or constant quality mode).

If you wish to prove your idea, encode a clip with a Q of 40, find the average bitrate, then encode the same clip with that average bitrate, and share the results.

I disagree though, I believe they will look identical.

[int 21h]

Originally posted by DDogg
[snip]

It would probably be a good idea to explain some of the basic terms involved, like quantization, bitrate, and the idea of rate control in video compression. What do you think?

[bobwillis]

int21h,

Figure A.8, A.9 & A.10 of cctsptv2.66.pdf clearly illustrate that I,P & B are constant, except where the bitrate would have to exceed 15Mbps in order to give the same value of Q. This of course is not possible.

Quote: If it were, looking at a graph of the quantization would reveal a nearly straight line, but it doesn't. A constant quantization would be a different mode (i.e. a CQ or constant quality mode). That is exactly what the plots show; a flat line, thus constant quantisation.

I am encoding someting now with OPV Q=12, if I look at the CCE window, I = 2.21, P= 2.21 & B = 3.23. I have looked at them for 2 minutes, and they have not changed by more than 0.02, but the bitrate swings wildly. Hence variable bitrate, constant quantisation.

Regards,
Bob

[int 21h]

Ah, it should be mentioned that I am using CCE 2.50, I don't have a license to 2.66, but perhaps I will experiment with it when I get a chance.

[DDogg]

Originally posted by jdobbs
Maybe I misunderstand, but I think that multipass does more than just manipulate the quantizer indirectly, it allows you to allocate the Q at a fixed level across the entire encoding at the highest level possible within the bitrate limitation. I'm thinking it may be more precise to say, "Multipass allocates the available bitrate to the areas of highest need, thus indirectly manipulating the quantization and providing the highest level possible within the bitrate limitation". Would you agree with that?Originally posted by int 21h
It would probably be a good idea to explain some of the basic terms involved, like quantization, bitrate, and the idea of rate control in video compression. What do you think? Yeah, you're right - Gee, that sounds like real exciting fun :) Other than that, no concept problems?

Here is another attempt at an analogy of multi-pass:

Multi-pass is roughly analogous to a person being given two instructions -
1> Take a presized container and pack a weight of material (source length) which is made up of a sequential set of blocks of unknown size and shape (compressibility)
2> Under no circumstances allow the packed material to exceed the size of the container, even if one must alter it to get it to fit.

Of course this works fine as long as the container is big enough. It is an easy job, and all pieces fit within the container although we may well have a lot of wasted space.

The big problems start to arise when the container is not large enough to contain the material when packed. The packer must take out a saw and start reshaping the more unruly pieces (constraint of bitrate) so they will fit within the container. Damage is done in order to make everything fit(degradation of video quality).

Neither of these situations would happen if we had been allowed to evaluate the shapes of the blocks before we rented the incorrectly sized truck. However the nature of multipass does not allow that, so we can get either of these situations. We can also just get lucky and everything fits without wasted space or any appreciable alteration. Depending on luck for a precise process seems contradictory.

Hey, This may be too much of a broad stretch. I'm just trying to take the define the broad concepts and take them out of of techno speak.

[int 21h]

Originally posted by bobwillis
int21h,

Figure A.8, A.9 & A.10 of cctsptv2.66.pdf clearly illustrate that I,P & B are constant, except where the bitrate would have to exceed 15Mbps in order to give the same value of Q. This of course is not possible.

Quote: If it were, looking at a graph of the quantization would reveal a nearly straight line, but it doesn't. A constant quantization would be a different mode (i.e. a CQ or constant quality mode). That is exactly what the plots show; a flat line, thus constant quantisation.

I am encoding someting now with OPV Q=12, if I look at the CCE window, I = 2.21, P= 2.21 & B = 3.23. I have looked at them for 2 minutes, and they have not changed by more than 0.02, but the bitrate swings wildly. Hence variable bitrate, constant quantisation.

Regards,
Bob

Its quite possible (and fairly likely) that I'm misinterpeting Bitrate Viewer's output, I did confirm your observation in the CCE manual... maybe Bitrate Viewer isn't showing Quantization scale used, I don't know, I don't really have much more time to spend researching this :(

[int 21h]

Originally posted by DDogg
...
Here is another attempt at an analogy of multi-pass:
[i]Multi-pass is roughly analogous to a person being given two instructions -
1> Take a presized container and pack a known weight of material (source length) which is made up of a sequential set of blocks of unknown size and shape (source complexity)
2> Under no circumstances allow the packed material to exceed the size of the container, even if one must alter it to get it to fit.
...

It all sounds fine to me... Good luck.

[jdobbs]

Originally posted by int 21h
You can't directly influence the quantizer level in multipass. If you specify an average bitrate of 2700, you have absolutely no way of knowing what kind of quantizer level that will yield for your source, if you specify a OPV Q level of 10, you know your quantizer will never go above 4 or 5. But you don't have to. It will be set to the highest it can be with the specified average bitrate (and within the high/low). If you have a fixed amount of output space -- the best you can hope for is the highest Q that will fit.

Originally posted by DDog
I'm thinking it may be more precise to say, "Multipass allocates the available bitrate to the areas of highest need, thus indirectly manipulating the quantization and providing the highest level possible within the bitrate limitation". Would you agree with that? Absolutely. I think that quality should be the number one consideration. But it has to be held to within what the environment allows.

[DDogg]

Absolutely. I think that quality should be the number one consideration. But it has to be held to within what the environment allows. I agree completely with what you say, but would simply add that if we could actually know what the environment requires in advance of occupying it, we could make better decisions about whether we want to live there. We can do that now with sampling techniques that can provide the additional missing knowledge of the environment to CCE multi-pass. It no longer necessary to go in semi-blind as we have had to do before. btw, I'm not talking specifically of DVD-RB, just generalizing.

[jdobbs]

Originally posted by DDogg
I agree completely with what you say, but would simply add that if we could actually know what the environment requires in advance of occupying it, we could make better decisions about whether we want to live there. We can do that now with sampling techniques that can provide the additional missing knowledge of the environment to CCE multi-pass. It no longer necessary to go in semi-blind as we have had to do before. btw, I'm not talking specifically of DVD-RB, just generalizing. Understand. The subject of quality and methods to improve it is always interesting to me.

[DDogg]

Just to wind down my thoughts:

While personally I'm a big advocate of Predictive OPV (RoBa) and use it exclusively, its not for everybody. D2SRoBa (http://home.tiscali.no/tylohome/), which is a damn brilliant piece of mature work incorporating r6d2's adaptation of the Newton-Raphson method of convergence (http://www.sosmath.com/calculus/diff/der07/der07.html), only works with DVD2SVCD (also does movie only DVD and VCD).

My thinking is that multi-pass is the standard used by the majority and works extremely well, well mostly it does. It would work even better if you knew in advance of the encode the appropriate minimum bitrate to allow a known quality. If we did know this figure in advance, multi-pass would be near perfect.

The other day it occurred to me (nothing original) that one 1% OPV sample gives all missing information needed to use multi-pass to its best advantage. From the size of the 1% sample, which should not take more than a couple of minutes to do, you can calculate the minimum multi-pass average bitrate (space) needed to obtain the quality you specified, approximately.

This provides some additional advantages for CCE MultiPass -

1> I'm guessing, but you could probably use less passes because this method allows multi-pass to start up with an adequate bitrate matched to the source complexity. The extra passes should not have as much work to do. It should yield a quality equivalent to OPV, or perhaps even better, because it has, at the least, the same minimum space to distribute the bits.

2> By knowing the minimum average bitrate/size you need in advance of the encode you can quickly solve for more unknowns and can make smarter decisions. This whether you are the programmer of an encoding solution, or as an individual that does not want to waste hours doing a multi-pass encode and then find out the encode is of poor quality because it was allocated too little space for the source complexity.

3> May be helpful in intelligently allocating average bitrate/space in some applications where the encode must share space with other objects so as to insure a minimum quality for the movie. Dunno. ;)

4> In short, provide many of the benefits of Predictive OPV without the hassle of implementing the PITA logistics of size prediction and doing all the weird convergence math.

Note: Jonny's QCCE Beta (http://forum.doom9.org/showthread.php?s=&threadid=68527) also provides some standalone Q prediction abilities, and will probably continue to improve if the author has time.

[DDogg]

If anybody wants to play around with this, and needs to know how:

1> Load you working AVS in VDub. Note frame-count and frame-rate in file menu > file information, Close VDub.
2> Insert as last line in your AVS script - SelectRangeEvery(1200,12)
3> Set CCE for OPV with a Q of 25 ( a good reference point), 0-9000 bitrate for DVD and the appropriate normal settings for the source style like normal. Encode the sample. Should take a couple of minutes.
4> Note the sample size in Windows Explorer - Right click > properties to get the size in bytes.
5> Since I can't upload my spreadsheet use this formula Thanks, RB!: (((Sample_size_in_bytes*100)*8)/1000)/(number_of_frames/frame_rate) [spreadsheet now six posts down]

Here is an example: Source is 150067 frames @ 23.976 (1:44:19:05 length). I solved for Q 25 which took 45 seconds to create the sample. The sample size created was 15,156,084 bytes. So we have -
(((15,156,084 * 100)*8)/1000)/(150,067/23.976)= 1937 kbph as a minimum to hold a average quantizer equivalent to a Q25 OPV

Surprisingly low, isn't it? This is a full 720x480 resolution of Xmen2. Q 25 provides a very high level of quality in a approximate filespace of 1.51 gigs for the video. A Q12 only needs 2,997 kbps for an approximate file space of 2.35 gig for video. Now go try SPR, you may be very suprised at the different bitrate that will be needed to hold a Q25

The resulting number is the minimum bitrate in kbph required to encode your source at the equivalent average quantizer value of a Q25 OPV. It is a custom number matched to this particular source complexity and is not transportable to other source. Hey, that's the point, we did all this to pre-analyze the source complexity to find out the minimum kbph required to achieve a known quality level. In this case it was 25 which is a good reference point. Personally I solve for a Q25 or less although if source complexity and/or space constraints demand it a Q 32 is acceptable.

Don't misunderstand, you don't need to use the above kbps number for your multipass encode. Remember, it was just the minimum you will need to achieve at least that level of quality. It took about three minutes to do the above. Not much time spent to find out so much valuable info before we even started the main encode.

Note: remember to remove the SelectRangeEvery line from your script before the main encode.

[jdobbs]

Hey DDOG -- I want to do some experimenting with RoBa as a possible option for DVD ReBuilder. I'm assuming some of what you're saying here is based upon it??? What I'd be interested in is a formula that calculates (after a 1% or so sample like you show here) a Q factor that would fit in n bytes...

[DDogg]

What I'd be interested in is a formula that calculates (after a 1% or so sample like you show here) a Q factor that would fit in n bytes... To clarify - by that, do you mean the process to predict the Q factor that will produce a final encode, entirely with OnePassVbr, that will fit in n bytes?

Actually, I hope that is not what you mean as much of this work was stimulated by my desire to help you avoid a standard RoBa process. Instead, the stuff above is how to use a hybrid multipass, but get the advantages of RoBa without the hassle of the multiple predictive passes and weird-assed calculus required to do it efficiently.

Let me know a little more of your thinking on your direction, either here or PM. I think I can at least be a resource to help you save the time of reinvention. Most of this stuff is in the can at one place or another in these forums, but it can be buried in some real flatulent threads (me being a primary gas producer :-))

BTW, some tests I just did are showing multipass, fed with the extra info from the OPV sample, is turning out a slightly less average quantizer than the equivalent OPV in as little as 1pass (1 VAF and 1 VBR)

A 5% sample of the source above -
OPV Q40 - sample size 76,812,840 bytes avg quant 4.02 avg br 1492
1passMP - sample size 76,896,936 bytes avg quant 3.76 avg br 1494

[DDogg]

Note the very linear relationship between the size of the OPV sample and the Multipass sample which used the kbps derived from the OPV sample size.

I have attached a [updated]spreadsheet and added the bitrate viewer information for CCE 1,2,and 3 pass. I also did 4 pass, but the information was so close to 3 pass that it would be wasted effort to put it in here.


Derived Bitrate Viewer OPV
OPV Q Sample size Kbps Peak Average
(bytes) BR Q BR Q
OPV01 46,788,608 5,980 6412 2.29 4488 0.65
OPV05 33,600,512 4,295 5534 2.65 3218 1.22
OPV10 25,199,616 3,221 4771 3.38 2411 1.89
OPV12 23,445,504 2,997 4406 3.75 2242 2.18
OPV15 20,904,960 2,672 4060 4.27 1999 2.66
OPV20 17,545,216 2,243 3889 5.05 1677 3.34
OPV25 15,156,224 1,937 3718 5.96 1448 4.05
OPV30 13,504,512 1,726 3446 6.51 1289 4.70
OPV35 12,391,424 1,584 3144 7.10 1183 5.30
OPV40 11,496,448 1,469 2913 7.69 1097 5.82
OPV45 10,756,096 1,375 2704 8.17 1027 6.33
OPV50 10,169,344 1,300 2533 8.61 971 6.78
OPV55 9,674,752 1,237 2385 9.12 923 7.23
OPV60 9,242,624 1,181 2257 9.57 882 7.66
OPV100 7,215,104 922 1552 12.89 689 10.65


Mpass (3) 1 VAF + 2 VBR
1% sample source used as above
File Size Bitrate Viewer
Used BR KB 1 Pass (VAF+1VBR)
(from (bytes) Peak Average
above) BR Q BR Q
MPass01 47,057,920 6230 2.51 4495 0.62
MPass05 33,828,864 4694 3.24 3230 1.17
MPass10 25,352,192 3839 4.31 2422 1.81
MPass12 23,587,840 3607 4.70 2253 2.04
MPass15 21,049,344 3252 5.33 2008 2.45
MPass20 17,666,048 2778 6.31 1685 3.15
MPass25 15,252,480 2449 7.07 1454 3.76
MPass30 13,605,888 2153 7.82 1295 4.34
MPass35 12,488,704 1964 8.29 1189 4.82
MPass40 11,581,440 1796 8.90 1102 5.31
MPass45 10,838,016 1653 9.51 1031 5.76
MPass50 10,248,192 1529 10.11 975 6.19
MPass55 9,752,576 1430 10.66 928 6.59
MPass60 9,312,256 1352 11.21 886 6.96
MPass100 7,278,592 1013 15.01 693 9.66


Bitrate Viewer Bitrate Viewer
2 Pass (VAF+2VBR) 3 Pass (VAF+3VBR)
Peak Average Peak Average
BR Q BR Q BR Q BR Q
6355 2.40 4507 0.61 6376 2.40 4506 0.61
4734 3.16 3236 1.16 4720 3.21 3235 1.16
3947 4.21 2422 1.81 3965 4.22 2424 1.82
3704 4.68 2253 2.04 3696 4.68 2255 2.04
3339 5.20 2012 2.44 3339 5.20 2012 2.44
2861 6.13 1688 3.15 2861 6.05 1688 3.15
2520 6.52 1485 3.93 2523 6.88 1457 3.77
2238 7.25 1326 4.40 2248 7.49 1300 4.33
2058 8.05 1199 4.82 2064 8.05 1193 4.83
1898 8.63 1105 5.33 1906 8.63 1106 5.33
1763 8.34 1045 5.75 1774 9.09 1035 5.79
1650 8.85 979 6.17 1658 9.62 979 6.20
1564 10.16 931 6.59 1574 10.08 931 6.60
1473 10.78 889 6.99 1483 10.64 889 7.01
1129 14.42 695 9.75 1091 14.03 695 9.78


Get attachment here - Spreadsheet of above information (http://forum.doom9.org/showthread.php?s=&threadid=74141)

[bobwillis]

I'm reading it; and would be very interested to learn of the differences in quantisation between the two techniques. I wonder if the difference would be dependent upon the Q value, so that we could determine if multipass was useful if the Q was above a certain level?

As ever, this is all interesting stuff. I think it has application for DVD-RB (not that I've used it yet), because you could let the user specify a high Q for the extras (since a lot of people don't give a toss about the quality of extras), leaving most remaining space for the main movie.

Regards,
Bob

[DDogg]

The attachment to this message is a derived bitrate calculator sheet that uses the sample size as input to derive kbps.

[jdobbs]

Sorry about that -- I wasn't on the ball.

[DDogg]

@jdobbs, that angry face wasn't directed at you, just at the silly way I can't even validate my own attachments.

@Bobwillis - Bob, reading between the lines of your note, no, I'm not going nuts :) I still love my OPV, but this method is specifically for situations that need several items to co-reside in a given space and does not require the hassle of doing the convergence. Multipass gives dead on accurate filesize every time. When the derived Kbps from the sample, or higher, is used as the average bitrate for multipass, it looks like it actually does a slightly better job than OPV (very slightly-see spreadsheet). Of course it takes at least twice as long as OPV, but OPV requires multiple predictive samples to be run and may require another sizing pass.

But this method is not about solving for an exact size using OPV. Instead, let's say you know you have 2.5 megs of space for your encode. With multipass you would give it the bitrate and forget it. The problem is you will not know if the encode is any good until many hours later. It is the luck of the draw, plain an simple.

If you do a 1% sample at a known Q, you can derive the filesize and the kbps needed for it. You don't need to use the derived Kbps from your sample except as a comparison to what you were going to use. If yours is bigger then continue on with multipass with the clear sight of knowing you will get even better than the Q you sampled.

On the otherhand, if the bitrate you were planning to use is less than the derived kbps from the sample run you did, you know you can not achieve the quality you solved for, and have a way to trigger additional logic or user input. Its all about not being blind.

[bobwillis]

Hi DDogg,

I understand. That's why I said you could let the user specify the Q of extras, so that a 1% OPV sample encode of them could be undertaken, in order to find out what space is required to deliver a pre-determined acceptable quality for the extras. What space is leftover could then be used for the main movie. This would give you the maximum space for the movie, but still deliver acceptable quality for the extras. It's a good idea, because it means valuable space isn't being wasted on extras encoding them at a bitrate which is higher than you actually need or want.

Of course, this forward OPV sampling technique could be used the other way round, sample 1% of the movie to see what bitrate is required to give a pre-determined quality and use the rest for extras.

Regards,
Bob

[DDogg]

Bob, I know you understand. Maybe if you have time we could discuss, and hopefully clarify the logic of what to actually do with the conditional situation established above. While it is not directly specific to DVD-RB, this whole fuzzy thought tangent is jdobbs fault :) because it started as a mental exercise when I saw this log of DVD-RB for xmen2:

-----------------
Phase I, PREPARATION started - Time: 09:15:31
- VTS_05: 2,721,696 sectors.
-- Scanning and writing .D2V file
-- 150,068/75,032 FRAMES/RFFS
-- Building .AVS and .ECL files
- Reduction Level for DVD-5: 28.3%
- Overall Bitrate : 1,490Kbs
Phase I, PREPARATION complete - Time: 09:18:42


A few quick samples showed this would yield a lower quality Q40 equivalent encode. From reading the dvd-rb threads, it seemed jdobbs' whole design premise strives to keep the entire DVD structure intact and keep it one-click simple for inexperienced people.

There were suggestions from people about manually hand setting bitrates, 1/2 D1 resolutions, and so forth, but those all seemed band-aid hacks and contrary to the very smart 'keep it simple' design philosophy jdobbs seemed to be targeting for dvd-rb.

How any application could deal with the space allocation situation shown by the log above, and still keep it transparently automatic and simple for everybody to use, was an intriguing (maybe manic :o) thought process at a personal level.

Creating conditionals based upon quality assessment prior to bitrate allocation seemed a straightforward way to go about it. Hopefully, the data presented in the spreadsheet linked above conclusively proves we can use sampling to pre-analyze source complexity and create a matching derived multipass bitrate which can be used to create the conditional. (more tests need to be done on highly complex source like SPR)

What to actually do with the conditional logic becomes the next question in my mind. Bob, or anybody else that is interested (yeah, right :-)) I've not thought this out well yet, but we have a large menu of options that seem to roughly reside in several areas -

1> Removing stuff - Seems contrary to the goal of recreating the DVD intact. Every item removed breaks a menu at some level. Even removing a soundtrack seems contrary to the goal, but ultimately that is a user decision as most applications allow.

2> Provide more space for main encode by making the non movie stuff like menus, trailers and special features smaller - via re-encoding at a custom bitrate different from main movie/s, lowering resolution, applying compression filters, or combination. Note: Undot and Deen can provide up to a 25% reduction of required bitrate - All could help a lot in the above log situation, but only a little if the DVD structure is simple and just has a long complex movie.

3> More efficiently compress the movie to lower bitrate requirement to achieve better quality - There are a lot of things possible here:
a) Compression filters - Simple Undot() and Deen() can buy up to 25% - May offend purists who think they can see the difference and slows encoding by 30 plus percent.
b) Reduce horizontal resolution to 704 buys about 10% supposedly(not tested by me). Should work on all players (assumption). May be viewed negatively because of perceived compatibility issues.
c) 1/2 D1 for main movie - Yuck!

4> Re-encode those whopping big 448 AC3s? Will besweet do that again?

What else?

[jcclow]

I was just about to post a topic questioning the quality differences between OPV and Multi-Pass VBR. This post has answered most of my questions, albeit raised some others.

The level of knowledge in this forum, never ceases to amaze me. Thanks!