View Single Post
Old 10th April 2004, 02:46   #1  |  Link
DDogg
Retired, but still around
 
DDogg's Avatar
 
Join Date: Oct 2001
Location: Lone Star
Posts: 3,058
Taking the dumbness out of CCE multipass - Revised 6

Summary - (See note 1 & 2 below)

The material below covers two areas.

1> How the complexity, and the resulting compressibility of a video source, can be quantified via the numerical bitrate derived from the physical filesize of a 1% sample where the sample has been created at a known Q value with CCE OnePassVbr (note 3).

Q values normally in the range of 10-40 would be considered the working range to provide excellent to acceptable quality. The goal is for the user to develop a personal quality constant that is always appropriate for the users display methods, and to be able to know in advance of encoding if that level of quality can be achieved. My personal 'quality constant' is Q28


2> How to provide a simple logic so that a program or user can know in advance the approximate quality that will be delivered from a CCE multipass encode, before actually doing the encode.

These goals, and the suggested method to achieve it, is based upon certain educated guesses Bach made since he did not having access to CCE source code (note 2 below). From a practical standpoint, and at the very least, it should provide the user with a very good quality indication before the full encoding process.

Explanation -

This average bitrate number derived from the filesize of the 1% OPV sample is the minimum needed to contain the complexity of the sampled video source and closely deliver the same quality of the OPV sample. This may seem obvious, yet the important fact is that the complexity, and thus the resulting compressibility of the sampled video source can, and has been quantified within a number, the actual bit size of the sample file. This size in bits can then be translated to an average bitrate in Kbps which is compatible with CCE VBR, and perhaps other encoders. This derived number will be refered to as DABR in the rest of the post.

This number, the derived average bitrate (DABR), can be used as a bridge between the two different encoding methods (OPV and VBR), each of which has distinct strengths and weaknesses when standing alone. By bridging these methods we can combine many of the strengths of both.

As a proof exercise, CCE Multipass VBR when using this DABR Kbps number and the same MIN/MAX very closely mirrors the encode that is produced by OPV using a Q number. This is shown in the attached spreadsheet, linked here and attached at the bottom of this post. The accuracy of the translation between Q and DABR expressed as KBPS seems clear.

Now, with a working method of translation, we can combine the strengths of both processes for certain situations.

One situation is where multipass is specified as the encoder tool because of its ease of use, and/or the need for its main strength of creating a final file size exactly on target. While multipass VBR is brilliant in achieving a size target by the nature of its size based process, it has an inherent weakness. It does not possess any internal process that allows it to understand when the ABR, specified by the user, will not be sufficiently high enough to yield a acceptable encode quality.Thus, the final quality of the encode is unknown until after the encode is finished (many hours later).

OPV has opposite characteristics in that quality, via a controlled quantizer approach, is achieved at the cost of non existent size control. OPV is one of the kings of quality and also the fastest software Mpeg2 encoding method available, but without accurate filesize prediction, OPV was only useful for encodes to be held on hard drive as the source complexity solely dictates the bitrate. The final encode size can fluctuate radically, depending on the unique compression characteristics of the individual source, even if those two sources are of the same running times/same total amount of frames. Thus, the final size of the encode is unknown until after the encode is finished .

Predictive sampling, sometimes called 1-PassBach or its iteration, 2-PassBach and best exampled in the D2SRoBa plugin used in combination with Dvd2Svcd/Dvd, was introduced to deal with this weakness in OPV. It uses multiple small samples and solves for the Q which will achieve a target size. Because it solves for Q, which indirectly manipulates bitrate, it effectively provided size control as well as the ability to predict the final quality before the actual encoding process. It does this brilliantly and yields a HQ encode in the shortest time possible. However, Conditional 2Pass Bach as implemented by D2SRoBa took a long time to develop and quietly does many complex actions not readily apparent to the user. The series of prediction samples, and the Newton-Raphson Method calculus (adapted by R6D2) as used by D2SRoBa, are complicated to implement. This can be beyond the scope of many casual hobby programmers, or manual encoding techniques.

Just to clarify, this post is not about using OPV, or any 1 pass method from any encoder to do the final encode. The above paragraph is provided for background. This post is about using only one small and quickly created 1 pass sample to help us lift the source's bitrate 'fingerprint' (DABR). This will show us how to identify, reasonably accurately, the true bitrate required to hold a certain percieved quality in advance of actually doing the full encode. Further, it will help us in any situation where we wish multiple encodes to be placed on one media disk and wish the viewer to enjoy the same approximent quality perception from all encodes on that one disk.

In this situation, the derived average bitrate of a known Q sample as described above, can be used to translate between the two processes. Because the OPV sampling process has actually encoded 1% of the video source as small equally spaced sections, the derived bitrate generated from the sample size inherently contains an awareness of the compressibility of the source. Thus, the D-ABR of the known Q sample is the minimum required bitrate that will be needed to achieve a similar quality in the bitrate based multipass VBR encode. In effect, the process matches the distinct compression characteristics, which are unique to the source complexity, to the appropriate Average Bitrate needed to hold and contain our desired quality. In addition, the identified Derived-ABR (as Kbps) will also provide the framework for creating branching program logic.

The derived Kbps value is used to establish a quality 'GO/NO-GO' conditional.

To do this, an extra step is added when using conventional multipass.

Normally an average bitrate would have been calculated to fill the space available and the encode started, whether the bitrate was adequate, or inadequate, to achieve an acceptable quality level for the particular compression needs of the source. There is not much difference in this method, except before starting the encode, a comparison is made between the Kbps calculated to fill the size available and the derived bitrate from the known quality sample.

GO - When the size calculated bitrate is equal or larger than the derived bitrate from the sample -

The encode is started as normal. This will yield a minimum guaranteed level of known quality within a known filesize. Note: if more space is available for the encode than the minimum requirement predicted by the sample D-ABR, the size calculated ABR will of course be larger than that of the sample's D-ABR. The quality will therefore also increase over that of the known quality sample.

NO-GO - When the size calculated bitrate is less that the derived bitrate from the sample -

Allows the programmer or manual user to detect, in advance of the encode, when a known quality cannot be achieved within a known filesize. Additional logic can then be triggered to deal with this situation such as resolution adjustment, compression filtering, downsizing of audio, etc., to make more space available for the encode.
---------------------------------------------------

Note 1: This is a work in progress as a self learning exercise and is based upon the original propositions posted by Bach and many subsequent revisions by the members of this community, especially those that relate to what we now call 1-Pass and 2-Pass Bach as used in D2SRoBa. This post targets how to apply some of these OPV techniques to CCE Multipass VBR. I highly doubt any of the information here is original, but I do hope this compilation may provide a useful method for those who wish to use CCE multipass and would like to know general quality information in advance of the actual encoding process. Also, I may have not been precise in the use of some of the technical terms, but they are accurate enough for now.

Note 2: This line from the original post is an important part of the assumptions here. However, it was never independently verified as correct, hence the word assumption [all emphasis is mine] - : Originally posted by Bach - "Attention that the Q.factor is an exclusive concept of CCE, directly related with the quantisation, but it does not mean QM nor S, and it is not very clear the way that it is calculated. Based in the results of my tests, I believe that Q.factor is in some way proportional to the average value of the product QM(i, j)*S. Anyway, the Q.factor is the direct measure of the quality to be gotten so that, if two different films are recompressed using the same settings and the same Q.factor, can be guaranteed that they have the same quality."

Note 3: The sampling method is only available when using AviSynth to frameserve video to CCE OnePassVbr (OPV). It assumes the reader understands the use of avisynth and is capable of editing an AVS script, loading the script into CCE, choosing OPV mode and a Q value. To produce an AVS for creating the sample, simple add this line at the bottom of your working script: SelectRangeEvery(1200,12). Obviously, this line must be removed after the sample has been created.

You will need the derived bitrate spreadsheet calculator, linked here, or attached to the NEXT post below, OR use this formula - (((Sample_Size_In_Bytes*(100/Sample_Size_Percentage))*8)/1000)/(Total_Frames/Frame_Rate)
Attached Files
File Type: zip opvq2.zip (6.0 KB, 2541 views)

Last edited by DDogg; 8th October 2006 at 13:20.
DDogg is offline   Reply With Quote