RB v1.26 bitrate redistribution

[archaeo]

@jdobbs:
Just wondering how the Base_Q is determined in the new feature... Is it done by using an OPV prediction pass with a given sample size, or a set value?

I just ran a quick test and ticked the enable bitrate redistribution, and RB came up with a Base_Q = 31. Not knowing where this value came from, I ran the same project through RB-Opt with 'calculate optimal Q' chosen, and the Q it gave me was 48.

Is the Base_Q set at a particular value in RB?


thanks

[jdobbs]

I used a routine I had already set up in OPV prediction. It generates an initial Q value based upon the desired output size and frame count. It wouldn't be exact (of course -- that takes a few passes) but gets you close enough to be within a reasonable distance of the output Q so you're getting a reliable bitrate distribution profile. Originally my plan was to use a fixed value, but I decided this would be better based upon Sharc's notes in this thread (http://forum.doom9.org/showthread.php?t=122351&page=10).

The Base_Q will change with each disc.

By the way. Currently the redistribution is applied only against the main feature. I have it set up, though, so I can have it applied against extras as well (if I decide that's a good path to take).

[Sharc]

@jdobbs:
Am I correct in assuming that the matrix specified as "Main Feature Matrix" is used for the determination of the Base_Q value?
As mentioned before, the Q depends largely on the matrix for a fix target size.

[archaeo]

OK thanks. So it is calculated... and as long as the calculated Base_Q is in the 'ballpark', the distribution profile won't be markedly different from one done using an exact Q prediction pass routine.


I used a routine I had already set up in OPV prediction. It generates an initial Q value based upon the desired output size and frame count.

And I assume that this routine also take into account the custom matrix that is selected in the advanced options?

By the way. Currently the redistribution is applied only against the main feature...

Good to know :) I could see the value to having it as an option to apply to extras as well.

[Sharc]

Oh I think I misunderstood.
The Base_Q in DVD-RB 1.26.0 is a rough estimate which does not yet consider the actual matrix. Correct?

[Sharc]

Here a comparison between DVD-RB 1.26.0 and RB-Opt for redistribution.

Matrix used is FHE, same as the original DVD.

RB-Opt calculated a Q factor of 58, corresponding to a DVD-5 target, for redistribution.
DVD-RB calculated a Base_Q of 31 for redistribution.

Sample size is 25% in both cases.

The results are similar, but as expected the Q=58 produced a somewhat higher swing, because at Q=31 the redistribution is not yet in the "saturated" range for the FHE matrix. Still, I don't think that one would notice a difference when watching the movie.

If I had used a lower bitrate matrix like CCE default, I would expect that the results would be very much the same for DVD-RB and RB-Opt, because at Q=31 the redistribution is typically in the "saturated" range, for the CCE default matrix.

("Saturated" means that changes in Q have little impact on the redistribution profile.)

P.S.: Could someone approve my former attachements in the RB-Opt thread and Redistribution thread?

[jdobbs]

Not sure what movie you're using, but I'd have to guess 31 would be closer to the actual Q factor needed. Run an OPV prediction and see what the end Q is calculated to be. I wouldn't think that you'd actually want to get into the "saturation" range... you want to stay where the encoder is using it's best judgement for the source being encoded.

[Sharc]

I will do this, and just noticed a minor issue: I can select both "Enable Redistribution" and "One pass VBR" at the same time in the RB menu. They should be exclusive, I believe.

[Sharc]

Here the results for OPV:

Movie is The Departed/PAL, Matrix is FHE.

DVD-RB: Q final = 60 (31, 52, 60), 4400 frames sample size (2%)
RB-Opt: Q final = 58 (from earlier tests), 2200 frames sample size (1%)

High bitrate matrices will generally end up in very high Q values.

[jdobbs]

I will do this, and just noticed a minor issue: I can select both "Enable Redistribution" and "One pass VBR" at the same time in the RB menu. They should be exclusive, I believe.They are... if you select OPV the Redistribution is ignored. I guess I should disable it.

[jdobbs]

Here the results for OPV:

Movie is The Departed/PAL, Matrix is FHE.

DVD-RB: Q final = 60 (31, 52, 60), 4400 frames sample size (2%)
RB-Opt: Q final = 58 (from earlier tests), 2200 frames sample size (1%)

High bitrate matrices will generally end up in very high Q values.
What happens if you use a standard matrix? That sure seems to be a high Q factor for something with a bitrate averaging in the mid 3000's...

Maybe I missed something earlier in the thread, but why would you want to use a matrix designed for a high bitrate when you are in this range?

[Sharc]

What happens if you use a standard matrix? That sure seems to be a high Q factor for something with a bitrate averaging in the mid 3000's...

Maybe I missed something earlier in the thread, but why would you want to use a matrix designed for a high bitrate when you are in this range?

That's another story: I used the original matrix in order to demonstrate that the combination of the original matrix with a certain (typically low) Q will closely "reproduce" the profile and file size of the original disk. I reported this here:
http://forum.doom9.org/showthread.php?p=1012203#post1012203

Secondly, if you do a standard multipass VBR with CCEAQM=1 you will often see the matrix switching to 1/2 CCE default which is +/- similar to FHE.

[Fishman0919]

Secondly, if you do a standard multipass VBR with CCEAQM=1 you will often see the matrix switching to 1/2 CCE default which is +/- similar to FHE.

But that is per GOP when the encoder feels it's needed or thinks it will be beneficial to the encoding. A matrix like FHE is design for high bitrate (5000k and up), using it on a bitrate in that range will most likely not yeild result as good as the default or even the Standard MPEG2 matrix

[Sharc]

Yes, agree.

[Sharc]

ok, here the OPV results for CCE default matrix, same movie:

DVD-RB: Q final = 19 (31, 19, 18 => 19), 4400 frames sample size (2%)
RB-Opt: Q final = 17 (from earlier tests), 2200 frames sample size (1%)

... and, btw:
RB-Opt using Q=17 with CCE default matrix yields exactly the same distribution profile as with FHE and Q=58.

[Boulder]

But that is per GOP when the encoder feels it's needed or thinks it will be beneficial to the encoding. A matrix like FHE is design for high bitrate (5000k and up), using it on a bitrate in that range will most likely not yeild result as good as the default or even the Standard MPEG2 matrixRemember that you need to consider the compressibility of the source along with the available bitrate. You can't make any accurate judgments based on the bitrate alone.

[linx05]

I have the new DVD Rebuilder version with the Bitrate Distribution. Now here are the questions:

1) How do I know when to know the best possible time to use it?

2) Is it good to leave it enabled all the time (supposing all the bugs have been ironed out)?


EDIT: 3) Does it make a difference if the DVD is a movie or episodic disc?

EDIT: 4) What difference does it make to change the hidden setting "Redist_Percent=" ?

[gurkan]

@jdobbs
Does the redistribution take settings like "steal space from extras" and "half d1" in consideration before the q-value is predicted?

[jdobbs]

It couldn't take "Half Space for Extras" into account -- since that is a reallocation of the result (just like the original bitrate) and therefore would be a contradiction. But it would take "Half-D1" into account.

[jdobbs]

I have the new DVD Rebuilder version with the Bitrate Distribution. Now here are the questions:

1) How do I know when to know the best possible time to use it?

2) Is it good to leave it enabled all the time (supposing all the bugs have been ironed out)?


EDIT: 3) Does it make a difference if the DVD is a movie or episodic disc?

EDIT: 4) What difference does it make to change the hidden setting "Redist_Percent=" ?1. Let's wait and see what results are posted from the beta. But I've been doing all mine with it set so I can see how well it works.

2. If it turns out that it improves the picture in the reports -- it could be a good thing to leave on.

3. It shouldn't be used with an episodic disc because it currently only works on the largest VTS. But, I have a version I'm using for testing that has a flag that allows you to use it across all VTSs.

4. The larger the sample, the more accurate the distribution, and the longer it takes. But you'll probably find that it is very close anyway, even with the default 10%.

[linx05]

Thanks for the replies. I had a feeling it wasn't working for episodic discs.

EDIT: If I did a prepare phase, it did what it should do (along with distribution) and then went into the preview/edit tab and changed a few bitrates for the credits for example, would this have a negative effect on it?

I am still a little fuzzy on this after reading Boulders thread again.

[Boulder]

Currently the method does work on episodic discs if the different episodes are in the same VTS.

[jdobbs]

Thanks for the replies. I had a feeling it wasn't working for episodic discs.

EDIT: If I did a prepare phase, it did what it should do (along with distribution) and then went into the preview/edit tab and changed a few bitrates for the credits for example, would this have a negative effect on it?

I am still a little fuzzy on this after reading Boulders thread again.
If you use the preview/edit to change bitrates, you are overriding the decision made by redistribution.

[linx05]

It would be good to have it both ways. But such is life :)

[archaeo]

Well, I would like to report one test project with a sizing problem using the new beta feature... For this test I did a comparison in which I first ran a project through DVDRB's redistribution feature, and then ran the same project w/ the same parameters thru RB-Opt (v.29) redistribution using 'calculate optimal Q'. The result was that DVDRB's final project size came in at 4.20 Gb, and RB-Opt's redistribution final size came in at 4.34 Gb. All the initial parameters were set up the same for both projects. The main difference I saw was with the base_Q value - DVDRB calculated it at 23, and RB-Opt Q calculated it at 40 using 'calculate optimal Q' feature. I used a 10% sample size for RB-Opt. The disparity in base_Q results appears to be where the sizing difference is coming from.


For these I applied a custom matrix in both (4000+). I need to add that I ran the same project thru DVDRB using the standard matrix (thinking that saturation was a problem), and it was still undersized. I used one filter, undot(), and used 2 passes in CCE.


Here's DVDRB's log:
------------------------
[20:58:33] Phase I, PREPARATION started.
- DVD-RB v1.26.0
- AVISYNTH 2.5.6.0
- CCE 2.70.2.12 encoder selected.
- AVS Filters are enabled.
- "Adaptive Quantizer Matrices" is enabled.
- "Movie Only" mode is enabled.
- Source: LAST_KING_OF_SCOTLAND
- VTS_01: 2,659,504 sectors.
-- Scanning and writing .D2V & .AVS files
-- Processed 176,993 frames.
-- Building .AVS and .ECL files
- Reduction Level for DVD-5: 85.8%
- Overall Bitrate : 5,368/4,295Kbs
- Space for Video : 3,870,030KB
- Redistributing using Base_Q: 23
- HIGH/LOW/TYPICAL Bitrates: 5,877/1,178/4,295 Kbs
[21:06:22] Phase I, PREPARATION completed in 8 minutes.
[21:08:21] Phase II ENCODING started
- Creating M2V for VTS_01 segment 0
- Creating M2V for VTS_01 segment 1
- Creating M2V for VTS_01 segment 2
- Creating M2V for VTS_01 segment 3
- Extracting Video for VTS_01 segment 4
- Extracting Video for VTS_01 segment 5
- Extracting Video for VTS_01 segment 6
- Extracting Video for VTS_01 segment 7
- Creating M2V for VTS_01 segment 8
- Creating M2V for VTS_01 segment 9
- Extracting Video for VTS_01 segment 10
- Extracting Video for VTS_01 segment 11
- Extracting Video for VTS_01 segment 12
- Creating M2V for VTS_01 segment 13
- Creating M2V for VTS_01 segment 14
- Creating M2V for VTS_01 segment 15
- Extracting Video for VTS_01 segment 16
- Extracting Video for VTS_01 segment 17
- Extracting Video for VTS_01 segment 18
- Creating M2V for VTS_01 segment 19
- Extracting Video for VTS_01 segment 20
- Creating M2V for VTS_01 segment 21
- Creating M2V for VTS_01 segment 22
- Creating M2V for VTS_01 segment 23
- Extracting Video for VTS_01 segment 24
- Creating M2V for VTS_01 segment 25
- Creating M2V for VTS_01 segment 26
- Creating M2V for VTS_01 segment 27
- Creating M2V for VTS_01 segment 28
- Creating M2V for VTS_01 segment 29
- Creating M2V for VTS_01 segment 30
- Creating M2V for VTS_01 segment 31
- Creating M2V for VTS_01 segment 32
- Creating M2V for VTS_01 segment 33
- Creating M2V for VTS_01 segment 34
- Creating M2V for VTS_01 segment 35
- Creating M2V for VTS_01 segment 36
- Creating M2V for VTS_01 segment 37
- Extracting Video for VTS_01 segment 38
- Extracting Video for VTS_01 segment 39
- Extracting Video for VTS_01 segment 40
- Creating M2V for VTS_01 segment 41
- Creating M2V for VTS_01 segment 42
- Extracting STILLS for VTS_01 segment 43
[21:44:26] Phase II ENCODING completed in 36 minutes.
[21:44:26] Phase III, REBUILD started.
- Processing VTS_01
- Reading/processing TMAP table...
- Rebuilding seg 0 VOBID 1 CELLID 1
- Rebuilding seg 1 VOBID 1 CELLID 2
- Rebuilding seg 2 VOBID 1 CELLID 3
- Rebuilding seg 3 VOBID 1 CELLID 4
- Rebuilding seg 4 VOBID 1 CELLID 5
- Rebuilding seg 5 VOBID 1 CELLID 6
- Rebuilding seg 6 VOBID 1 CELLID 7
- Rebuilding seg 7 VOBID 1 CELLID 8
- Rebuilding seg 8 VOBID 1 CELLID 9
- Rebuilding seg 9 VOBID 1 CELLID 10
- Rebuilding seg 10 VOBID 1 CELLID 11
- Rebuilding seg 11 VOBID 1 CELLID 12
- Rebuilding seg 12 VOBID 1 CELLID 13
- Rebuilding seg 13 VOBID 1 CELLID 14
- Rebuilding seg 14 VOBID 1 CELLID 15
- Rebuilding seg 15 VOBID 1 CELLID 16
- Rebuilding seg 16 VOBID 1 CELLID 17
- Rebuilding seg 17 VOBID 1 CELLID 18
- Rebuilding seg 18 VOBID 1 CELLID 19
- Rebuilding seg 19 VOBID 1 CELLID 20
- Rebuilding seg 20 VOBID 1 CELLID 21
- Rebuilding seg 21 VOBID 1 CELLID 22
- Rebuilding seg 22 VOBID 1 CELLID 23
- Rebuilding seg 23 VOBID 1 CELLID 24
- Rebuilding seg 24 VOBID 1 CELLID 25
- Rebuilding seg 25 VOBID 1 CELLID 26
- Rebuilding seg 26 VOBID 1 CELLID 27
- Rebuilding seg 27 VOBID 1 CELLID 28
- Rebuilding seg 28 VOBID 1 CELLID 29
- Rebuilding seg 29 VOBID 1 CELLID 30
- Rebuilding seg 30 VOBID 1 CELLID 31
- Rebuilding seg 31 VOBID 1 CELLID 32
- Updating NAVPACKS for VOBID_01
- Rebuilding seg 32 VOBID 2 CELLID 1
- Rebuilding seg 33 VOBID 2 CELLID 2
- Rebuilding seg 34 VOBID 2 CELLID 3
- Rebuilding seg 35 VOBID 2 CELLID 4
- Rebuilding seg 36 VOBID 2 CELLID 5
- Rebuilding seg 37 VOBID 2 CELLID 6
- Rebuilding seg 38 VOBID 2 CELLID 7
- Rebuilding seg 39 VOBID 2 CELLID 8
- Rebuilding seg 40 VOBID 2 CELLID 9
- Rebuilding seg 41 VOBID 2 CELLID 10
- Rebuilding seg 42 VOBID 2 CELLID 11
- Updating NAVPACKS for VOBID_02
- Rebuilding seg 43 VOBID 3 CELLID 1
- Updating NAVPACKS for VOBID_03
- Updated VTS_C_ADT.
- Updated VTS_VOBU_ADMAP.
- Updated IFO: VTS_01_0.IFO
- Correcting VTS Sectors...
[21:52:11] Phase III, REBUILD completed in 8 minutes.

Done.
_______________________

[jdobbs]

The difference in Q couldn't have any affect at all. The total amount of space stays the same (within a few sectors) between a regular encode and a reallocated encode... it just gets distributed differently. The Q value isn't used in the encode, only in determining how to spread the available sectors.

So that just indicates that the encoder is not reaching the specified bitrate in the encode.

[archaeo]

Oh yes, you're right, it's only used in the bitrate distribution calculation... I'm wondering why then, when using the same parameters for each project, RB-Opt's redistribution process allows it to come in right on the money? In other words, what difference in RB-Opt's redistribution process would allow the encoder to reach the specified bitrate?

[Sharc]

The only explanation I would have is that RB-Opt produced a significantly higher swing (by it's higher Q) than DVD-RB. See my previous posts and graphs in this thread.
Hence the encoder may run less into a saturation with RB-Opt's redistribution profile.

Edit:
I just noticed that you have CCEAQM enabled. In my experience this can produce sizing problems when used with 2 passes only. I always use 3 passes with CCEAQM=1.
Why RB-Opt hit the target size better may -- as said before -- be related to the higher swing which also has an effect on when the matrix is modified by CCEAQM.

[Boulder]

Maybe the calculations are simply different? With RB-Opt's redistribution, I've always got a similar result to a regular DVD-RB encode.

[Sharc]

They are certainly different. This becomes most evident for high bitrate matrices when you get Q in the order of 30 with DVDRB as opposed to Q in the order of double that value (60) with RB-Opt.

I understand that DVDRB estimates the Base_Q based on number of frames etc., whereas RB-Opt makes the estimation of the Q by repeated trials using the actual matrix -- similar to the OPV process.
The DVD-RB method is faster though, however the RB-Opt method produces a Q-estimate which is closer to the one that corresponds to the target size.
DVD-RB and RB-Opt produce similar Q estimates for the standard matrix, however the Q estimate may differ significantly (by a factor of 2) for high bitrate matrices according to my experience.

[Sharc]

Here another example.
- The matrix of the original is FHE.
- Matrix for redistribution is CCE default.
- a) RB-Opt calculated Q =10
- b) DVD-RB calculated Base_Q = 24
- Redistribution sample size set to 25% in each case.

As expected, the higher Q (24) of DVD-RB introduced a higher swing, however the Q which corresponds to a DVD-5 is actually 10, i.e. same as would be used for OPV.

This example is one of the (rare?) cases where the redistributed bitrate exceeded the original bitrate for some of the cells.
I assume that DVD-RB would reset these peaks to the bitrate of the original which may finally be the reason for undersizing (?). In such case the undersizing would be higher for b) than for a).

Btw., the huge swing brings me back to the question if further (marginal) improvement would be possible by a final matrix reallocation, see
http://forum.doom9.org/showthread.php?p=1013444#post1013444

[jdobbs]

Just MHO, but I think we may be putting the cart before the horse.

Instead of comparing the two (DVD-RB and RB-Opt), I think it would be more beneficial to do tests that might help decide whether there is any real advantage to the redistribution on non-CBR/limited-VBR sources. Then if it looks like there is we can start optimizing.

As for CBR and limited-VBR sources I think it's already decided. So my concentration would probably be on examining the bitrate of the source (not the segments, but the picture level) and doing the redistribution automatically.

...peaks to the bitrate of the original which may finally be the reason for undersizingThat may be possible... I'll take a look at it. The allocated bitrate should never exceed the peak bitrate of the original. It won't in the encoding (because the max_bitrate will limit it) -- but I'll have to look and see if there is the possibility of bits being lost there. Probably not, because the peak is going to be significantly higher than the average (which you graph is displaying, except on CBR).

Also, looking at the way is distributed in your graphs -- I wouldn't think the difference in swing between the two distribution methods is really enough to be significant.

[jdobbs]

Oh yes, you're right, it's only used in the bitrate distribution calculation... I'm wondering why then, when using the same parameters for each project, RB-Opt's redistribution process allows it to come in right on the money? In other words, what difference in RB-Opt's redistribution process would allow the encoder to reach the specified bitrate?Are you using non-standard matrices in your encodes?

[jdobbs]

Btw., the huge swing brings me back to the question if further (marginal) improvement would be possible by a final matrix reallocation, see
http://forum.doom9.org/showthread.ph...44#post1013444I think Robot1 is right there. You have to run the redistribution pass on the matrix that was selected -- otherwise the Q wouldn't match the matrix and the redistribution might be incorrect to begin with. Something I didn't think of is that when DVD-RB does the Q check, it only uses the "Main Feature" matrix -- because it treats the encode like an OPV pass. But when the 2-pass encode is completed, it uses the specified matrices for encoding. That's why I asked archeao if he were using alternate matrices on the disc that undersized.

[Sharc]

I think Robot1 is right there. You have to run the redistribution pass on the matrix that was selected -- otherwise the Q wouldn't match the matrix and the redistribution might be incorrect to begin with.

No question here, agree.

The idea was rather if it would make sense to re-allocate the matrices based on the final redistributed profile, just before starting the subsequent multipass encoding phase.
I have done this manually in a test for a segment that got a really low bitrate allocated by the redistribution, and found that by re-allocating the "low bitrate matrix" manually the quality of the picture improved marginally (less macroblocking). Well, just an idea. May be I need to do more tests.

[jdobbs]

DVD-RB does select the matrix based upon the reallocated bitrate, because the ECL write is done after redistribution. So that much is ok. Except, of course, that the reallocated bitrate was determined without the matrix. That may be why encoder saturation could be more likely.

The problem is that at the point of redistribution, you don't know yet what the redistributed bitrate will be yet... so you can't use it to determine a matrix to use for the redistribution pass -- it's a "chicken before the egg" scenario. I'm wondering if it may be best to force the "Main Matrix" matrix across the board when a redistribution was done.

I have doubts as to whether the matrix originally selected by DVD-RB should be used in the redistribution with RB-Opt either, because there are some significant bitrate differences. I'm not sure if Robot1 changes it, but I don't think so.

[Sharc]

....But when the 2-pass encode is completed, it uses the specified matrices for encoding.

Does it use the matrices according to the redistrubuted bitrate? Means if before redistribution the cell bitrate is say 2700 and the redistribution pass would set it to 1900, would then the "very low bitrate" matrix be employed for the multipass encode?

Edit: We were just posting at the same time. Your last post clarified my qestion above. Thanks

[jdobbs]

Yes. But that bitrate (the 1900 in your example) was allocated based upon redistribution with a Q and a different matrix... that's the problem (whether you do it with RB-Opt or DVD-RB).

[Sharc]

Agree.

[robot1]

In the graph above, I think the value calculated by RB-opt is higher than the original bitrate. In this case, RB-Opt doesn't use it, but uses the original bitrate, and the saved bitrate is spreaded among the other cells (and I think DVD-RB acts in the same way). This behaviour can be changed in RB-Opt.ini, but I don't recommend to change it.

If there are different matrices in the segments, RB-Opt uses the matrix of the "average segment" for the prediction, and then uses the actual matrices of every segment for the redistribution pass.
Anyway, I'm convinced that if you want to use redistribution, you should use the same matrix for the whole movie.

("average segment" is the segment whose bitrate is more similar to the average bitrate of the movie)

[Boulder]

Instead of comparing the two (DVD-RB and RB-Opt), I think it would be more beneficial to do tests that might help decide whether there is any real advantage to the redistribution on non-CBR/limited-VBR sources. Then if it looks like there is we can start optimizing.One thing that comes to my mind immediately is that doing the redistribution will nullify any effects of a possible bias setting in the original encode. I think that the bias is one reason why the redistributed bitrate swings a lot compared to the original.

[jdobbs]

Good point. That's definitely possible.

[robot1]

Anyway with an average bitrate over 4.000 I don't think you will see any visual difference between a regular and a redistribuited encoding. Redistribution could be useful for lower encoding, I think.

[jdobbs]

Anyway, I'm convinced that if you want to use redistribution, you should use the same matrix for the whole movie. I agree. Otherwise you're very likely negating any advantage you might gain from the redistribution.

[archaeo]

Are you using non-standard matrices in your encodes?


Yes, I used the high (4000+) matrix to encode with, for 'main feature' , 'low', 'very low' and 'extras'.

[jdobbs]

That probably explains it then. I'm working on a fix for that.

[archaeo]

Anyway with an average bitrate over 4.000 I don't think you will see any visual difference between a regular and a redistribuited encoding. Redistribution could be useful for lower encoding, I think.

Yes, true in this case, as I wasn't able see a noticeable difference between these two. But in response to a question earlier in this thread by linx05 (where he asked if it was a good idea to leave it enabled 'all the time'), I thought I'd throw a non CBR hi bitrate source at it to see how it would work out in this case. Since sizing did become an issue with this particular sample, it seemed to help answer that question.

[robot1]

Probably I've done less tests than yours. From my tests, the redistribution has never damaged image quality (but sometimes it hasn't improved it).
I think an advanced user could leave always on the redistribution, looking at the results and tweaking the cells with bitrate spikes using the segment editor.
But, for a normal user, and a normal encoding (not CBR or limited VBR), the standard DVD-RB mode could be really fine, expecially if the average bitrate is over 3.500

[Sharc]

One thing that comes to my mind immediately is that doing the redistribution will nullify any effects of a possible bias setting in the original encode. I think that the bias is one reason why the redistributed bitrate swings a lot compared to the original.
I see the "nullification" of the bias as a clear advantage of redistribution. I imagine that if one has plenty of space -- like on a DVD-9 -- one might be tempted to increse the bias. More even so because the maximum bitrate is limited anyway by the mpeg/DVD standard or by encoder saturation. so why not simply boost the less demanding segments on a comfortable DVD-9.
The situation changes when one wants to reduce such a high-bias DVD-9 to a DVD-5. Then it becomes essential for which scenes to spend the high bitrates and where one can tolerate lower bitrates. Here comes the benefit of redistribution.

[Sharc]

In the graph above, I think the value calculated by RB-opt is higher than the original bitrate. In this case, RB-Opt doesn't use it, but uses the original bitrate, and the saved bitrate is spreaded among the other cells ....
This makes sense to me.

.....(and I think DVD-RB acts in the same way).....
Not sure. May be jdobbs will tell. I thought that DVD-RB will just ensure that the instantaneous bitrate will never exceed that of the original, but it won't touch the average cell bitrate.

Anyway, I'm convinced that if you want to use redistribution, you should use the same matrix for the whole movie.
Means you recommend to use one single matrix for initial Q-estimation, then for the redistribution pass itself and for the encoding?
I agree insofar as this will preserve the concept of "constant quality" best.

[Sharc]

As the principal question is still when to go for redistribution in case of "regular VBR disks", I personally tend towards saying "always", because my expereince with redistribution has been "equal or better than regular multipass", in particular since the handling of tiny cells has been addressed.

I attach here an example (WTC) where the swing increase by redistribution is actually huge. I was most suspicious about the qaulity of cell 9, 10 with bitrates in the range of 1000 kbps using CCE at this extraordinary low bitrate. To my surprise I didn't find any noticeable deficiencies in these (dark) scenes when watching the encode. However a loss of sharpness/details is noticeable when making a zoomed picture-by-picture comparison with the original (I did not compare with a standard VBR re-encode though).

I used one single matrix (similar to CCE default) throughout the process i.e. for Q estimation, redistribution pass, and final encoding. I used RB-Opt for redistribution. I will repeat with RB 1.26 but I wouldn't expect meaningful differences.

Noteworthy perhaps is the matrix of the original, which I consider to be quite exceptional:

Intra Luma and Chroma Matrix
8 5 7 9 11 13 14 17
5 6 8 11 13 14 17 18
7 9 10 11 14 17 17 19
9 10 13 13 14 17 18 127
10 13 13 14 16 17 127 127
13 13 14 16 17 20 127 127
13 13 14 17 19 23 127 127
13 14 17 19 127 127 127 127

NonIntra Luma and Chroma Matrix
8 5 7 9 10 10 11 11
5 6 9 10 10 11 11 12
7 9 10 10 11 11 12 12
9 10 10 11 11 12 13 127
10 10 11 11 12 13 127 127
10 11 11 12 13 13 127 127
11 11 12 13 13 14 127 127
11 12 12 13 127 127 127 127

[Sharc]

..... I'm wondering if it may be best to force the "Main Matrix" matrix across the board when a redistribution was done.........

I have doubts as to whether the matrix originally selected by DVD-RB should be used in the redistribution with RB-Opt either, because there are some significant bitrate differences. I'm not sure if Robot1 changes it, but I don't think so.

Anyway, I found that the redistribution profile becomes widely independent from any selected matrix as long as we set the "Base_Q" to a value that would yield a DVD-5 target size in OPV for the given matrix.

@jdobbs:
I understand that the Base_Q -- without iteration -- is used for the redistribution pass, correct? In my tests this Base_Q normally deviated significantly from the OPV value that would yield a DVD-5, even for the standard CCE default matrix. Perhaps something to look into.

[jdobbs]

Base_Q woud not be the final Q value, it would be the start point. In order to get the final figure I would have to do all the prediction passes -- and that just isn't justified in this circumstance, because the difference in redistribution would be negligible.

[kumi]

I think DVD-RB 1.26 redistribution with HC 0.21 has a problem when calculating BaseQ for matrices other than the encoder default. The calculated BaseQ for a redistribution using a high-bitrate matrix (the one last posted by Sharc) is 21, but the calculated BaseQ in RBOpt for that same matrix is 43. The curve seems to return to normal in DVD-RB when I use the HC default MPEG matrix:

http://img508.imageshack.us/img508/3430/hc021redistdvdrbvsrboptxi2.png

EDIT: I see now that archaeo already posted about this problem.

[Sharc]

As reported earlier in this thread and in the other thread about redistribution, DVD-RB and RB-Opt use different methods for the estimation of the Q for redistribution:
- DVD-RB uses the initial Q estimate (= Base_Q) of the OPV method without further iteration.
- RB-Opt uses the final Q value after some iterations which takes more time but ends with the Q which corresponds to the DVD-5 target size for the applied matrix.

Depending on the matrix the 2 methods can deliver substantially different values for the Q, which in turn may result in substantially different redistribution profiles, typically for - but not limited to - the case of high bitrate matrices. For example a low Q can even produce a redistribution profile which is very similar to the standard multipass profile. There are however cases, e.g. for "average" matrices, where the resulting profiles are similar for both methods because there exists a kind of "saturation" for the Q where the distribution profile becomes insensitive of the Q.

I personally would prefer if DVD-RB would have the option of using the "final Q" after few iterations rather than the Base_Q.
May be this is something for optimization.

Added:
Anyway, in your example with an average bitrate in the order of 5000 kbps I don't expect you will get any benefit from redistribution, whatever method/profile you apply.

[kumi]

As reported earlier in this thread and in the other thread about redistribution, DVD-RB and RB-Opt use different methods for the estimation of the Q for redistribution

I see, so you're saying that the deviations between "DVDRB Hi BaseQ 21" and "DVDRB MPEG BaseQ 21" in my graph are a result of the DVDRB estimation routines, and not a bug?



I personally would prefer if DVD-RB would have the option of using the "final Q" after few iterations rather than the Base_Q.

Is there any possible benefit from not performing additional iterations, besides greater speed?

[Sharc]

I see, so you're saying that the deviations between "DVDRB Hi BaseQ 21" and "DVDRB MPEG BaseQ 21" in my graph are a result of the DVDRB estimation routines, and not a bug?
Exactly, that's what I believe. As a test you may set the Q of RB-Opt to the DVD-RB value and you will most likely get the same profile with RB-Opt as with DVD-RB.

Is there any possible benefit from not performing additional iterations, besides greater speed
In my opinion no, but may be I don't see the full picture.

Added: I should mention that my experience is with CCE rather than HC.

[Sharc]

When the redistributed bitrate of a segment exceeds the bitrate of the original disk, the segment is marked for "no reencode".
Will the filters defined in the Filter Editor be skipped in this case?

Edit:
Sorry, I just noticed that the similar question has already been put in this thread (in the context of possible undersizing):
http://forum.doom9.org/showthread.php?p=1016359#post1016359

[jdobbs]

[Removed]

[jdobbs]

After much testing I'm satisfied that this method doesn't hurt the picture, and can improve it under some circumstances. The only bug report I have is related to ILVU, and I've corrected that for the next version -- so I'm going to remove it from beta status.

:)

[acrespo]

Can I use redistibute encode with CCE Basic?
What's happen if I enable redistribute encode and use CCE Basic?
I asking this because as I know, CCE Basic don't support OPV mode.

[Boulder]

@jdobbs: regarding CCE Basic, would it be a reasonable option to allow using one encoder for redistribution and other for the final encode? In this case, HC would do the redistribution phase and CCE Basic would then do the final phase.

[jdobbs]

I'd have to do some comparisons of the bitrate distribution for CCE and HC. One would hope they'd be close -- but I haven't tried it to see.

[kumi]

I've compared CCE, HC and ProCoder on 4 discs and the redistributed bitrates of HC and CCE were essentially identical in all four cases. This was with RBOpt, not DVD-Rebuilder, mind you.

EDIT: I can see the value for ProCoder projects, if jdobbs ever adds support for that. The CQ scale or whatever it is in ProCoder causes some pretty wonky redistributions, at times.

[Sharc]

I did a comparison between CCE and QuEnc. The redistribution profiles were virtually identical. I used the default matrix.
See this post:
http://forum.doom9.org/showthread.php?p=1005914#post1005914

[laserfan]

I need someone to tell me in simple terms what this is about and when/how I would use it.

I thought it had something to do with converting DVDs that were encoded at a set-and-forget constant bitrate, and so DVD-RB should leave certain scenes alone to not detract further from the borderline they were encoded at.

Am I close? If not, what the heck is this about? How does one determine which DVDs to use it with, or does one just select the option and leave it on all the time?

I don't THINK I'm stupid, but I have not followed this discussion closely enough to appreciate it... :confused:

[kumi]

Here's how I understand it:

Redistribution re-calculates the optimal bitrates on a segment-to-segment basis, using a constant Q factor as the base. And since there is a (admittedly loose) correlation between "Q" and perceived visual quality, an ideal redistribution will result in a DVD5 where all segments look equally "good" to your eyes.

Now, the DVD studio that mastered your DVD9 has a larger pool of bits to work with than you do. At DVD9 sizes, swings in the video's Q affect perceived visual quality less than at DVD5 sizes. Because of this, mirroring the original bitrate distribution with a VBR re-encode might result in greater perceived visual quality inconsistencies from segment to segment. Bitrate redistribution "equalizes" these inconsistencies, (ideally) resulting in a output that is at maximum quality, throughout the entire movie.

I really can't think of any cases where redistribution could actually be worse than VBR. Maybe someone else can.

[jdobbs]

On typical sources I personally can't tell the difference, on or off. But I guess the bottom line is: I haven't seen it hurt the output on any sources yet -- but I have seen it help. On CBR or limited VBR sources it seems to make a significant difference. So I use it on all my encodes now.

As to whether you add the "redist_All=1" parameter, that's still debatable, because it could conceivably steal bits from the main movie in order to bring the visual quality of the extras to the same level as the feature.

[blutach]

Just a quick question - does multiple encoder processes not work with the "redistribution pass"? Just trying it for the first time (albeit on a source which would not be improved 1 iota) and it seems to take a while using a single processor.

Regards

[jdobbs]

No. The way they are called is different. Redistribution will only do one segment at a time.

[laserfan]

I just tried my first backup w/1.26.1, BD selected, and Full Movie/Menus/Extras and have some questions. At the end of the Prepare stage, a file is written called REDISTRIBUTION.TXT wherein it appears to show before & after bitrates by segment. But following the Prepare stage, I used the Editor to blank a bunch of Extras, and then re-assign the saved space to the Feature.

It all appears to have worked perfectly i.e. it exactly hit the DVD-5 target size but I wonder: After I blank some extras and give the feature more space, doesn't that render the REDISTRIBUTION.TXT information invalid? Or does RB just change all the target bitrates based on the ratios in the original analysis? Or am I misunderstanding the process (very possible!). :) :confused:

[jdobbs]

It will reallocate the recovered space just like it always did -- but the new distribution will be used as the baseline. So the benefit of the redistribution is retained.

[jdobbs]

[Remove - wrong thread]

[laserfan]

...the benefit of the redistribution is retained.:thanks:

[jdobbs]

Just as an add-on note to substantiate some of the experiences reported on redistribution. The list below is the result of an encode I did on an old 1960's series recently. Notice that the distribution for all the segments was virtually identical in the original encode... probably fine when you have a lot of bandwidth (DVD-9) to play with. But running redistribution showed a significant difference in the required bitrate -- and the picture is visibly better.

This is an example of where this method shines. I've decided that I'm personally going to just leave this option enabled all the time -- that way I don't have to do any kind of preliminary analysis... just click-and-go.

Hat's off to Boulder for exploring this concept, and to Sharc and others for testing/proving... even though I was originally a little resistant to the idea, I've very pleased with the result.

:thanks:

BITRATE APPLIED TO FEATURE SEGMENTS (SEGMENT ORIGINAL REDISTRIBUTED)
(Created using a 10% sample)

V01000000001001 1,799 Kbs 2,331 Kbs
V01000100001002 1,798 Kbs 1,845 Kbs
V01000200001003 1,798 Kbs 2,391 Kbs
V01000300001004 1,798 Kbs 2,008 Kbs
V01000400001005 1,798 Kbs 1,949 Kbs
V02000000001001 1,798 Kbs 1,999 Kbs
V02000100001002 1,798 Kbs 2,404 Kbs
V02000200001003 1,798 Kbs 2,439 Kbs
V02000300001004 1,798 Kbs 1,647 Kbs
V02000400001005 1,797 Kbs 2,296 Kbs
V03000000001001 1,798 Kbs 2,117 Kbs
V03000100001002 1,798 Kbs 1,800 Kbs
V03000200001003 1,798 Kbs 1,115 Kbs
V03000300001004 1,798 Kbs 1,456 Kbs
V03000400001005 1,798 Kbs 1,091 Kbs
V04000000001001 1,799 Kbs 1,885 Kbs
V04000100001002 1,797 Kbs 1,390 Kbs
V04000200001003 1,798 Kbs 2,085 Kbs
V04000300001004 1,798 Kbs 2,046 Kbs
V04000400001005 1,798 Kbs 1,222 Kbs

[archaeo]

Hat's off to Boulder for exploring this concept, and to Sharc and others for testing/proving... even though I was originally a little resistant to the idea, I've very pleased with the result.

:thanks:


I second that. :)
I have found that this new method has definitely improved the CBR stuff I occasionally work with, and have noted similar improvements with a few of the 'standard fare' dvd sources.

[Boulder]

Thank you for the compliments. I'd like say a big thank you to robot1, who did the dirty work and turned the idea into actual code and made the clumsy spreadsheet process unneeded. Without his work, it would have taken a long time to prove anything :) Also thanks go to the testers, especially Sharc has been very active with his graphs and screenshots - I myself have been buried in real work most of the time so I couldn't have done everything myself.

[jdobbs]

I'd like say a big thank you to robot1, who did the dirty work and turned the idea into actual code and made the clumsy spreadsheet process unneeded. Hear, hear!

:thanks:

[laserfan]

I just wanna say thanks too for the new Sticky FAQ entry; some of us need a little extra spoon-feeding to "get with the program"!

:thanks:

[laserfan]

I so enjoyed the preceding "group hug" that I forgot to ask my question!!!?!! :) I've noticed looking at my Sopranos backups that each episode's opening sequence (always the same, lots & lots of very fast movement and quick-cutting) can look blocky as hell, particularly when the backup is from a 4-episode disc. But then once TonyS slams the door on his SUV and this week's program starts, it looks fine, just fine, with none of the blockiness from the credit sequence.

I have thought about re-doing them to remove the openings (then again, will I ever watch them...). But here's a theoretical for this group re: backing-up a disc like this leaving the credits alone: wouldn't the new BR option best be turned-OFF for such discs i.e. the option would allocate additional bits to the credit sequence (where we don't need/want it)?

I know, I should just experiment with it and maybe I will if it keeps raining like it has... ;)

[gurkan]

wouldn't the new BR option best be turned-OFF for such discs i.e. the option would allocate additional bits to the credit sequence (where we don't need/want it)?

Redistribution tries to achieve the same visual quality throughout the whole movie/series.
If you would like to have a better/worse quality than average in a specific section, do a redistribution, increase/decrease
the quality for preferred segments and then reallocate extra bits to main feature.
This might create a more even quality in the remaining parts.
Of course there's no need to do this if you're already satisfied with the regular rb-encoding.

[linx05]

But if you do a redistribution and then change the quality on segments THEN allocate the extra bits to the main feature, wouldn't all the work done in the redistribution phase be a waste of time?

[jdobbs]

Yes, so you probably don't want to do that within the segments of the feature. But you could, after redistribution, reallocate bits from extras and they would be used in a way consistent with the redistribution. You could redistribute bits from something like the credits if you wanted to... I personally wouldn't, though.

[TBL]

First of all, :thanks: to all parties involved in developing this feature!

I ran across a disc that has following information:

- Reduction Level for DVD-5: 56,7%
- Overall Bitrate : 2 550Kbs
- Space for Video : 4 002 486KB
- Redistributing using Base_Q: 41
- HIGH/LOW/TYPICAL Bitrates: 3 658/2 028/2 550 Kbs

Redistribution.txt:
BITRATE APPLIED TO FEATURE SEGMENTS (SEGMENT ORIGINAL REDISTRIBUTED)
(Created using a 10% sample)

V01000000001001 2 946 Kbs 3 058 Kbs
V01000100001002 2 881 Kbs 2 794 Kbs
V01000200001003 2 597 Kbs 2 273 Kbs
V02000000001001 2 694 Kbs 3 012 Kbs
V02000100001002 2 714 Kbs 3 234 Kbs
V02000200001003 2 278 Kbs 2 397 Kbs
V03000000001001 2 375 Kbs 2 083 Kbs
V03000100001002 2 423 Kbs 2 063 Kbs
V03000200001003 2 229 Kbs 2 028 Kbs
V04000000001001 2 949 Kbs 3 658 Kbs
V04000100001002 2 861 Kbs 3 648 Kbs
V04000200001003 2 706 Kbs 3 240 Kbs
V05000000001001 2 291 Kbs 2 116 Kbs
V05000100001002 2 335 Kbs 2 092 Kbs
V05000200001003 2 455 Kbs 2 143 Kbs
V06000000001001 2 449 Kbs 2 170 Kbs
V06000100001002 2 472 Kbs 2 178 Kbs
V06000200001003 2 346 Kbs 2 038 Kbs

It is an episodic disc (pretty small local release), the episodes are about 36 minutes long and split only into 3 parts. The average bitrate is quite low, but there isn't really anything to remove, I already shrinked the menus. The question is, should I split the episodes to smaller parts and second guess the production studio or keep the episodes as they are (which one do you think makes less damage)?

[blutach]

You've already got quite a variation between high and low. As I understand it, it may not be worth the extra time taken to run the redist in these circumstances (which you have already done).

In any event, the BR looks pretty low. I'd split the disks.

Regards

[Sharc]

You've already got quite a variation between high and low. As I understand it, it may not be worth the extra time taken to run the redist in these circumstances.

This is no reason to skip redistribution, in my experience.
You may find a high variation of the swing in the original and still get quite a different profile after redistribution, depending on the source material and the combo of the matrix and the quant.
One should also not forget that a linear reduction of the bitrate reduces the swing by the same percentage (say 65%) , the absolute motion and picture content however remain the same for the original and the re-encode.
The extra time redistribution takes is low with sample sizes of 10%, so why should one bother too much?

[blutach]

The extra time redistribution takes is low with sample sizes of 10%, so why should one bother too much?Thanks for the info Sharc. I did a couple of redists and one I can remember took 77 minutes to get thru the prepare (using HCEnc on best, FHE matrix on what appeared to be a CBR source encoded at 8mbps and I think Base Q was 20). This must not be normal. Am I missing a variable somewhere?

Regards

[Boulder]

Do you do any filtering? It might also be that buffer underflows are being fixed constantly in HC, q2 is very low so the bitrate will be very high.

[Sharc]

The 77 minutes appear long to me, however I am not yet very experienced with HC.
Perhaps you check the Redist_Percent=10 under the [Options] in rebuilder.ini.
If you should have filters applied they will also add up in time during the redistribution process.

[jdobbs]

I do 10% redistributions, and I typically see it taking an extra 5-6 minutes in PREPARE.

[blutach]

This might be my issue - there is no redist_percent in my INI file :eek: (Also, I do not use filters)

Perhaps I should add a line. 10% is considered sufficient? I saw the initial thoughts of Boulder saying 100% was best, but this view is superceded now?

I'll do another test and revert. I was pretty sure though that it wasn't encoding the lot. 77 minutes would have been way too fast for a single processor.

Regards

[jdobbs]

It should default to 10% if it isn't there... but I'll check to be sure.

My testing doesn't show a significant difference in the allocation between 10% and higher amounts. But with 100% you can be absolutely sure it is allocated exactly. I personally think 10% is plenty.

[blutach]

OK - I think I can see the issue. Just doing a prepare now on a real easy souce (fits into a DVD-5).

HC021 is struggling along at 4-5 fps!!!! :( (a normal encode gets 35 fps)

Anyone else seeing this?

Regards

[jdobbs]

I see some slowdown with HC, but not that much. It's probably because of scene detection. When running through a sampling it will in most cases determine a new scene every 12 frames. Maybe I should turn scene detection off for the redistribution passes when using HC?

I've done most of my REDISTRIBUTION testing with CCE SP.

[blutach]

I fed the AVS into HC itself and changed scene detection to off. It is going about the same speed as it does (maybe a fraction slower) when doing the 2% sampling pass of CQ mode.

EDIT: The encoder races along at 45 fps with a redist AVS in CQ mode. See encoder log attached. So, it's not the scene change detection, I don't think.

EDIT2: Turning on the VBV check in the encoder slows things down a lot! Need more experimentation but this could be the issue. I note that with VBV off, the quant is as selected. But if it is on, it is nowhere near that selected.

Regards

[jdobbs]

Yeah, I looked at scene detection -- and it doesn't seem to have an effect. I'll do some testing with the VBV.

[Sharc]

Off the records:
May be your findings will eventually help me to to a redistribution with HC at all. I reported my weird problem earlier: HC cannot write the sample file, seems to be something odd with the script. Very obscure ..... I assumed that something is possibly odd with my system, and simply gave up and restricted my tests to CCE.

Edit:
Maybe it's related to what is discussed here:
http://forum.doom9.org/showthread.php?p=1012413#post1012413

[jdobbs]

The VBV point is interesting. I tested it on one of my systems and it had no impact at all -- then on another (older) one it seemed to speed things up. The redistribution seems to be identical whether it's on or off -- so I've turned it off during redistribution (with HC) for the next release.

[laserfan]

HC021 is struggling along at 4-5 fps!!!! :( (a normal encode gets 35 fps)I have one going right now avg 33fps current 45fps. I just have "Best" for HC and all defaults.

[Boulder]

Yeah, I looked at scene detection -- and it doesn't seem to have an effect. I'll do some testing with the VBV.You'll probably want to disable scene change detection when the redist percent is not 100. Otherwise the result will not be as accurate as it could be, there'll be a scene change pretty much after every GOP. If you use autoGOP together with scene change detection and don't use 100% redistribution, the GOP structures will be far from what they would be in regular encode.

Would it also be possible to add the "disable all filtering" option and the option to choose the profile to use in HC for redistribution? The disable all filtering option should also remove any cpu=xx statement from MPEG2Source. I use the "normal" profile for redistribution in Rb-Opt as it is somewhat faster. I don't think the results differ much from "best". One might even test using "fast" - the most important thing is in any case that all segments are treated the same way.

[Carpo]

is this feature classed as stable now? have a few encodes to do and am thinking about trying it out

[Sharc]

Stability and performance is much ensured by many users trying it and giving feedback. I feel it's safe to use at the stage it is, with perhaps in exceptional cases a tendency to slightly undersizing, depending on the source and redistribution profile.

[Carpo]

i'll think i'll pass for now then, have done enough alpha/beta testing on other tools ;)

[laserfan]

i'll think i'll pass for now then, have done enough alpha/beta testing on other tools ;)No reason to do that--it's perfectly stable AFAICT.

There is one difference in workflow that I noticed last night. I tried a backup of Full w/Menus and Steal 25%. As I usually do, after the Prepare phase I went into the Editor looking for garbage to blank. I was easily able to find & blank a Warner logofilm, a Rating display, and a Warning, but because these were in the "Main Feature" titleset I was not able to increase the main feature bitrate as I usually do, to take advantage of the blanks. I could see that the cells in the feature all had differing bitrate allocations, but I was not able to adjust them--at least, when I tried, the indicator that would show me approaching 4.34Gb didn't change--I wasn't able to allocate the extra bits to any of the movie cells. I instead went to the one Extra (a featurette) which, owing to the "Steal 25%" was really crunched, and gave the extra space to IT. The encode/rebuild went as expected and the disc hit the target perfectly.

I guess for discs where "things you wanna blank" are in the Feature titleset it would be best to use maybe Vobblanker on it first, unless someone knows of another way to do this, or maybe something I did wrong...

p.s. this movie was (perhaps a typical?) action/adventure but with a lot of drama and a love story so it's slower than most A/A movies in parts, and you could easily see where RB was allocating less bitrate to the stationary scenes, and more to the outdoor action scenes. At least that's the way I interpreted the variation in compression from 45% on some cells to 80% on others!!!?!

[Boulder]

Here's the graph for my tests, doing the redistribution with HC's three different profiles:
http://img154.imageshack.us/img154/9923/comptestcg2.th.png (http://img154.imageshack.us/my.php?image=comptestcg2.png)

As you can see, one can definitely use "Fast" for doing the redistribution pass, it doesn't affect the result in a noticable way.

[blutach]

@Boulder - have you experienced the same issues I reported earlier (see post 95) with HC in BR Redist (ie very slow to complete the prepare)? If not, I'd be real interested in your settings.

Regards

[Boulder]

The slowdowns are due to HC re-encoding the GOPs where buffer underflows occur. You can verify that by checking the "Show buffer" checkbox and seeing how the buffer level is gradually lowered when you experience a slowdown, you can also see that the quants are raised in those GOPs.

For me it happens only when the avg bitrate is rather high, say over 5000kbps. On lower bitrates it is quite rare but does occur when there are GOPs that require lots of bits, hitting the maximum bitrate that you have set (in this case HC needs to re-encode the GOP as well).

[jdobbs]

On the next release of DVD-RB I've set it up so HC redistribution disables VBV and SCD and also runs in "Fast" mode. In addition I've set it up so that HC will be used for redistribution with ProCoder and CCE Basic.

[blutach]

Excellent. Thanks for that jdobbs and for the info Boulder.

Also thanks to you, Sharc and jdobbs (and others) for exploring this idea. Kudos.

Regards

[Carpo]

ok my next question regaurding this feature would be the threshold on which to use it - by that i mean at what percentage would it be a waste of time using it on i have a few dvd9s i want to back up but the percentange is at 53.4% - br is about 2653 roughly, is it worth using this feature on discs like that ?

[jdobbs]

So far my recommendation would be that if you've decided to use it, just turn it on and leave it on. But, keep in mind that on most discs you won't be able to tell any difference from a standard "no redistribution" encode. It's the exceptions that give you the "bang".

[Carpo]

cant say fairer than that :) guess i'll leave it on :), have had to do a backup due to hdd error - all i need to do is add redist_all=1 to rebuilder ini correct ?

[jdobbs]

"redist_all=1" should only be turned on when you are doing a series disc. All others should only have "redistribute=1" set (which is done by checking "Enable Bitrate Redistribution" from the MODE menu. In the next release I've made it so both can be turned on/off from the menu.

[Boulder]

On the next release of DVD-RB I've set it up so HC redistribution disables VBV and SCD and also runs in "Fast" mode. In addition I've set it up so the HC will be used redistribution with ProCoder and CCE Basic.Could you please leave SCD on if the redistribution percentage is 100%?

[jdobbs]

Makes sense... done.

[kumi]

In addition I've set it up so that HC will be used for redistribution with ProCoder and CCE Basic.Awesome feature, thanks jdobbs.

[Carpo]

"redist_all=1" should only be turned on when you are doing a series disc. All others should only have "redistribute=1" set (which is done by checking "Enable Bitrate Redistribution" from the MODE menu. In the next release I've made it so both can be turned on/off from the menu.

just seen the option you are on about in the new release, giving it a test now

[jdobbs]

Awesome feature, thanks jdobbs.That's why I'm here. :)

[blutach]

Just reporting that the BR dist on HC goes a lot quicker. Still somewhat slow (21 fps - a 14 minute prepare compared to 2 minutes without - I might have expected say 7 minutes on this source at 45 fps, so there is still something holding it back).

-----------------
[11:15:22] Phase I, PREPARATION started.
- DVD-RB v1.26.2
- AVISYNTH 2.5.6.0
- HC v0.21.0.0 encoder selected
- VTSM_01: 38,163 sectors.
-- Scanning and writing .D2V & .AVS files
-- Processed 1,887 frames.
-- Building .AVS and .ECL files
- VTS_01: 1,847,486 sectors.
-- Scanning and writing .D2V & .AVS files
-- Processed 133,200 frames.
-- Building .AVS and .ECL files
- VTS_04: 43,572 sectors.
-- Scanning and writing .D2V & .AVS files
-- Processed 2,260 frames.
-- Building .AVS and .ECL files
- VTS_05: 50,310 sectors.
-- Scanning and writing .D2V & .AVS files
-- Processed 2,748 frames.
-- Building .AVS and .ECL files
- Reduction Level for DVD-5: 100.0%
- Overall Bitrate : 5,525Kbs
- Space for Video : 4,344,918KB
- Redistributing using Base_Q: 19
- HIGH/LOW/TYPICAL Bitrates: 8,808/3,550/5,525 Kbs
[11:29:51] Phase I, PREPARATION completed in 14 minutes.

This source could fit on a DVD5 so it was only for testing and, given its high BR already, there was no meaningful change.

BITRATE APPLIED TO FEATURE SEGMENTS (SEGMENT ORIGINAL REDISTRIBUTED)
(Created using a 10% sample)

V01000000001001 5,538 Kbs 5,534 Kbs
V01000100001002 5,235 Kbs 5,340 Kbs
V01000200001003 5,453 Kbs 5,313 Kbs
V01000300001004 5,389 Kbs 5,389 Kbs
V01000400001005 5,272 Kbs 5,272 Kbs
V01000500001006 5,349 Kbs 5,349 Kbs
V01000600001007 5,504 Kbs 5,524 Kbs
V01000700001008 5,377 Kbs 5,377 Kbs
V01000800001009 5,460 Kbs 5,460 Kbs
V01000900001010 5,573 Kbs 5,573 Kbs
V01001000001011 3,550 Kbs 164 Kbs

The only thing I noticed that is a bit strange is the last segment - obviously a blank cell. Now, given it goes for 15 frames, the redist up to 3550 is irrelevant, but for such short cells, does it make sense to redistribute?

Regards

[jdobbs]

If a segment is less than 60 frames the it will be encoded at a minimum bitrate of 4500Kbs anyway. That's because of a bug in CCE than can cause it to randomly crash if you try.

I do think it's a good idea to redistribute, though, regardless of segment size. Only 15 frames wouldn't make a lot of difference -- but why use any bitrate that you don't really need? Apply it (however small) to another segment.

[blutach]

Thanks for that. Any idea why an encode with HC alone is at 45fps but under DVDRB is 20?

Regards

[jdobbs]

Are you saying it is always slower, or only when doing redistribution pass?

During a redistribution pass (or an OPV prediction pass) you are not reading contiguous frames. DVD-RB is using the AVISYNTH function SelectRangeEvery() to get a sampled subset of the entire stream. To do that it has to jump through the source -- for example it might encode 12 frames, skip a couple of hundred, and then encode another 12 (depending upon the sample percentage). When it does that it has to seek the frame, then step back and find any dependent preceding frames because of the way MPEG works (suppose the start point is a B or P frame, for example) -- and in some cases (like a GOP that isn't closed) you may have to even read another GOP. All that extra work makes it a lot harder -- and slower -- during the encode.

[blutach]

Only during redist. I will try an experiment tomorrow with the same AVS fed into HC alone and see how it goes.

EDIT: Same times both within and outside DVDRB.

Regards

[Carpo]

im giving the redist option a try and am finding that a cce prepare is 11mins and hc is 21 mis - this is the same disc, so any idears why hc is 50% slower than cce.

Please note im not bashing HC i have used it many times in the past and found it to kick cce's arse on one or two discs :)

Ok this may be a stupid thing to ak but i will ask anyway, i have a disc i want to bck up but i do not wish to keep all the extras just a few - so im guessing i will need to select redist_all, question is if i do the prepare phase and let it do the redist then use segmant viewer and blank out the extras i dont want, how will this affect the redist ? I know it will, and prob am going to be told not to use redist option for these circumstances, but thought it best to ask anyway :)

edit: one other question does anyone here do all their backing up with real time anti virus on or off ? does it affect making isos and burning them with av on ? i use avg and eset nod32, normaly i have it off and do a full pc scan everynight while i am in bed, when i do use dvdrb to back up i norm leave it off

[kumi]

Hi, can bitrate redistribution be safely used with these options?

1/ Steal Space
2/ Movie and Menus Only Mode
3/ Half-D1 and Half Space for Extras

(If it makes a difference, I'm always using redist_all=0)

[Fishman0919]

1) Yes
2) Yes
3) Yes

jdobbs had stated before that it was fine.

bitrate redistribution is done 1st, then the space adjustment for extra is done

[kumi]

Thank you.

[jdobbs]

The redistribution is done before those actions... so the actions will be applied against the redistributed allocation.

[archaeo]

I wanted to report a particular instance where redistribution definitely gave me an inferior result compared to a regular VBR encode. Source for this backup is non CBR, commercial disc.

I report this primarily in response to another thread (couldn't find the exact one) where a forum member asked if redistribution should be enabled 'all the time' even for non CBR sources. Since v1.26, I have been doing almost all my encodes with 'enable bitrate redistribution' ticked until I saw this come up.

The source is 'Apocalypto', and I ran the project first with 'Enable Bitrate Redistribution' checked, CCE 3 pass, movie only, 87.2% reduction, avg bitrate @ 3943. DVDRB base_Q came in at 26. The output sizing was good at 4.32 Gb. The problem was with many of the jungle scenes, where the source had a fair amount of motion and 'noise' in the background, ie layers of leaves, vegetation, etc... As you will see the output had a great deal of blocking and pixellation. I submit my results for one typical scene during a chase in the jungle: apoc_redis_2.jpg (1st image)

After I saw this, I wondered if this blocking/pixellation problem was due to the redistribution process, so I simply unchecked redistribution, and ran the same project through the same parameters. The results are here: apoc_vbr2.jpg (2nd image)

As you can see, the regular VBR encode did not macro-block or pixellate as did redistribution encode. There seemed to be some issue in the redistribution process that had trouble with the complexity of the scene, motion, etc.. where the regular VBR encode did not. The macro blocking was apparent in many of the chase scenes in the jungle.

I have to say that up to this point I had not noticed a problem with any of my encodes when using redistribution on non-CBR sources. However, catching this, I wanted to post this comparision so that one is aware that there may be a risk attached to keeping 'enable bitrate distribution' checked by default, primarily for a non CBR source.

[Boulder]

Did you use the default amount of 10% in redistribution? If so, please try 100% and redo the whole project. If you don't use 100%, there is always a chance of inaccuracy in the redistribution.

It would also be interesting to see the redistribution log in both cases to compare them.

[archaeo]

Did you use the default amount of 10% in redistribution? If so, please try 100% and redo the whole project. If you don't use 100%, there is always a chance of inaccuracy in the redistribution.

It would also be interesting to see the redistribution log in both cases to compare them.


Yes, I did use the default amount of 10%. I'm currently running the project again through RB-Opt, to see if anything differs, (edit:which it does, in the calculated Q: RB-OPt uses 55, and RB used 26). I'll check the results visually as soon as it is finished.

[archaeo]

Ok, it appears that the redistribution process in DVD RB is causing the problem - RB Opt's redistribution results for the same project and same parameters are good - I've posted another screenshot (above, bottom image), showing the results using RB Opt's (v30) redistribution process, using 'calculate optimal Q'.

As you can see (when they're approved) the image quality is good and is comparable to that when using DVDRB's normal 3 pass VBR.

Here's the redistribution profile from DVD RB (apoc_redis_2.jpg):

BITRATE APPLIED TO FEATURE SEGMENTS (SEGMENT ORIGINAL- REDISTRIBUTED)
(Created using a 10% sample)

V01000000001001 4,095 Kbs 4,523 Kbs
V01000100001002 4,149 Kbs 4,469 Kbs
V01000200001003 3,738 Kbs 4,090 Kbs
V01000300001004 4,204 Kbs 3,521 Kbs
V01000400001005 3,917 Kbs 4,165 Kbs
V01000500001006 3,642 Kbs 3,762 Kbs
V01000600001007 3,975 Kbs 4,298 Kbs
V01000700001008 3,767 Kbs 3,766 Kbs
V01000800001009 3,539 Kbs 4,057 Kbs
V01000900001010 4,001 Kbs 4,655 Kbs
V01001000001011 3,890 Kbs 4,459 Kbs
V01001100002001 3,805 Kbs 4,362 Kbs
V01001200002002 3,622 Kbs 3,765 Kbs
V01001300002003 3,319 Kbs 3,805 Kbs
V01001400002004 4,002 Kbs 4,547 Kbs
V01001500002005 4,346 Kbs 1,605 Kbs
V01001600002006 4,360 Kbs 4,717 Kbs
V01001700002007 4,588 Kbs 4,698 Kbs
V01001800002008 4,925 Kbs 4,529 Kbs
V01001900002009 4,346 Kbs 3,907 Kbs
V01002000002010 2,207 Kbs 2,503 Kbs
V01002100003001 4,226 Kbs 114 Kbs

______________________________________________
Here's the redistribution profile results from RB-Opt:
(created using 10% sample)

--- Calculating Bitrate distribution
Bitrate for VTS 1 VobID 1 segment 1: original 4086 - redist. 4284
Bitrate for VTS 1 VobID 1 segment 2: original 4141 - redist. 3988
Bitrate for VTS 1 VobID 1 segment 3: original 3731 - redist. 3540
Bitrate for VTS 1 VobID 1 segment 4: original 4195 - redist. 2680
Bitrate for VTS 1 VobID 1 segment 5: original 3909 - redist. 3860
Bitrate for VTS 1 VobID 1 segment 6: original 3635 - redist. 2909
Bitrate for VTS 1 VobID 1 segment 7: original 3967 - redist. 4064
Bitrate for VTS 1 VobID 1 segment 8: original 3760 - redist. 3401
Bitrate for VTS 1 VobID 1 segment 9: original 3532 - redist. 4279
Bitrate for VTS 1 VobID 1 segment 10: original 3993 - redist. 4986
Bitrate for VTS 1 VobID 1 segment 11: original 3882 - redist. 4805
Bitrate for VTS 1 VobID 2 segment 12: original 3797 - redist. 4761
Bitrate for VTS 1 VobID 2 segment 13: original 3614 - redist. 3018
Bitrate for VTS 1 VobID 2 segment 14: original 3312 - redist. 3949
Bitrate for VTS 1 VobID 2 segment 15: original 3994 - redist. 4578
Bitrate for VTS 1 VobID 2 segment 16: original 4337 - redist. 4094
Bitrate for VTS 1 VobID 2 segment 17: original 4351 - redist. 4703
Bitrate for VTS 1 VobID 2 segment 18: original 4579 - redist. 4650
Bitrate for VTS 1 VobID 2 segment 19: original 4915 - redist. 4080
Bitrate for VTS 1 VobID 2 segment 20: original 4338 - redist. 3213
Bitrate for VTS 1 VobID 2 segment 21: original 2202 - redist. 2215


Edit: After visually checking through the final results, the segment that had the problem is V01001500002005 (seg 16) which happens to be the one that stands out: DVDRB original = 4,346 Kbs and redistributed = 1,605 Kbs. No surprise that a 1605 bitrate would yield these results! Question is why DVDRB redistribution gave it only 1605 kbps when RB-Opt allocated over 4000?

[jdobbs]

The original bitrates don't seem to match between the two?

Any idea why these two might be so different? Best guess is that's where your jungle scene is... it sure looks out of place -- and none of the other output bitrates seem to be far enough off to make a lot of difference.

V01001500002005 4,346 Kbs 1,605 Kbs
Bitrate for VTS 1 VobID 2 segment 16: original 4337 - redist. 4094

Is this the R1 version of Apocalypto?

[archaeo]

The original bitrates don't seem to match between the two?

Not exactly. See edit... The problem segment somehow was only allocated around 1600 kbps.


Edit: yes every other segment appears to be OK thru visual inspection and bitrate allocation - seg 16 is kind of the oddball segment thats thrown a wrench into DVDRB's redistrib process.
**Yes, It's the R1 version

[jdobbs]

Sorry -- it appears I was editing my response the same time you were typing.

Was this a particularly short segment?

[archaeo]

Sorry -- it appears I was editing my response the same time you were typing.

Was this a particularly short segment?

Not too short - when looking in the D2VAVS folder the m2V segment is 127,445 KB.

[mc2man]

since I've been fooling around with this new feature i've saved most of the .tx's Same title,similar setting's produced this
V04000000003001 4,103 Kbs 4,388 Kbs
V04000100003002 4,158 Kbs 4,197 Kbs
V04000200003003 3,746 Kbs 3,528 Kbs
V04000300003004 4,213 Kbs 2,883 Kbs
V04000400003005 3,926 Kbs 3,848 Kbs
V04000500003006 3,650 Kbs 3,036 Kbs
V04000600003007 3,984 Kbs 4,121 Kbs
V04000700003008 3,775 Kbs 3,442 Kbs
V04000800003009 3,546 Kbs 4,057 Kbs
V04000900003010 4,010 Kbs 4,587 Kbs
V04001000003011 3,898 Kbs 4,459 Kbs
V04001100004001 3,813 Kbs 4,362 Kbs
V04001200004002 3,630 Kbs 3,204 Kbs
V04001300004003 3,326 Kbs 3,805 Kbs
V04001400004004 4,011 Kbs 4,588 Kbs
V04001500004005 4,356 Kbs 4,131 Kbs
V04001600004006 4,369 Kbs 4,738 Kbs
V04001700004007 4,598 Kbs 4,757 Kbs
V04001800004008 4,936 Kbs 4,176 Kbs
V04001900004009 4,356 Kbs 3,513 Kbs
V04002000004010 2,211 Kbs 2,539 Kbs
V04002100005001 4,235 Kbs 137 Kbs
Ive found for med.-high bitrate movie only the redis. is useful as a "preset" for the segment editor. After prep i go in and do some fine tuning, in the case of this title knocked down seg. 20, bumped up seg 3 and pushed a couple of others to 100%, i think I had 7 or 8 extracted which was a decent timesaver in encoding

[kumi]

Are you using non-default matrices, by any chance? See what kind of bitrate redistribution you get using with "Encoder Default" selected for all the matrices.

[archaeo]

Yes, I used 'high_medium 4000' mtx on all samples, since the source was close to that avg bitrate. I will run it again using the encoder default matrix to see if that solves the one problem segment.

Still, the problemmatic segment did not cause an issue with RB Opt's Redistribution pass. It came in at an acceptable bitrate. :confused:

[jdobbs]

I'll run it through tomorrow and see what I get for that segment. My suggestion has always been to use standard matrices in most instances, and only change the matrix when you have a specific reason to do so.

Theres something odd going on here... because I can't figure why, out of that entire feature, only one segment came in at that very low bitrate -- especially a segment with lots of detail (jungle action). :confused: Any chance the encoder is bombing out during the redistribution phase and the small output file was used to make the size determination?

Just as a test, could you do a PREPARE phase with HC and see what it's distribution table looks like?

[archaeo]

Ok... using the standard matrix in DVDRB's redistribution solved the 'odd segment' issue, getting the bitrate up to 4186. Here's the result using the CCE default mtx:


V01000000001001 4,086 Kbs 4,380 Kbs
V01000100001002 4,141 Kbs 4,112 Kbs
V01000200001003 3,731 Kbs 3,497 Kbs
V01000300001004 4,196 Kbs 2,726 Kbs
V01000400001005 3,910 Kbs 3,820 Kbs
V01000500001006 3,635 Kbs 2,895 Kbs
V01000600001007 3,967 Kbs 4,184 Kbs
V01000700001008 3,760 Kbs 3,396 Kbs
V01000800001009 3,532 Kbs 4,057 Kbs
V01000900001010 3,993 Kbs 4,587 Kbs
V01001000001011 3,882 Kbs 4,459 Kbs
V01001100002001 3,797 Kbs 4,362 Kbs
V01001200002002 3,615 Kbs 3,182 Kbs
V01001300002003 3,312 Kbs 3,805 Kbs
V01001400002004 3,994 Kbs 4,588 Kbs
V01001500002005 4,338 Kbs 4,186 Kbs
V01001600002006 4,351 Kbs 4,804 Kbs
V01001700002007 4,579 Kbs 4,824 Kbs
V01001800002008 4,915 Kbs 4,015 Kbs
V01001900002009 4,338 Kbs 3,503 Kbs
V01002000002010 2,202 Kbs 2,530 Kbs
V01002100003001 4,218 Kbs 160 Kbs


Just to be clear, I confirmed that the matrix used in RB Opt's redistribution process was in fact the custom 'high_med 4000' mtx... For some reason, DVD RB's redistribution process required the use of the standard mtx to avoid this peculiar 'odd segment' issue in this particular sample. Perhaps just a rare occurance.

[Insomniak4700]

Is it just me, or are the menus not being reencoded when redistribution is activated (yes, the option for menu reencoding is active)??

[Sharc]

@archaeo & jdobbs:

Differences between RBOpt and DVD-RB:

- RBOpt always uses a minimum of 500 frames for redistribution for small segments, whereas DVD-RB increases the 10% to 25% which may perhaps still be too few frames for small segments.

- RBOpt uses the "final_Q" for the given matrix, whereas DVD-RB uses the "Base_Q" which may lead to significantly differetnt Q values and in some cases to significantly different bitrate profiles - depending on the selected matrix.

- Further, I have noticed that CCE may just silently abort without warning during the redistribution pass. It will then simply continue with the next segment. I have seen this happening for very low Q values only. The result was a ridiculous low bitrate for the aborted segment.

[Sharc]

Did you use the default amount of 10% in redistribution? If so, please try 100% and redo the whole project. If you don't use 100%, there is always a chance of inaccuracy in the redistribution.

In my experience it is important for an "accurate" redistribution profile to use the "Final_Q" rather than the "Base_Q", in particular when non-standard matrices are used. I consider this even more critical than a large sample size.

[jdobbs]

I disagree. Look at the historical values given for bitrates -- and it doesn't seem to make a whole lot of difference at least 95% of the time.

I think that segment was probably an anomaly caused by CCE aborting. I'd also guess that the choice of matrix in use is the most likely catalyst for the aborting.

@archeao

Did you try it with HC? Did you try it more than once with CCE (using the matrix)?

[Boulder]

I disagree. Look at the historical values given for bitrates -- and it doesn't seem to make a whole lot of difference at least 95% of the time.Which still leaves room for error ;) But in this case I also agree that it probably is an encoder issue.

[jdobbs]

Well... I used 95% -- but it's probably much higher. The point is that if you're looking for perfection in this world, you're always going to be disappointed. ;)

[archaeo]

@archeao

Did you try it with HC? Did you try it more than once with CCE (using the matrix)?

I'll run those two this morning...

[Sharc]

I disagree. Look at the historical values given for bitrates -- and it doesn't seem to make a whole lot of difference at least 95% of the time.

I think that segment was probably an anomaly caused by CCE aborting. I'd also guess that the choice of matrix in use is the most likely catalyst for the aborting.

Hmmm.... I think the low Q is the reason for CCE to abort. If the "Base_Q" is low compared to the "final Q" I suspect that the bitrate is driven to high values beyond the mpeg standard, causing the abortion. This happens in particular for high bitrate matrices where the Base_Q of DVD-RB is normally too low (say 25 instead of 60).

I verified the abortion by selecting a low Q with RB-Opt.

Again, no proof, just observations .....

[archaeo]

@jdobbs,
I ran the two additional redistribution samples. Once again I was able to repeat the same segment error when running the original parameters using DVDRB w/CCE, and custom mtx, Base_Q=26:
V01001500002005 4,338 Kbs 1,601 Kbs

However, using DVDRB w/ HC .21, and custom matrix, Base_Q=26:
V01001500002005 4,317 Kbs 4,076 Kbs

and as posted previously...

DVDRB, CCE, standard matrix, Base_Q=26:
V01001500002005 4,338 Kbs 4,186 Kbs

RB Opt, cutom matrix, Q=55:
segment 16: original 4337 - redist. 4094

The problem occurs only when using CCE w/ DVDRB using the High_med 4000 matrix, and only at that one particular segment.

[mc2man]

using cce with any high bit rate matix will cause 1 or more segments to abort. Which segment(s) are affected changes with the matrix used and the available space for video, i.e. how many audio streams kept.
for example with
fox h.keeping all audio - seg.15
fox h. 1 ac3 - seg, 0, seg. 15, seg. 16
high_med. with all audio - seg.17, seg. 15
high_med with 1 ac3 - seg. 1, seg. 15, seg. 17,
ect., ect.
all these seg. have high bitrate swings from 3-8+
Using the default matrix produces no aborts with only some minor variances with cceaqm on or off

[Carpo]

so is it safe to assume that we should not use any matrix other than the dvdrb default if we are to use the redist option ?

[Sharc]

@mc2man
I would bet if you were using the Q from RB-Opt you won't run into the abortion problem even with a high bitrate matrix. The Q will be much higher compared to DVD-RB, though.

[jdobbs]

That's hardly the point. CCE shouldn't crash -- and that's where the error is. In similar circumstances RBOpt will very probably do the same thing!!!

Can you post the matrix that was used?

[jdobbs]

so is it safe to assume that we should not use any matrix other than the dvdrb default if we are to use the redist option ?
Don't know that for a fact. The problem is that there are so many matrices out there -- you never know whether they are even using legal values. Let me get ahold of the matrix and test it.

My guess (but only a guess) would be that any of the "standard" matrices would likely work just fine.

[archaeo]

That's hardly the point. CCE shouldn't crash -- and that's where the error is. In similar circumstances RBOpt will very probably do the same thing!!!

Can you post the matrix that was used?

Yes, here's the matrix that was used for the encode:

08 08 10 11 13 14 15 17
08 08 11 12 14 15 17 19
10 11 13 14 15 17 17 19
11 11 13 14 15 17 19 20
11 13 14 15 16 18 20 24
13 14 15 16 18 20 24 29
13 14 15 17 19 23 28 35
14 15 18 19 23 28 35 42

08 09 09 10 10 11 11 12
09 09 10 10 11 11 12 12
09 10 10 11 11 12 12 13
10 10 11 11 12 12 13 14
10 11 11 12 13 13 14 14
11 11 12 12 13 14 14 15
11 12 12 13 14 14 15 16
12 12 13 14 14 15 16 17


Original (I believe identical, unless I missed a value):

08 08 10 11 13 14 15 17
08 08 11 12 14 15 17 19
10 11 13 14 15 17 17 19
11 11 13 14 15 17 19 20
11 13 14 15 16 18 20 24
13 14 15 16 18 20 24 29
13 14 15 17 19 23 28 35
14 15 18 19 23 28 35 42

08 09 09 10 10 11 11 12
09 09 10 10 11 11 12 12
09 10 10 11 11 12 12 13
10 10 11 11 12 12 13 14
10 11 11 12 13 13 14 14
11 11 12 12 13 14 14 15
11 12 12 13 14 14 15 16
12 12 13 14 14 15 16 17


*AQM enabled for all tests

[jdobbs]

Hmmm... it looks like all the values are approximately cut in half from the standard MPEG matrix. One thing I can say -- this probably isn't the right matrix for the bitrates I'm seeing in this example... but then on the other hand, they really should crash CCE.

[mc2man]

the orig. source uses cceaqm matrices, i.e. mpeg standard, 1/2, 1/4

[Boulder]

Would it be an reasonable option to always check whether the encoded m2v file really has the correct number of frames? It shouldn't make the process that much slower. It would also be useful in cases when the kids have been messing with the computer and I'm never sure whether all segments have been correctly encoded. My projects usually tend to take more than 24hrs to finish so I have to leave the encoding process running while I'm at work.

[robot1]

What about changing the base_q according to the matrix used?
I know it's not a linear process, but maybe it's possible to find a good compromise to change the base_q proportionally to the average of the values in the matrices, for example.

[kumi]

I would go one step further and ask for a Q-calculation pass a la RBOpt, IMO the added time required is well worth it. The way it works now doesn't inspire much confidence: non-identical (sub-optimal?) redistributions dependant on matrix choice.

[mc2man]

non withstanding that multiple prep runs on same source will produce slightly different results I think the behavior on this title will prove to be fairly rare and hard to miss/ignore

[jdobbs]

I just don't seem to be sinking in... no matter how you do this you still have the problem that CCE will crash with certain combinations of matrices and q values. If I redo the Q value for this one, and it goes away -- it will return with another disc. It could just as easily have happened to RBOpt and not DVD-RB... it all depends on the combination. The problem has to either be fixed in CCE, or another encoder (HC?) has to be used for the redistribution.

Scanning each resulting M2V to count frames will double the PREPARE phase time -- and the odds are that retrying it again will only get the same crash.

[Sharc]

.....In similar circumstances RBOpt will very probably do the same thing!!!

YES, if you set in RB-Opt the same (low) Base_Q as for DVD-RB, and NO, if you let RB-Opt calculate the Q by few iterations.
Unfortunately I do not have this DVD, but I have seen similar in another cases happening.

In general, DVD-RB overestimates the Q for low bitratre matrices and underestimates the Q for high bitrate matrices which may cause the abortion - just my experience.

I do agree however that CCE should never abort, irrespective of the Q that is used for redistribution.

[jdobbs]

I feel like I'm talking to myself. The point is completely missed. The Q value DVD-RB uses has absolutely nothing to do with the problem. Are you telling me that with some matrices you can't use certain Q values? That's just plain silly.

I feel like I'm wasting my breath here. I'll let you guys finish this discussion... I'm outta this one.

[Sharc]

Sorry to be silly. I am just sharing my expereince and make a suggestion for a workaround (?) of a CCE problem. No less and no more. I am not an expert, but I have done a lot of testing with redistribution - and shared the results. I also did report about the CCE abortion much earlier but at that time I thought that there might be somethiong odd with my system, so I didn't take it serious until I read the recent post by archaeo.

[archaeo]

I guess I'm missing something here, too... and sorry to waste your time jdobbs if it's ignorance on my part...

But, If RB Opt can take the same project, same parameters, calculate the redistribution w/CCE, and then complete the redistribution without that weird segment allocation, why can't this same thing be done through DVDRB? It's a CCE crash, I understand - but why isn't the crash occurring w/CCE (as far as I can tell) when this process runs thru RB-Opt?

Is it that it could happen through RB opt, but just hasn't in this particular sample?

[jdobbs]

But, If RB Opt can take the same project, same parameters, calculate the redistribution w/CCE, and then complete the redistribution without that weird segment allocation, why can't this same thing be done through DVDRB? It's a CCE crash, I understand - but why isn't the crash occurring w/CCE (as far as I can tell) when this process runs thru RB-Opt?Because RBOpt is using a different Q... but, if it had used that same Q, it would have done the same thing -- and under other circumstances it very well might -- and the crash assuredly will happen then whether using DVD-RB or RBOpt. It just happens that this time it was with DVD-RB... but that may not be the case next time.

So my changing the Q will not fix the problem. My making other predictions for Q will not fix the problem. My checking the length of the output file will not fix the problem. The problem is within CCE and happens when you try certain Q values with certain matrices -- but it can only be fixed in CCE.

Of course on the other hand, if you don't feed it matrices with exceptionally low values on encodes that are using bitrates in need of stronger matrices it may not happen in either case.

Is it that it could happen through RB opt, but just hasn't in this particular sample?Exactly.

[archaeo]

OK, thanks for clarifying.
When I ran a quick test through RB Opt, it 'hiccuped' as well on the same Q value of 26:

--- Calculating Bitrate distribution
segment 1: original 4086 - redist. 4784
segment 2: original 4141 - redist. 4651
segment 3: original 3731 - redist. 4303
segment 4: original 4195 - redist. 3568
segment 5: original 3909 - redist. 4453
segment 6: original 3635 - redist. 3946
segment 7: original 3967 - redist. 4562
segment 8: original 3760 - redist. 4054
segment 9: original 3532 - redist. 4734
segment 10: original 3993 - redist. 4942
segment 11: original 3882 - redist. 4971
segment 12: original 3797 - redist. 4871
segment 13: original 3614 - redist. 3802
segment 14: original 3312 - redist. 4653
segment 15: original 3994 - redist. 4730
segment 16: original 4337 - redist. 1730
segment 17: original 4351 - redist. 5045
segment 18: original 4579 - redist. 3374
segment 19: original 4915 - redist. 2064
segment 20: original 4338 - redist. 3869
segment 21: original 2202 - redist. 2640


Segment 19 is also showing low bitrate, too.

[Sharc]

Does this mean that CCE will also fail in OPV mode under similar circumstances? Is there any experience of such failures?

[Boulder]

OK, thanks for clarifying.
When I ran a quick test through RB Opt, it 'hiccuped' as well on the same Q value of 26:

segment 16: original 4337 - redist. 1730

Segment 19 is also showing low bitrate, too.Could you run the redistribution pass and see (I mean actually watch through the encoding process of those segments) what really happens there? I've never seen such a thing occur with CCE so this is quite interesting. Q26 isn't even a low value.

[mc2man]

While it's ironic that the title in question here has nothing to gain in terms of pq from redis (IMHO). As far as the cce error(s) here's a different view that you may find some info in (mostly greek to me). aborts on seg. 15, 16 fox high 1 ac3

***** CCE SP Trial Version started at 2007/07/16 02:29:37 *****
-- CCE SP Trial Version version 2.70.02.06
-- SDK version 2.70 (built at 17:19:40 Apr 18 2005)
cce created.
fexp thread created.
fixme:listview:LISTVIEW_SetColumnOrderArray iCount 6 lpiArray 0x900963c
TC in: 00:00:00:00
TC out: 00:01:05:12
video file: C:\SPOT\D2VAVS\V040015_REDIST.m2v
encoder initialized.
uncompressed YUY2.
encoding started at 2007/07/16 02:29:38.
>> received encoding start notification
>> received encoding start notification
>> received encoding stop notification
cce encoding failed: OPV VBV ovf frame# 564 (00:00:23:14) I 122384 max 121322.23 rel 1061.77 1.01 qsv 11.60->11.60
sync stopped.
encoding stopped at 2007/07/16 02:29:53.
fdev0 closed.
fdev1 closed.
>>>> Performance <<<<
Source : 24.274 seconds (582 frames)
Elapsed: 14.866 seconds
---------------------------------------------------
>> File reading 8.194 55.122 %
>> Decoding 0.000 0.000 %
>> Resizing 0.000 0.000 %
>> Deinterlacing 0.000 0.000 %
>> RGB -> YUY2 0.000 0.000 %
---------------------------------------------------
>> MPEG encoding 6.672 44.878 %
fexp thread terminated.
fixme:avifile:AVIFileExit (): stub!
***** CCE SP Trial Version started at 2007/07/16 02:29:54 *****
-- CCE SP Trial Version version 2.70.02.06
-- SDK version 2.70 (built at 17:19:40 Apr 18 2005)
cce created.
fexp thread created.
fixme:listview:LISTVIEW_SetColumnOrderArray iCount 6 lpiArray 0x900963c
TC in: 00:00:00:00
TC out: 00:00:43:00
video file: C:\SPOT\D2VAVS\V040016_REDIST.m2v
encoder initialized.
uncompressed YUY2.
encoding started at 2007/07/16 02:29:55.
>> received encoding start notification
>> received encoding start notification
>> received encoding stop notification
cce encoding failed: OPV VBV ovf frame# 60 (00:00:02:12) I 128920 max 127322.73 rel 1597.27 1.01 qsv 7.67->7.85
sync stopped.
encoding stopped at 2007/07/16 02:29:57.

[Boulder]

It's the classic VBV buffer overflow error. For some reason, CCE sometimes doesn't recover from these whereas HC will grind the GOP in question till the problem with the VBV buffer overflow is taken care of (hence the slowdowns people have observed).

[Carpo]

am doing a few tests with this redist option to see of using different matricies actually do affect the outcome, what is dvdrbs default? is it mpeg standard ? just saves me having to use that one if its the deafult - 1 less test to do :)

on a side note has anyone used 1800-3500 AVAMAT7.mtx and 3500-9500 AVAMAT6.mtx

AVAMAT7.mtx

8 16 19 22 26 28 32 38
16 16 22 24 28 32 38 44
19 22 26 28 32 38 44 48
22 22 26 32 38 44 48 54
22 26 32 38 44 48 54 64
26 32 38 44 48 54 64 74
32 38 44 48 54 64 74 84
38 44 48 54 64 74 84 94

16 20 24 28 36 42 46 52
20 24 28 36 42 46 52 58
24 28 36 42 46 52 58 62
28 36 42 46 52 58 62 68
36 42 46 52 58 62 68 78
42 46 52 58 62 68 78 88
46 52 58 62 68 78 88 99
52 58 62 68 78 88 99 99

AVAMAT6.mtx

8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 35
19 22 26 27 29 34 35 38
22 22 26 27 29 34 35 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 50
26 27 29 35 40 48 50 60
27 29 35 40 48 50 60 62

16 20 24 28 32 36 40 44
20 24 28 32 36 40 44 48
24 28 32 36 40 44 48 52
28 32 36 40 44 48 52 56
32 36 40 44 48 52 56 58
36 40 44 48 52 56 58 60
40 44 48 52 56 58 60 62
44 48 52 56 58 60 62 62

[archaeo]

Could you run the redistribution pass and see (I mean actually watch through the encoding process of those segments) what really happens there? I've never seen such a thing occur with CCE so this is quite interesting. Q26 isn't even a low value.


@boulder, it looks like mc2man ran the test you were looking for...

Interesting.
I wonder if all CCE versions will show this buffer overflow error?

[Boulder]

I've seen it for a long time now so I suspect every version may cause it.

[dynamis]

i just tried to encode my problem segment with CCE SP 2.67.00.23 and it went through.
this segment has a still picture with credits fading in and out:


--- Encoding VTS 4 VobID 1 segment 12
Original Bitrate: 3549
- Waiting for encoder...

=== Completed encoding VTS 4 VobID 1 segment 12
Size of the segment: 5632 KB
Segment bitrate: 3466

20% samples, Q=1
edit: got the same bitrate at Q=5. it this... good? :P

with 2.70.02.12 and 2.70.02.01 the segment 12 calculation would die and spit out 327.
this is only one of the few movies i had this problem with, so... i'm gonna try some more. i'm going to try with CCE SP 2.50 that's on the other rig too.



edit: 2.50 gave me .... 3792

[stereo]

Does all this mean, we'd better not use the CCE + redistribution method (DVD RB as well as RB-Opt) untill this issue has been solved?

I just encountered this in RB-Opt:

--- Found Q factor: 24
[...]
--- Calculating Bitrate distribution
Bitrate for VTS 1 VobID 5 segment 1: original 3078 - redist. 2835
Bitrate for VTS 1 VobID 5 segment 2: original 3905 - redist. 3671
Bitrate for VTS 1 VobID 5 segment 3: original 3409 - redist. 3468
Bitrate for VTS 1 VobID 5 segment 4: original 2832 - redist. 2618
Bitrate for VTS 1 VobID 5 segment 5: original 4608 - redist. 4973
Bitrate for VTS 1 VobID 5 segment 6: original 2872 - redist. 2117
Bitrate for VTS 1 VobID 5 segment 7: original 3663 - redist. 3210
Bitrate for VTS 1 VobID 5 segment 8: original 4089 - redist. 3329
Bitrate for VTS 1 VobID 5 segment 9: original 5579 - redist. 6029
Bitrate for VTS 1 VobID 5 segment 10: original 5132 - redist. 5307
Bitrate for VTS 1 VobID 5 segment 11: original 4674 - redist. 4573
Bitrate for VTS 1 VobID 5 segment 12: original 2632 - redist. 1367
Bitrate for VTS 1 VobID 5 segment 13: original 4484 - redist. 4510
Bitrate for VTS 1 VobID 5 segment 14: original 4222 - redist. 4457
Bitrate for VTS 1 VobID 5 segment 15: original 4601 - redist. 4527
Bitrate for VTS 1 VobID 5 segment 16: original 5124 - redist. 5491
Bitrate for VTS 1 VobID 5 segment 17: original 5025 - redist. 5492
Bitrate for VTS 1 VobID 5 segment 18: original 3654 - redist. 3522
Bitrate for VTS 1 VobID 5 segment 19: original 2543 - redist. 1556
Bitrate for VTS 1 VobID 5 segment 20: original 3176 - redist. 1854
Bitrate for VTS 1 VobID 5 segment 21: original 4193 - redist. 3983
Bitrate for VTS 1 VobID 5 segment 22: original 2593 - redist. 2167
Bitrate for VTS 1 VobID 5 segment 23: original 3268 - redist. 2701
Bitrate for VTS 1 VobID 5 segment 24: original 4308 - redist. 4955
Bitrate for VTS 1 VobID 5 segment 25: original 3708 - redist. 4136
Bitrate for VTS 1 VobID 5 segment 26: original 3064 - redist. 3041
Bitrate for VTS 1 VobID 5 segment 27: original 4515 - redist. 5397
Bitrate for VTS 1 VobID 5 segment 28: original 4270 - redist. 4887
Bitrate for VTS 1 VobID 5 segment 29: original 4967 - redist. 5827
Bitrate for VTS 1 VobID 5 segment 30: original 4932 - redist. 5917
Bitrate for VTS 1 VobID 5 segment 31: original 3959 - redist. 4557
Bitrate for VTS 1 VobID 5 segment 32: original 3305 - redist. 3345
Bitrate for VTS 1 VobID 5 segment 33: original 2906 - redist. 2186

- and segment 12 (as well as af few others) seems odd or at least very, very low.

Using DVD RB 1.26.2 RB-Opt 0.33, CCE 2.70.02.06
Matrix: Encoder default

[archaeo]

Best to check the segments visually - some of these low bitrates may be OK, depending on what's going on.

[stereo]

OK, so you suggest I re-encode it, and check each of the (possibly) problematic segments?

[robot1]

Bitrate for VTS 1 VobID 5 segment 12: original 2632 - redist. 1367

- and segment 12 (as well as af few others) seems odd or at least very, very low.

Using DVD RB 1.26.2 RB-Opt 0.33, CCE 2.70.02.06
Matrix: Encoder default

Have you checked the quality of the encoded segment?
If CCE crashes with std matrix and Q 24, it could have great problems.

[stereo]

No, I haven't checked the segments yet. I'll encode the disc and see what happens.

[stereo]

BTW, is there a way to encode just one segment in DVD RB (I mean without having to encode the entire disc)?

[kumi]

You can do it from the Preview/Edit Window. Set all segments except the one(s) you want encoded to "No Reencode". Then press "Save & Quit".

ALternatively, you can cut out the unwanted segments' [item] and [file] sections from the REBUILDER.ECL file in the Working Path with a text editor.

[stereo]

OK, thanks, will give it a try.

[Carpo]

has anyone tried this with autoqmatenc ? - i think i'll just have to use HC like jdobbs said in an earlier post if cce is being a tit, or is it a case as long as you check the redist.txt file and it all loks ok in there to go ahead and use cce ?

[stereo]

Have you checked the quality of the encoded segment?
If CCE crashes with std matrix and Q 24, it could have great problems.

I have encoded and checked the following (bold) segments:

--- Found Q factor: 24
[...]
--- Calculating Bitrate distribution
Bitrate for VTS 1 VobID 5 segment 1: original 3078 - redist. 2835
Bitrate for VTS 1 VobID 5 segment 2: original 3905 - redist. 3671
Bitrate for VTS 1 VobID 5 segment 3: original 3409 - redist. 3468
Bitrate for VTS 1 VobID 5 segment 4: original 2832 - redist. 2618
Bitrate for VTS 1 VobID 5 segment 5: original 4608 - redist. 4973
Bitrate for VTS 1 VobID 5 segment 6: original 2872 - redist. 2117
Bitrate for VTS 1 VobID 5 segment 7: original 3663 - redist. 3210
Bitrate for VTS 1 VobID 5 segment 8: original 4089 - redist. 3329
Bitrate for VTS 1 VobID 5 segment 9: original 5579 - redist. 6029
Bitrate for VTS 1 VobID 5 segment 10: original 5132 - redist. 5307
Bitrate for VTS 1 VobID 5 segment 11: original 4674 - redist. 4573
Bitrate for VTS 1 VobID 5 segment 12: original 2632 - redist. 1367
Bitrate for VTS 1 VobID 5 segment 13: original 4484 - redist. 4510
Bitrate for VTS 1 VobID 5 segment 14: original 4222 - redist. 4457
Bitrate for VTS 1 VobID 5 segment 15: original 4601 - redist. 4527
Bitrate for VTS 1 VobID 5 segment 16: original 5124 - redist. 5491
Bitrate for VTS 1 VobID 5 segment 17: original 5025 - redist. 5492
Bitrate for VTS 1 VobID 5 segment 18: original 3654 - redist. 3522
Bitrate for VTS 1 VobID 5 segment 19: original 2543 - redist. 1556
Bitrate for VTS 1 VobID 5 segment 20: original 3176 - redist. 1854
Bitrate for VTS 1 VobID 5 segment 21: original 4193 - redist. 3983
Bitrate for VTS 1 VobID 5 segment 22: original 2593 - redist. 2167
Bitrate for VTS 1 VobID 5 segment 23: original 3268 - redist. 2701
Bitrate for VTS 1 VobID 5 segment 24: original 4308 - redist. 4955
Bitrate for VTS 1 VobID 5 segment 25: original 3708 - redist. 4136
Bitrate for VTS 1 VobID 5 segment 26: original 3064 - redist. 3041
Bitrate for VTS 1 VobID 5 segment 27: original 4515 - redist. 5397
Bitrate for VTS 1 VobID 5 segment 28: original 4270 - redist. 4887
Bitrate for VTS 1 VobID 5 segment 29: original 4967 - redist. 5827
Bitrate for VTS 1 VobID 5 segment 30: original 4932 - redist. 5917
Bitrate for VTS 1 VobID 5 segment 31: original 3959 - redist. 4557
Bitrate for VTS 1 VobID 5 segment 32: original 3305 - redist. 3345
Bitrate for VTS 1 VobID 5 segment 33: original 2906 - redist. 2186


- and while CCE didn't crash, the segments looked absolutely horrible.

Setup:
Movie: Windtalkers Pal (Nordic Special Edition)
Avg. bitrate: 3.950
Redistribution: RB-Opt 0.33 (q sample: 25 % - OPV prediction sample: 1 %)
Q: 24
CCE: 2.70.02.06 (2 passes: 1 vaf + 1)
Matrix: Encoder Default

I don't know if this has to do with the questions and problems described in this thread, but the Rebuilder.ecl did show the line "qmat= [matrix]" in exactly these - and only these - segments. I thought there wasn't supposed to be a matrix involved with a constant Q.

Anyway, I'll just go with a regular 3(4) pass vbr on this one.

[dynamis]

hey stereo, i don't know if this will work for you, but have you tried using CCE SP 2.70.02.01 or earlier for Bitrate Redistribution?

[stereo]

hey dynamis, no I haven't tried these early versions on redistribution. I actually did try 2.70.02.12, and CCE SP2 - none of them worked with RB-Opt.

For the moment being, I've decided to drop redistribution on the sources that look like trouble. I've been doing hundreds of great 3(4) passes using 2.70.02.06, and I don't mind going back to that (for the time being) - even though I quite like the idea of bitrate redistribution...

[mc2man]

reading thru this and other threads concerning methods of 'blanket" style redistribution (vs.seg.editing) it strikes me that it's receiving too much credit based on seg. averages, graphs showing this or that ect. Ultimately though it still just an average. A more relevant comparison would be to chart/graph an individual seg. created by the various methods. i've compared a number of segs. produced by cce w/custom matrix; cce w/cceaqm; cce w/cceaqm and segment editor; and cce default w/redistribution. My money's on cce w/cceaqm and seg. editor tweaking in terms of retaining PQ and possibly shortening encoding times to to extraction. (in most cases).
Personally i was hoping that the redis. feature could give me a decent 'preset' for the segment editor, unfortunately once you run a redis. pass and then open the segment editor and click on any segment the seg. editor basically becomes fubar till you do a reinstall. It's behavior becomes erratic and starts blanking segs.
on it's own. This continues even on new projects without using a redis. pass (at least on my machine)

[dynamis]

sorry, stereo, i might have done something wrong earlier. only 2.67.00.23, 2.67.00.27 and 2.50 seemed to work for me.

but your way sounds best for now.

[stereo]

No prob, I wasn't going to try these versions; they're simply too old (I used to use 2.67.00.23 a lot).

But thanks for the heads up.

[Carpo]

now i always used to use cceaqm and AVAMAT6/7 depending on the bitrate untill someone told me that cceaqm is not compatible with some players, always had grat results mind you (still use those maticies though and still have great results :) ) , but can we have a deffinitive answer as to wether or not we use the redist option with cce, im guessing it works fine with HC just not with cce, i have tried a fair few discs and all looks fine in the redist file, although i am using avisynth 2.5.7 with no filters - although that shouldnt make a difference

[Sharc]

The discussion on the CCE bug seems to be stuck.
I wonder if there is no workaround such as to trap the unusual exit of CCE (is this possible?), increase the Q by 1 and try again until CCE exits normally.
In the few cases I have seen CCE aborting, increasing the Q has always helped.

[Boulder]

Pretty much the only thing that can be done is to check the number of frames in the encoded m2v file and compare it against the expected number of frames. If they differ, increase Q and try again.

[Sharc]

I thought there might be a flag or something set by CCE indicating that CCE terminated abnormally. Maybe it's not that simple, unfortunately.

[Carpo]

i have a few backups to do and am wondering if its safe to use redist or what

[archaeo]

i have a few backups to do and am wondering if its safe to use redist or what

As jdobbs mentioned in an earlier post:

Of course on the other hand, if you don't feed it matrices with exceptionally low values on encodes that are using bitrates in need of stronger matrices it may not happen in either case.


So carpo to answer your question - yes - when using the standard matrix with CCE as a 'failsafe' method. The problems seem to come when using a custom matrix with too low of values, as I chose in my apocalypto backup. Generally speaking, you may be taking some chances if you use a custom mtx with CCE when using bitrate redistribution.

[Boulder]

If you use HC, you are perfectly safe :)

[Carpo]

i did do a test with hc and the results were visually apparent, guess i'll just have to use the encoder default with redist on i suppose, shame as AVAMAT6/7 do normally give good resualts

[Boulder]

You could first do the redistribution with the standard matrix and then use RB-Opt to change it to whatever you want. The redistribution doesn't depend much on the matrix as can be seen in some tests in the RB-Opt thread.

[Voodoochild]

Hi
I've done my first Movie with the redistributed feature.
1. I'm very pleased with the results, Many Many Thanks for the new feature.
2. I've noticed in some segments that the bitrate drops below 2000, example 2,923 Kbs ----> 1,825 Kbs. I looked at the result and it was dark scene, so it came out good, but on the edge...

My question is, wouldn't it be wise in those few cases to take some bitrate from high scene let's say above 5000 and give a bit to those scenes below 2000 manually through the DVD-RB segment viewer, just to be on the safe side, that the scene won't be blocky ?

10x
Elad

[blutach]

Perhaps use the segment editor for that Voodoo or do a quick encode using RME?

Regards

[Sharc]

@Voodoochild
Just for my curiosity: Which encoder / matrix did you use?

[Voodoochild]

@Voodoochild
Just for my curiosity: Which encoder / matrix did you use?

I used the default cce Matrix for all segments with 25% sample
cce sp2.
I will try now with avama6, using the technique of adding a bit more to those below 2000, to see if three any difference in that scene. I will also check the scene I took some bitrate, to see weather the quality declined, though I don't think it will, as long as it stays above 80% compression.

Perhaps use the segment editor for that Voodoo or do a quick encode using RME?

Regards

Currently the last RME doesn't work.. I did sent Rockas the problem in the mail.

10x Elad.

[Voodoochild]

After testing again same DVD, with added bitrate in scene that were below 2000 br to 2500 and taking from scene that were above 80% compression and from end credits (no point having end credits with br above 2000) . I didn't see any differences at all while looking at scenes that demanded high br and the one that I added br to 2500 from 1800.
Still just to be on the safe side I prefer to add bitrate to those scenes below 2000.

10x
Elad

[jdobbs]

MHO: It's a waste to add bits to a section the doesn't need it. The redistribution gives you what it takes to keep the quality current. So if you have one section that has 5000Kbs in use and another that has 1800 -- it is telling you that you are going to get the same quality out of the 1800 (on that segment -- because of lack of demand) that 5000 would give you (in the other -- because of high demand).

The result of manually changing it: The 1800 would be at a higher quality level than the rest of the stream, and the 5000 would be lower.

Demand means a lot. In the example after the manual change, the segment that was changes from 5000 to (say) 4800 would acually have lower quality than the one that changed from 1800 to say 2000

[Sharc]

I second that. I did similar tests with scenes that got bitrates as low as approx. 1100 kbps.
As you have been using the default matrix there was little risk that CCE didn't do the redistribution properly - that has actually been my concern.

[Voodoochild]

......
Demand means a lot. In the example after the manual change, the segment that was changes from 5000 to (say) 4800 would acually have lower quality than the one that changed from 1800 to say 2000

I see your point jdobbs and logically it's true, but I've noticed in allot of my backups that when a movie is above 80% compression I can't tell the difference from the original at all.
So I thought if I have a scene that has 5000 and the segment viewer tells me it's 86%, for me it means that I can take 4% off for example and still won't tell the difference from the source. By giving those 4% to a very low bitrate scene that has below 2000 I can maybe avoid the tiny chance that this scene will suffer luck of BR. I only did it to 1 scene and couldn't tell the difference from the original 1800 BR. So in my case what you're saying is true.


10x
Elad

[Carpo]

to those that are getting low Q values what version of avisynth are you using ? in a few tests i did using 2.5.6a and 2.5.7 - i found that low Q values were given out more under 2.5.7 that 2.5.6a.

One film under 2.5.6a gave Q38 - and under 2.5.7 Q24 was given, this was repeated using a number of films (all PAL) and using the encoder default matrix

[Boulder]

Would you mind uploading a small sample of the same frames that have been encoded using Avisynth 2.5.6a and 2.5.7 using the exact same Q? Some people have reported sizing issues with 2.5.7 but no one has provided samples to show what really is happening there.

[jdobbs]

to those that are getting low Q values what version of avisynth are you using ? in a few tests i did using 2.5.6a and 2.5.7 - i found that low Q values were given out more under 2.5.7 that 2.5.6a.

One film under 2.5.6a gave Q38 - and under 2.5.7 Q24 was given, this was repeated using a number of films (all PAL) and using the encoder default matrix
This is the Q used for redistribution? If so, I'm not sure how that is possible -- as it is determined by the size of the source and the total frames. AVISYNTH isn't a player in the determination of the Base_Q.

[Sharc]

@carpo:
Are you on CCE SP or CCE SP Trial?
Did you in DVD-RB select both "Enable bitrate redistribution" (Mode menu) and "One pass VBR (w/analysis)" (Settings menu) at the same time? This would involve Avisynth for producing a sample file for OPV - still, it wouldn't explain why you get different values for Q.
If you want to do redistribution, only the "Enable bitrate redistribution" should be selected, the "One pass VBR" should not be selected.

[Carpo]

This is the Q used for redistribution? If so, I'm not sure how that is possible -- as it is determined by the size of the source and the total frames. AVISYNTH isn't a player in the determination of the Base_Q.

yes that is correct - same film same - same settings - diff avs version - due to my isp monitoring upload stream i doubt i can upload - although i know this would help they have a wired AUP - they seem to think if you upload more than a few hundred meg you must be a heavy user and cap you, maybe this is an error close to home as i do need to re-install soon - maybe then things might sort themselfs out

[Carpo]

@carpo:
Are you on CCE SP or CCE SP Trial?
Did you in DVD-RB select both "Enable bitrate redistribution" (Mode menu) and "One pass VBR (w/analysis)" (Settings menu) at the same time? This would involve Avisynth for producing a sample file for OPV - still, it wouldn't explain why you get different values for Q.
If you want to do redistribution, only the "Enable bitrate redistribution" should be selected, the "One pass VBR" should not be selected.


see above - same settings same dvd source, same everything apart from avisynth version, well apart from defraged drive ;)


cce sp 2.70.02 - and droping back to avisynth 2.56.a has also yeied a speed boost

[Options]
redistribute=1

is the only thing that is changed

have done with and without over two diff versions of avisynth

[Boulder]

although i know this would help they have a wired AUP - they seem to think if you upload more than a few hundred meg you must be a heavy user and cap you, maybe this is an error close to home as i do need to re-install soon - maybe then things might sort themselfs outYou don't need to UL hundreds of megabytes, even 10-15 frames might do which would mean a couple of megabytes - if there is a difference between the two Avisynth versions, it should be observable in pretty much every frame.

[Carpo]

has anyone had cce freeze when doing the redist pass ? i have had it happen on 2 segmants on a test disc of mine - cpu hits 50% on one core mem is about 256meg, i notice is does this when the gop changes from open to close, if i kill cce on the segmant, it pops back up doing the next one in the list

[Sharc]

I never had CCE freezing. I had CCE silently aborting with high bitrate matrices (like FHE) and low Q's - apparently a CCE bug as discussed previously in this thread. I did however never see CCE aborting with the Q calculated by RB-Opt ("final Q"). Perhaps just by coincidence, and I was just lucky.

[Carpo]

this isnt aborting it sits there at 66% on the two segmants and thats it, mem stays steady at 260meg and its using 50% cpu time, have reripped the disc, ran it thru fixvts, trying dvdshrink on it now, and will give the drives a stern checking just incase, but seeing as i have done 5 other dvds without issue im not overly convinced that its a drive issue

[Sharc]

......The redistribution doesn't depend much on the matrix as can be seen in some tests in the RB-Opt thread.
... only - at least for CCE - as long as Q is selected for equal target sizes (means "final_Q" for DVD-5). However, this "final_Q" depends largely on the selected matrix, for example Q=75 for FHE, Q=34 for CCE default, Q=18 for Avamat6, to name a few. If this applies only for CCE and if this is related to the CCE bug - I don't know. Anyway, this "final_Q" is obtained by RB-Opt and yields distribution profiles which are in fact almost independent from the particular matrix. Just my experience from many tests.

[Voodoochild]

Hi All.
I'm trying to back up A Sopranos DVD, which has 4 episodes.
Compression level is quite High 58%. I thought this is classic for redistribution, so I did prepare phase cce sp2, 20% sample, default matrix, q=43. When looking at the results, I noticed that I have 4 peeks; each one is the 1.35 minuets of the beginning trailer. Wouldn't it be wiser since it's only the beginning trailer, not the episode it self to lower those high value back and give more to the rest of the segments that are actually the episode it self?

attach a graph of the redistributed vs orginal


10x Elad

[Boulder]

A change request to the HC redistribution : use the GOP options that are selected for the encode process. Now it seems to use 12-2 but I'd much rather use autoGOP with HC.

[tom942]

I've been doing several tests with HC and I would like to know what others do with LUMGAIN.

If I use LUMGAIN=4 in prepare and encode phase, I can see little squares or blocks in dark scenes and also in the clear ones.

Then I set both phases LUMGAIN=0, and the image improves a lot in both cases. In the clear ones, I can´t notice any blocks but in the dark ones, I can still see blocks (less than before).

Would it be a good idea to do the prepare phase with LUMGAIN=0 and later, the encode phase, with LUMGAIN with 1-4?

I ask this because I've seen that depending if you've got LUMGAIN OFF or ON, the bitrate distribution changes, and then I don´t if I could be doing a wrong thing :confused:

[jdobbs]

I personally set LUMGAIN to 2 -- I think setting it to 4 is probably too much for most situations. If you do "2" it does a good job of preventing blocking in dark transition scenes.

[laserfan]

I'm trying to back up A Sopranos DVD, which has 4 episodes...Wouldn't it be wiser since it's only the beginning trailer, not the episode it self to lower those high value back and give more to the rest of the segments that are actually the episode it self?I made a lot of Sopranos backups prior to the RB Redistribution feature. The opening sequence looks like absolute crap in the 4 episode discs, but in all cases the main show looks just fine.

Probably the BEST solution is to edit-out the openings altogether, and give all the bits to the rest of the episode(s). That opening challenges the process beyond anything in the actual shows themselves.

If you like the openings it's prolly best to just leave BR off.

[blutach]

Hi voodoo,

If you're talking about the bit with the "Woke up this morning" song at the start which is the same in all eps, I have a different idea if you wanna keep it.

Given all eps are in the same VTS, change the VCID of the segment in eps 2, 3 and 4 to match the VCID in ep 1. The excised VCIDs will become unreferenced material, which can be eliminated with VobBlanker (I expect DVDRB would also eliminate it).

The only thing that you would notice is a lack of seamlessness in eps 2, 3 and 4 (the layer break flag should be on) - big deal if you can save 5 minutes of challenging video.

If you wanna pursue this, the easiest way is to do it in PgcEdit. Would take no more than a minute.

Regards

[tom942]

@jdobbs

Thank you. I'll give it a try.

Do you use LUMGAIN=2 always, or depends of the source?. And do you use the "encoder default" matrix or use other matrices that compress less?.

Could it be other idea to set LUMGAIN=0 and set CPU=4 in MPEG2SOURCE_OPTS?

[Voodoochild]

Hi voodoo,

If you're talking about the bit with the "Woke up this morning" song at the start which is the same in all eps, I have a different idea if you wanna keep it.

Given all eps are in the same VTS, change the VCID of the segment in eps 2, 3 and 4 to match the VCID in ep 1. The excised VCIDs will become unreferenced material, which can be eliminated with VobBlanker (I expect DVDRB would also eliminate it).

The only thing that you would notice is a lack of seamlessness in eps 2, 3 and 4 (the layer break flag should be on) - big deal if you can save 5 minutes of challenging video.

If you wanna pursue this, the easiest way is to do it in PgcEdit. Would take no more than a minute.

Regards

Excellent idea blutach, I will give it a try,
P.S, in which program do you mean by saying "the layer break flag should be on".. ?

10x Allot
Elad

[jdobbs]

@jdobbs

Thank you. I'll give it a try.

Do you use LUMGAIN=2 always, or depends of the source?. And do you use the "encoder default" matrix or use other matrices that compress less?.

Could it be other idea to set LUMGAIN=0 and set CPU=4 in MPEG2SOURCE_OPTS?I use the default matrices on almost everything. I change the matrix only when I have a specific reason for doing so.

[blutach]

Voodoo - see your PMs. The cell flag will need to register the non-seamless nature of the cell and the discontinuity in the system time clock for eps 2, 3 and 4 (flag value = 2). PgcEdit should do this for you.

Regards

[laserfan]

If you're talking about the bit with the "Woke up this morning" song at the start which is the same in all eps, I have a different idea if you wanna keep it.That IS an interesting idea blutach, but note the lead-in is NOT the same--the titles change to reflect actor appearances and (at least) writing and directing credits for each episode as well. At least one episode I recall too that the "lead-in" did not start at the beginning.

I do find the opening credits interesting, particularly when I see for example that Michael Imperioli has written the episode we're about to watch. We also like the DVDs' "last week on...", the written synopsis of the episode we're about to see, and then afterward the "next time on...". I know others that just pound thru the episodes without looking at ANY of that stuff...! To each his own, there is no accounting for taste, YMMV, etc. etc. The great thing is that we have all these tools to do what we want!

[Voodoochild]

... great thing is that we have all these tools to do what we want! ...


Amen for that :)

[blutach]

For mine, if you can save 5 minutes of video, and the tradeoff is some bloke's name in the writing credits, I'd save the video (particularly if, as voodoo as said, compression is high and presumably quant is too). But each to their own as you say.

Regards

[laserfan]

OTOH I even like that opening HBO logo with the dramatic music and the noisy video screen--but boy does that wreak havoc on my Sigma-based media players (can't show it without stuttering)!!!!

Now I know how to manipulate or edit-out these things, but for me "what's done is done" ie. at some point you have to decide when to stop messing with it and just sit and enjoy it... :)

[Carpo]

are people getting their dvds on target when using redist? of the five i have done 2 were 4.32gb 2 were 4.34gb and 1 is 4.30gb - is this normal or it depend on the film

[jdobbs]

Actully mine are pretty much coming in at 4.33 -- which is very, very close to the specified size. I have seen a couple that are smaller, but usually when the reduction is fairly small.

[Insomniak4700]

Has anybody so far tried using CCEAQM=1 in the rebuilder.ini file and use redistribution at the same time? (I set the redist percent at 100)

My eyes tell me the results are quite good, but I'd like to know if anybody actually did any tests, with data?

[Fishman0919]

I use CCEAQM=1 with all my encodings with CCE SP/SP2.
No problems at all.

[archaeo]

I use CCEAQM=1 with all my encodings with CCE SP/SP2.
No problems at all.



Same here, w/no problems. Sizing has been on target, too.

[jdobbs]

I've been using it (on-and-off) also with no problems.

[Carpo]

hmm i was always told not to use that setting as some players dont like it - although i have done one or two and the cheap (at the time) £50 player i bought parents plays them ok and the pc does - should i reconsider this option ? and wouldnt this option effect the redist ? or is it totally safe to use CCEAQM=1 and redist ?

[Fishman0919]

as some players dont like it

Some 1st gen DVD Player (Sony DVPS-3000/7000 some problems and Pioneer DV-09 same if will play DVD+/-R... Pioneer DV-05 seems fine but has issues playing DVD+/-R's sometimes) and a few 2nd Gen Players.

[jdobbs]

Custom Technologies warns in their documentation that it may be incompatible with some players -- that's why I add that warning when we talk about it... but there are a lot of people using it, and not many who have run into problems. I thnk Fishman0919 might be right -- it's probably something that only affects old players.

[Carpo]

one i bought my parents is old and not a top brand make and it seems to plays the ones i did with the ccq setting on - but to the other Q how safe is it with redist option, suppose i could do a few dvds and see :)

[jdobbs]

It works fine with redistribution.

[tom942]

With Bitrate Distribution enabled, has it got sense to set the bias as I ask in this thread?

http://forum.doom9.org/showthread.php?t=128884

[Insomniak4700]

Thanks for your opinions everyone, I did a couple of discs with both CCEAQM and redist, both my players seem to play them back fine, and sizing was spot on.

[jamewoong]

Did anyone notice a huge difference when enable the bitrate redistribution?

Between, high or low Base_Q is better?

Thanks.

EDIT:
This feature is really ugly... Very disappointed with the result.

Without redistribution:
552 mins - quality ok.

With redistribution:
786 mins - quality is okay, but sometime, blocky image come up. Also, when the image is dark, it will make it worst. When the image is clear (light), the quality is better.

Using Procoder

[Carpo]

have you tried doing the same encode but using HC or CCE and seeing if it still looks bad ? it could be procoder having a fit were as other encoders may be ok

[jdobbs]

Did anyone notice a huge difference when enable the bitrate redistribution?

Between, high or low Base_Q is better?

Thanks.

EDIT:
This feature is really ugly... Very disappointed with the result.

Without redistribution:
552 mins - quality ok.

With redistribution:
786 mins - quality is okay, but sometime, blocky image come up. Also, when the image is dark, it will make it worst. When the image is clear (light), the quality is better.

Using ProcoderWas the blocky output on a very small segment (like less than 500 frames)? I noticed that on one I did where there was a fade-in to an announcement. It was caused by the fact that only the blackness (before the fade-in) was used as the sample. I've made some adjustments for the next release to prevent that.

As I've said before, this feature is something that needs to be tested and proven. While I haven't seen negative results (beyond what I just mentioned), I have seen positive on several discs.

...786 mins...That's a lot of extra time for doing a sampling??? You didn't change anything else?

[Sharc]

Was the blocky output on a very small segment (like less than 500 frames)? I noticed that on one I did where there was a fade-in to an announcement. It was caused by the fact that only the blackness (before the fade-in) was used as the sample.

I have seen this recently on a short segment with few seconds blackness (voices only) followed by a short flash of few motion picture frames, then again blackness with voices for few seconds etc => The sample retained mainly the blackness and the redistributed bitrate was set too low.
I would suggest to use 400... 500 frames as the minimum for redistribution. If the segment is shorter than these 400 ...500 frames all frames of the segment would be used. In my case this has solved the problem as the short flashes of motion pictures were fully included in the sample.

[jdobbs]

Yep. That's what I've done. If it is 500 frames or fewer I just do a 100% sample.

[jamewoong]

Was the blocky output on a very small segment (like less than 500 frames)? I noticed that on one I did where there was a fade-in to an announcement. It was caused by the fact that only the blackness (before the fade-in) was used as the sample. I've made some adjustments for the next release to prevent that.

The problem occurs on black areas and during a FADE-IN. Don't know how many frames, but it's less 3 secs during the fade-in. For the black area, it always mess up.

That's a lot of extra time for doing a sampling??? You didn't change anything else?
Well, I navigate through this thread and as you said, people give good comments, so I trust them and BAAM, I got the bad result after encoding a DVD with this feature. Normally, I don't add any filters as I don't have any knowledge on it. So, no extra setting.

[Carpo]

nevermind

[jamewoong]

Well, this is a problem when I enable the redistribution:
http://www.mediafire.com/?d1yz1dn9igg

It give some quality to other part, but it destroy some part too.

And this one occurs 1/50 times (really rare case):
http://www.mediafire.com/?dwhmcuuojg4

I'm not sure, but I think the redistribution was enable. And no, the original DVD was not scratched or anything.

[Sharc]

File 1:
Average bitrate of 1 Mbps in this short test sequence is low. Is the same scene without redistribution ok? Can you post the redistribution.txt file?
Which encoder did you use for redistribution and for encoding?

File2:
Looks like a bad rip or poor burn/media problem to me.

[jamewoong]

File 1:
Average bitrate of 1 Mbps in this short test sequence is low. Is the same scene without redistribution ok? Can you post the redistribution.txt file?
Which encoder did you use for redistribution and for encoding?

File2:
Looks like a bad rip or poor burn/media problem to me.
Well, it's a serie and I encoded it few weeks ago with Procoder, so everything (info) is deleted. I think this problem could be solved if I've got the txt info, but... D*mn it, why did I delete it?

Between, each episode was reduced from 1400MB to 880MB. It's a lot, but if you compare the quality of the good image with the broken one, then this is not normal. The other part is okay, but this one is messup... I only remember that the Q base was 48. Don't know if it helps...

Anyway, I notice it when burnt it to DVD, then watched it on TV as it's impossible to test it on PC because I'll lose my fun about watching that serie.

FILE 2: It was like that after the encoding. The original source is perfect. It could be a problem from the redistribution...

[Sharc]

.....so everything (info) is deleted.....
Difficult to help then.
You could however quickly redo the re-distribution in 3-click mode - without encoding - and post the redistribution.txt. That doesn't take much time.
You may want to redo everything with HC - means redistribution and encoding - and set lumgain to 3.
If the problematic scenes have been in a very small segment, you may also select a higher sample size instead of the default 10%.
Actually I can't comment much on the combo HC (for redistribution) with Procoder (for the encoding) because I have no experience with it.

[Dashiell]

I think I have found quite the match for the redist feature.

National Geographic's "Inside 9/11" disc one. granted there is nearly 4 hours of content on this disc, but I noticed an extremely low bitrate and blocky picture during program #2, about an hour in...

It is a frenetically edited program, with a multitude of flash cuts and various types of source material... too numerous to even try and mention. there were entire sections of programming that were under 1000k and extremely blocky. It would then shift to different types of video and clear up.

I was using version 1.26.2 of rebuilder, CCE Basic and the HC Redist feature. Perhaps this disc can be considered a "tough case scenario" for testing?

[Boulder]

Out of interest, what does a non-redistributed backup look like?

[Dashiell]

Out of interest, what does a non-redistributed backup look like?

Good question, and I have not tried as of yet. I may try it tonight and see what happens.

[jdobbs]

And the original.

[Dashiell]

Hey, J.

I have watched the original many times in the past, no such conditions exist.

Now that I have the moment, I have one other question as well...

When using HC as the redist encoder for CCE Basic, should the ConvertToYUY2() feature be unchecked in the settings? I know that ultimately CCE will be doing the encoding, but I was just wondering if this would have an adverse affect on the redist with HC...

[jdobbs]

I wrote the code so it will automatically adjust for that during REDISTRIBUTION.

Any way you could post the REDISTRIBUTION.TXT file from the "Inside 9/11" D2VAVS directory? I'd like to see if there are som obvious problems. Also -- give it a try with v1.26.3. I made some improvements related to this.

[Dashiell]

I wrote the code so it will automatically adjust for that during REDISTRIBUTION.

Any way you could post the REDISTRIBUTION.TXT file from the "Inside 9/11" D2VAVS directory? I'd like to see if there are som obvious problems. Also -- give it a try with v1.26.3. I made some improvements related to this.

Ach! I was going to do that very thing and then realized I had deleted the directory. I have made a couple rebuilds since.

Let me know what would be the most helpful... shall I do a re-encode under the same conditions using the new ".3" version or a re-encode without using redist at all. If both, let me know which one should be "priority." I will then check the quality in the same areas and post the .txt file.

[jdobbs]

All you'd need to do is a PREPARE to get the REBUILDER.TXT file. If you do it from .3 though it could very possibly be different -- especially if .3 fixed it.

If you can compare the two REDISTRIBUTION tables it will help identify whether REDISTRIBUTION had anything to do with the blockiness (or if it was just too hard to encode in the space available).

[Dashiell]

I do remember on the orginal encode the specified bitrate under "typical" in the H/L/T area was only 1,001. Even at that point I found it odd. I don't know if this info helps. I regret now removing the temp dir.

[jdobbs]

Wow. If the typical bitrate was that low it would be very difficult to get a good encode out of it.

[rack04]

jdobbs, do I need to have "apply redistribution to all VTSs (series)" if all episodes are in the same VTS? For your reference I'm refering to Heroes Season 1.

[jdobbs]

No. If they are all in one VTS just leave it at the standard setting. I found the same thing with "The Sopranos".

[rack04]

Does something change in the AVS creation with redistribution enabled? The reason I ask is because when I try to open the avs files created by DVD Rebuilder in Media Player Classic all I get is mess.

[Dashiell]

All you'd need to do is a PREPARE to get the REBUILDER.TXT file. If you do it from .3 though it could very possibly be different -- especially if .3 fixed it.

If you can compare the two REDISTRIBUTION tables it will help identify whether REDISTRIBUTION had anything to do with the blockiness (or if it was just too hard to encode in the space available).

Sorry for the delay...


Here's the redistribution.txt file you were looking for. This is the .3 version redist. (I've also included the rebuilder.txt)

This is the NG "Inside/911" disc 1. The issue occurs frequently within VTS_04

[Boulder]

Does something change in the AVS creation with redistribution enabled? The reason I ask is because when I try to open the avs files created by DVD Rebuilder in Media Player Classic all I get is mess.Post the contents of the script in question.

[rack04]

Post the contents of the script in question.

Sorry I don't have access to the files at the moment. I will post them when I get off work.

[rack04]

Well I found of the problem is with Haali renderer and not DVD Rebuilder. VMR renderer works fine.

[Sharc]

If there are different matrices in the segments, RB-Opt uses the matrix of the "average segment" for the prediction, and then uses the actual matrices of every segment for the redistribution pass.
Anyway, I'm convinced that if you want to use redistribution, you should use the same matrix for the whole movie.

("average segment" is the segment whose bitrate is more similar to the average bitrate of the movie)

@robot1:
Could it be added as an option (e.g. with an entry in the RBOpt.ini) to use the matrix of the "average segment" for the redistribution pass, i.e. keep the matrix which was used for the prediction?

[robot1]

@robot1:
Could it be added as an option (e.g. with an entry in the RBOpt.ini) to use the matrix of the "average segment" for the redistribution pass, i.e. keep the matrix which was used for the prediction?
Sorry for the late reply. I hope to be clear in my poor english.
I think keeping the matrix used for the prediction, in this case, will bring at useless results.
Example: you have a Q factor = 30, calculated using the "average matrix" (let's say a medium-bitrate matrix).
If you have only one segment with a high-bitrate matrix, and you perform the redistribution pass with the medium-bitrate matrix, you will get a filesize lower than the one needed to have a quality similar to Q =30. In the actual encoding by DVD-RB you could get a bitrate equivalent to (let's say) a Q = 45: you could have pixellation.

I'm convinced that if you want to use redistribution, you should use the same matrix for the whole movie.

[Sharc]

Sorry for the late reply. I hope to be clear in my poor english.
I think keeping the matrix used for the prediction, in this case, will bring at useless results.
Example: you have a Q factor = 30, calculated using the "average matrix" (let's say a medium-bitrate matrix).
If you have only one segment with a high-bitrate matrix, and you perform the redistribution pass with the medium-bitrate matrix, you will get a filesize lower than the one needed to have a quality similar to Q =30. In the actual encoding by DVD-RB you could get a bitrate equivalent to (let's say) a Q = 45: you could have pixellation.
It's clear what you say. I have found that using the actual matrix - instead of the one that was used for the Q calculation - for the redistribution pass increases the swing: The positive peaks are getting higher and the negative peaks are getting lower - which in tendency exposes the low bitrate cells to the risk of macroblocking. In a way a chicken/egg situation.

After all I agree that using one and the same matrix for Q calculation, redistribution pass and encoding is a good and safe strategy.

[blutach]

Which is why limiting the szie of the swings of important IMHO.

Regards

[march2006]

I dont know how to use this option for PROCODER3 and CCE SP2 ENCODING

[tom942]

After all I agree that using one and the same matrix for Q calculation, redistribution pass and encoding is a good and safe strategy.

So, when we are using Bitrate Distribution, is it recommended to use only one matrix for the whole process (CQ calculation, redistribution and encoding) instead of using in the encoding phase different matrices than in the CQ calculation and redistribution?.

I mean, for instance, in "Main feature" use "encoder's default", "low bitrate" use "avamat6" and "very low bitrate" use "avamat7".

Actually, I use the first way (one matrix for all), but I would like to know from the experience and opinions of others in the forum.

Regards.

[dstalker]

If you use the preview/edit to change bitrates, you are overriding the decision made by redistribution.

It will reallocate the recovered space just like it always did -- but the new distribution will be used as the baseline. So the benefit of the redistribution is retained.

Yes, so you probably don't want to do that within the segments of the feature. But you could, after redistribution, reallocate bits from extras and they would be used in a way consistent with the redistribution. You could redistribute bits from something like the credits if you wanted to... I personally wouldn't, though.


Hello everyone.
After reading some of the posts I became confused concerning redistribution and segment editor.
To my understanding redist distributes bitrate among cells within VTS (thats why it is usually used just for main feature - to redistribute bitrate inside one VTS). So it should be possible to allocate different total space for individual VTSs and it would distribute that bitrate among cells (or segments) inside them.
But if I use it for all VTSs and then go to editor and change bitrate for extras, will it negate redist benefit?
According to second and third quotes that is not the case, but I don't understand how it retains those benefits. For example I have episodic disc so I use redist for all VTSs. There are some extras and I want to keep them but with lower bitrate then the series.
I can't use option to steal space from extras, because smaller series would be considered extras. Then again some extras are more usefull then others, i.e. I would assign less bitrate for trailers and more to music clips or other interesting extras. In this particular case trailers for some reason are automatically set to "no reencode" which results in them having higher average bitrate than any of the episode, while they are just good to have and quality is not as important.
So finally I would reduce extras and get more space for episodes. Question is if I should leave that extra space and DVD-RB will automatically increase bitrate for series (and not extras which I already reduced) proportionally to redistribution or should I manually increase bitrate for whole VTS (for each episode) and bitrate proportions inside it would be somehow maintained?
I tried to do the last, but when I lower reduction for whole VTS, reduction for individual segments inside it becomes even and the same as VTS, so I can't understant how the results of redist have any influence to final bitrate distribution. It looks the same as if I would adjust bitrate for individual segments inside VTS, and that was said to override decision made during redist.

http://img225.imageshack.us/img225/3021/rb1br7.jpg
Reduction level for VTS_04 after redistribution

http://img225.imageshack.us/img225/7366/rb11kw9.jpghttp://img216.imageshack.us/img216/1465/rb12te6.jpg
Segments inside VTS_04 have different reduction as it should be

http://img232.imageshack.us/img232/802/rb2ic0.jpg
VTS with extras is set to not reencode and gets higher bitrate than main features

http://img232.imageshack.us/img232/6793/rb3rv5.jpg
I reduce bitrate for whole VTS to ~60% to free up more space for main feature

http://img232.imageshack.us/img232/7688/rb4ck7.jpg
http://img232.imageshack.us/img232/8760/rb41oa9.jpghttp://img232.imageshack.us/img232/6684/rb42xg6.jpg
Then I change reduction for the whole VTS_04, and I would expect segments to have their bitrates increased proportionally to the values set by redistribution, but as you can see they are set to the same reduction value. And that makes their bitrates much more similar then it was before changing reduction for VTS.

So final question is - if I use editor not to blank something, but to change reduction for features of secondary importance in order to have more room for main features, how to keep the benefits of redistribution for these features (and maybe for secondary features if they contain several segments in one VTS)? If someone could explain the principle behind it, I would be grateful :)

[Sharc]

It looks as if the redistribution profile is flushed for VTS_04, i.e. whenever you change the reduction at the VTS_xx level, as all segments assume the same % reduction in proportion to the original.

You may want to give RB-Opt a try in your case.

[jdobbs]

1. If you reallocate the other VTSs (non-Feature) and then select "Allocate Saved Space to Feature", the feature will retain its redistribution.

2. If you select a reduction percentage at the VTS level (any VTS) -- you are instructing DVD-RB to reduce ALL SEGMENTS in that VTS to that level... so you are manually overriding any redistribution that may have previously occurred on that VTS.

[Sharc]

Thanks for clarification. May be this should go to the FAQ.

[dstalker]

Thanks for explaining. But now I'm wondering if "Allocate Saved Space to Feature" means to the main feature, aka biggest VTS? But then what about episodic DVDs?
It would be good to have some way of allocating that saved space to all VTSs that weren't reallocated manually, or maybe just to the selected ones, but I guess I'm asking too much :)

[jdobbs]

Yes, feature means the biggest VTS. There is no "feature" in an episodic disc... so you shouldn't use it.

[JohnGalt]

A stupid question: how do I change the percentage of video analyzed in the redistribution phase? Is it a hidden option in rebuilder.ini?

[Fishman0919]

Yes,

Redist_Percent=##

[JohnGalt]

great -- thanks!