RB-Opt v0.23 BETA a tool to change titles bitrate, CCE parameters and AVS scripts.

[Boulder]

Yes, the performance is optimal with SMP disabled and multiple simultaneous instances of the encoder.

[blutach]

Not always. For example, if the last segment is lengthy. This should be an **option** only, IMHO.

Regards

[Boulder]

Not always. For example, if the last segment is lengthy. This should be an **option** only, IMHO.In that case, true. What about always enabling SMP for the last 2-4 (depending on the number of detected CPUs) segments or so? Though that might be a bit too pedantic ;)

[blutach]

Can't do it on such a simple formula sad to say Boulder. Suppose it's the 2nd last segment that's the long one and the last is a little one (and 2 cores for simplicity). You want the CPU usage to jump up once it is free. The answer is enabling SMP. Also, if I am not mistaken, SMP possibly makes the redist'n pass quicker.

Regards

[Boulder]

But if you always enabled SMP for the last 2-4 segments, your case would be covered :)

Then again, the overall performance reduction shouldn't be big if SMP was always enabled. HC uses one thread for decoding and the other for MPEG2 encoding. If you have a simple script, most of the CPU time is spent in the MPEG2 encoding thread so running two instances with SMP enabled (and a simple script) means that two threads are utilized heavily and two threads are used very little. It should be close to running two instances with SMP disabled.

[blutach]

Yes. I've actually found very little gain with NOSMP (on DVDs where segments are "normal sized") and as discussed, some loss where there are largish segments near the end.

Regards

[mikenadia]

I have a DVD with a lot of small videos where only a small area of the frames is moving (the background is mostly static).
DVD_RB shows me a 55 % reduction level for DVD-5 and an avg bitrate of 3380 Kbs.
Because the background is mostly static, the optimal Q computed by Rb-opt is 18. I was wondering if the choice of the matrix should be based on optimal Q (and in this case, what about the fact that optimal Q is much higher for interlaced video : which is the case of this DVD ) instead of high-medium-low bitrate.
Thx for your advice.
P.S. May be the title of the thread should be changed (v.037).Thx,robot1 for the excellent program!

[Boulder]

I would use a high-bitrate matrix (such as the High-High from DVD-RB) myself. It looks like you might get encoder saturation with the current matrix selection.

[mikenadia]

Well, just MHO -- but I still think that if you have a CBR source the best you can do is a bitrate that is equal to the one used in CBR... otherwise you'd have to be getting the additional missing detail out of the cosmic ether. DVD-RB sets the max to the CBR rate because anything higher would probably only amplify the error (blockiness).

But, as I always say, too each his own.
Can we deduct from the quote that the new redist_BIAS should not be smaller than the reduction level computed by DVD-RB during the Prepare phase?
Thx

[jdobbs]

redist_BIAS?

[mikenadia]

Here is the new build:
RB-Opt v 0.37 (http://www.savefile.com/files/1341148)

Changelog:
Bugfix: fixed wrong OPV prediction for CCE SP2 and Pal 4/3 movies, as reported by ercofra.

Added more info in the redist.txt file after redistribution (with actual bitrates stored).

Added a new parameter in the .ini file: redist_BIAS (values: 0 - default redistribution; 100 - no redistribution)
The "redistribution_bias" parameter limits the min_cell_bitrate for example to 55% of the DVD-RB standard value, and the max_cell_bitrate to 1/55% = 182%
More examples
bias min_cell_bitrate max_cell_bitrate
0 0 (infinite) i.e. full redistribution
50 50% 200%
60 60% 167%
75 75% 133%
100 100% 100% ie same as DVD-RB "original" means no redistribution.

Probably a BIAS between 50 and 65 would be a good choice.

Anyway, with RB-Opt you can always tweak the redistribuited bitrates, without losing the benefit of redistribution for other cells (I often tweak the end credits cells after redistribution)
Redist_BIAS added in Rb-opt.037 ( divided by 100) compared to
- Reduction Level for DVD-5: in DVD-RB

[Sharc]

You can do the redistribution either with DVD-RB, or you may use RB-Opt for the same purpose. Both have a parameter in their respective .ini files which controls the swing of the redistribution.
In DVD-RB this is a hidden option, and the parameter is called "REDIST_LOW_THRESHOLD=" It's default value is 0.5 (means 50%).
RB-Opt uses a similar parameter "redist_BIAS=". The value there has to be given as a percentage number, for example "redist_BIAS=50" in the RBOpt.ini will basically have the same effect as "REDIST_LOW_THRESHOLD=0.5" in the rebuilder.ini.

Leaving the default value for DVD-RB (actually you don't have to set anything there) or set "redist_BIAS=50" in RB-Opt is recommended from many tests.

[mikenadia]

Thx, but I wanted to know if redist_BIAS should not be at least equal to the reduction level.
I have a DVD authored with a CBR (reduction level to fit DVD5 equal to 66 %). After running Rb-opt (with redistr_BIAS equal to 50), I have some bitrates (biased in redist.txt) that are higher than the actual (bitrate without redistribution in the compressed DVD5) by more than 51 %(1/0.66=1.51) and therefore are higher than the one in the uncompressed DVD9. (That is my understanding).
Thx again.

[mikenadia]

I have a VTS-VobID with the first cell (only 39 frames ) flagged as interlaced (according to the avs file of that cell ) and all the others ( for a total of more than 80 000 frames ) flagged as progressive.
It seems that during optimal Q computation, rb-opt is using HC with interlaced flag (because of the first cell).
Will it have an impact of optimal Q and if yes what can I do (including using DGindex)?
Can I assume ( probably not ) that the redistribution profile within that VTS-VobID will be the same wthether it is flagged interlaced or not (I mean the biased bitrate for each cell within that VobID)?

BTW, I would like to compute optimal Q with DC prec=8 (if that is what I will use for the encoding).
Will it have an impact on optimal Q and how do I change it (probably should put HC_AutoDC=0 in rb-opt.ini but what else?)
Thx.
P.S. I have the interlaced flag during optimal Q computation (in HC encoder settings) even for VTS-VobID that have all cells progressive for the same DVD.
Is it what Boulder described in post 801 or not related at all?

[jdobbs]

Don't put too much weight on that Q value. It's nice to get close, but the distribution really doesn't drift that much unless you have dramatic changes in Q.

[Dashiell]

Hello, all!

Is there any line I can add/alter on the .ini file that will turn the "Natural Filter" in CCE Basic ON for all encodes?

Yes, yes I know this can be done with RB-Opt but I'd like to be able to switch it on for 1-click mode as well.


Thanks, all!

[Sharc]

Thx, but I wanted to know if redist_BIAS should not be at least equal to the reduction level.
I have a DVD authored with a CBR (reduction level to fit DVD5 equal to 66 %). After running Rb-opt (with redistr_BIAS equal to 50), I have some bitrates (biased in redist.txt) that are higher than the actual (bitrate without redistribution in the compressed DVD5) by more than 51 %(1/0.66=1.51) and therefore are higher than the one in the uncompressed DVD9. (That is my understanding).
Thx again.
The redist_BIAS does not depend on the reduction level. It only sets the lowest and highest acceptable cell bitrate. One can leave the redist_BIAS at 50 - irrespective of the reduction level.
For example when the CBR of your original disc is say 6000 kbps it will become 66% of 6000 = 3960 kbps (CBR) to fit onto a DVD5.
Setting redist_Bias=50 means that after redistribution the lowest redistributed cell bitrate will never become less than 50% of 3960 kbps = 1980 kbps (low limit), and the highest cell bitrate will not exceed 1/0.5 = 2 x 3960 kbps = 7920 kbps (upper limit).
If redistribution would actually lift a cell to the said 7920 kbps, it will - in a final step of the redistribution process - be reduced to the original 6000 kbps, following the rule that the redistributed cell bitrate must not become higher than the bitrate of the original uncompressed DVD.

[mikenadia]

Thx a lot for your explanation. I was not aware of the "final step". It is possible that when you have more than 1 cell that needs that final step, some are rounded, due to "Autosize". In one example, I have 3 cells with Reduction set to 99.4% 99.6% and 100%.
May be the cells above 99% should be forced to 100% and avoid reencoding.

[jdobbs]

Hello, all!

Is there any line I can add/alter on the .ini file that will turn the "Natural Filter" in CCE Basic ON for all encodes?

Yes, yes I know this can be done with RB-Opt but I'd like to be able to switch it on for 1-click mode as well.


Thanks, all!I'll add it to the menus.

[robot1]

Download here (http://www.savefile.com/files/1400505) or here: RB-Opt v0.38 beta (http://forum.gleitz.info/attachment.php?attachmentid=91918&d=1203877329) (mirror by Doom9.de)


Changelog:
- Bugfix: fixed a wrong parameter in OPV prediction for CCE SP2 (thanks to ercofra for pointing it out)
- Added *PROGRESSIVE statement in HC.ini for progressive sources (HC 0.22 compatible)
- Added *NOSMP statement in HC.ini if multiple processes used (the last segments are econded in multiprocess mode)
- Set to 15% the maximum Sample Size for prediction (1-3% suggested)

If you use multiple processes, SMP mode in HC is used when
frames_current_segment > all_frames_to_encode/(nr_process-1)

Usually the last 2-4 segments (according to the number of threads used) are encoded in SMP mode, but this should cover also strange cases (as a very long segment followed by many small segments)
Regards

[Boulder]

Thanks, will fire up a new project right away :)

[Dashiell]

I'll add it to the menus.

Thanks, J! Fantastic program, btw!

[blutach]

Just a feature request (for a lazy old man).

How about the ability to drag and drop the Rebuilder.inf file onto the RBOpt GUI to open it?

Regards

[mikenadia]

Just a concept. I found out that on some DVDs the Q for the Extras (using a low bitrate matrix) was similar or smaller than the one for the main movie (using MPEG standard).
Therefore the idea of setting a minimum Q for the Extras, compute bitrate of Extras for that Q and that low bitrate matrix and if the result is smaller than the "Autosized" one, reallocate automatically bitrate to the main movie.( A maximum Q could suggest a lower bitrate matrix).
Similarly, a minimum bitrate for the main movie could indicate a higher bitrate matrix if necessary.

[Sharc]

Interesting proposal - basically what you are suggesting is a compressability test for the Extras in order to determine the minimum acceptable bitrate/file size automatically. The problem I see is the selection of the "minimum Q", as it depends on the selected matrix itself and the source (sort of chicken and egg situation). For example a "high Q/high bitrate matrix" combo often produce visually better results than a "low Q/low bitrate" combo for the same file size, or in other words a low bitrate matrix for the extras will normally require a low Q at the same time for acceptable visual results. Your proposal may work best for encoders which follow a "constant quality" strategy rather than a "constant quantization" strategy though. Interlaced/progressive material in the Extras may be another difficulty for selecting the appropriate Q. Just some thoughts. Did you already do some verification tests?

[mikenadia]

I have very bad visual judgment (and only a 26" TV) so I cannot make the difference between HighQ-High bitrate combo and the LowQ-Low bitrate one. I have made the previous comment because I am choosing the bitrates and the matrixes on a random basis and I rely on the forum members to set up a methodology ( as you say, may be different matrixes for interlaced and progressive or having a multiplier to make a relationship between constant quality and constant quantizer: optimalQ=average( Q for progressive and alpha x Q for interlaced) alpha determined by visual tests on an average of different sources and users).
Also (and I do not know how) take into consideration encoder saturation, blocking .
If you think it may have some traction, may be start a new thread.
Thx.

Edit: on a unrelated subject, because I am always preprocessing the rip with VobBlanker, I split (for fun) every non-negligible cell in 2. Got a smoother (smaller difference in bitrate between 2 consecutive segments) distribution . If there is any negative consequence of doing it , please let me know (other than the risk of preprocessing, which I am assuming).
Before splitting:6674 frames. bitrate 2926
Same cell after splitting: part1:2975 frames . bitrate 2362
part2:3699 frames . bitrate 2942
The missing bitrates were transferred to another area of the movie ( distribution made over 36 segments instead of 18).

[Sharc]

As I said, I personally find your proposal interesting. Today, the settings for Extras are based on file size (0%,10%,25%,33%,50% steal space from extras -- what is the right choice for acceptable quality?). Your approach is Q (quant or quality) based. I have often seen that the titles in the extras are inconsistent regarding quality and bitrate. That's why I sometimes do a special redistribution pass using RB-Opt for the Extras, just for equalizing their quality. Your approach would also help in this respect.
At the end, such feature needs to be implemented -- by probably robot1 and/or jdobbs. If they are convinced that it is worth to be followed up their motivation may also - to some extent - depend on donations ....... ;)

[jdobbs]

You can always set "Apply Distribution to all VTSs" and then the entire disc (including extras) is adjusted to a single Q.

[Sharc]

Yes, clear. But this has sometimes the tendency to 'steal bits from the feature' rather than to accept a minimum Quality (set by the Q) for the Extras and give the saved space to the feature.
But I agree that this may had helped in mikenadia's case where he/she found equal or lower Q's for the Extras - although I am not sure because the bitrate distribution for the Extras would have been done with the default matrix (DVD-RB).

[jdobbs]

You've underscored exactly the problem with selecting a Q for extras. If yoiu set it you have to live with it -- even if it steals from the feature. The problem is that you don't know which way it is going until after you've set it...

[mikenadia]

I was not thinking of any change neither in DVD-RB nor Rb-opt, but notwithstanding the valid issues raised by Sharc in post 827 (cf proposed solution in post #837), a process similar to:
- compute optimal Q for Extras and save.
- If optimal Q below arbitrary minimum Q, make an educated guess for Extras bitrates that will make optimal Q close to minimum Q.
- Input those new bitrates in Extras and compute optimal Q for Main movie.

[Sharc]

Well, without modifying RB-Opt or DVD-RB you can do this manually today.
- After the Prepare you run a separate redistribution pass with RB-Opt on the Extras only, using your selected matrix.
- If you find the calculated Q for the Extras to be too low (means you assume (!) you could have reduced the Extras more and still get acceptable quality), you can reallocate bitrates manually between the Extras and the feature (using RB-Opt and Save or using the DVD-RB Editor and Save)
- or you could rerun the standard Prepare (few minutes) selecting a higher value for "steal space from Extras" and repeat the process .........

When I want to keep some Extras, I normally select Steal Space from Extras 25% or 33%@Half D1 for the Prepare. This reduction for the Extras will in very most cases not lead to unexpected low Q's - even with low bitrate matrices selected.
After the Prepare I inspect the bitrates of the extras and then I decide if I want to run a separate redistribution pass on the Extras only (with RB-Opt) in order to "balance" their quality.

[mikenadia]

1) Talking about decreasing bitrate in endcredits cell. If I do it before redistribution, I have to do it after redistribution (there may be a conceptual reason but do not know why) and the autosize function modifies the optimal distribution (did the following test. Optimal Q of the main movie with an ABR of 3800.Save.Take some bits from the extras so the ABR of the main movie is 4000. Take a note of the bitrate distribution of the main movie. Do optimal Q. The new distribution is different ( for some cells , by more than 2 %).
2) by splitting the cells of the main movie in 2, I got a very different (and supposedly more efficient if the concept of OPV is working) bitrate distribution.
At least, on a conceptual basis, splitting the avs script in N scripts using Trim functions and then join the sub-cells M2v into a cell M2v (in a non-transparent process) may be closer to constant quality (if the OPV concept is working).

P.S. Will be nice to have a SSIM for the main movie so we can measure the gains of the OPV concept.

[Sharc]

1) When going for redistribution, the tweaking of cell bitrates before the redistribution pass does not make sense as the cell bitrates will be overwritten by the redistribution process. So if you want to reduce the end credits, do it after the redistribution pass.
Don't worry about a 2% discrepancy. You won't notice a difference in viewing.
2) Your new cells are different from the original ones (half the number of frames per cell) so the redistribution will produce different results based on your new cell structure. If for example the frames in cell 1 of your split cell contains a lot of details or action and cell 2 is a black screen, cell 1 will get all the bits allocated. Good luck with your cell granularity tests.

[Boulder]

Hi robot1,

are you planning on adding support for the new AQ feature of HC023? Or possibly add support for overriding the ini file to cover any later changes/additional features in HC023?

[robot1]

Yes, I will add support for AQ feature, and will have an ini option for custom changes for HC.

[Dashiell]

Hello, all!

Is there any line I can add/alter on the .ini file that will turn the "Natural Filter" in CCE Basic ON for all encodes?

Yes, yes I know this can be done with RB-Opt but I'd like to be able to switch it on for 1-click mode as well.


Thanks, all!

I'll add it to the menus.

Not to be a pain, but was this ever implemented?


Thanks in advance!:thanks:

[J-Wo]

How important is the sample size during redistribution? I notice that DVD-RB defaults to 10%, but RB-Opt does 20%

[blutach]

Given faster machines these days, doing 20% still only takes a short time. I use that.

Regards

[Boulder]

I use 100% - the difference in time needed to go through the phase is very small compared to the total time of the encode/rebuild process on my E6750.

[Devilman1]

Can somebody upload the latest version?
Is not available anymore.

TIA

[manolito]

No problem...

https://www.sendspace.com/file/u3ao2o

Cheers
manolito

[Devilman1]

Thank you